Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17158

TRUNCATE is not atomic after MDEV-13564

Details

    Description

      I observed this both on 10.2 and 10.3 when testing MDEV-13564. The following test would occasionally fail on a cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_ASAN=1 build:

      ./mtr --parallel=40 --repeat=100 --mysqld=--debug=d,ib_log innodb.truncate_crash{,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,}

      The reason for this appears to be that the following call does not always guarantee a write to the redo log:

      log_write_up_to(mtr.commit_lsn(), true);

      Such a call is present in all callers of fil_name_write_rename_low(), yet sometimes we end up with losing the table, because no undo log was persisted for the incomplete transaction despite this call.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13564 TRUNCATE TABLE and undo tablespace truncation are not compatible with Mariabackup

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-14717 RENAME TABLE in InnoDB is not crash-safe

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            elenst Elena Stepanova added a comment -

            The test fails for me like this:

            innodb.truncate_crash 'innodb'           w4 [ 52 fail ]
            		 
                    Test ended at 2018-09-07 23:06:13
            		 
             
            		 
            CURRENT_TEST: innodb.truncate_crash
            		 
            mysqltest: At line 20: query 'SELECT * FROM t1' failed: 1932: Table 'test.t1' doesn't exist in engine

            If it's not an indication of the reported problem, then could you please paste the output which does indicate it?

            elenst Elena Stepanova added a comment - The test fails for me like this: innodb.truncate_crash 'innodb' w4 [ 52 fail ] Test ended at 2018-09-07 23:06:13   CURRENT_TEST: innodb.truncate_crash mysqltest: At line 20: query 'SELECT * FROM t1' failed: 1932: Table 'test.t1' doesn't exist in engine If it's not an indication of the reported problem, then could you please paste the output which does indicate it?
            elenst Elena Stepanova made changes -
            Component/s Tests [ 10800 ]
            marko Marko Mäkelä added a comment -

            I made a wrong diagnosis. log_write_up_to() appears to behave as it should. The actual problem is in MDEV-13564: ha_innobase::truncate() will invoke innobase_rename_table(), which will commit the transaction prematurely. Because of this, the original table would be persistently renamed to a temporary name before the new empty table has been created.

            marko Marko Mäkelä added a comment - I made a wrong diagnosis. log_write_up_to() appears to behave as it should. The actual problem is in MDEV-13564 : ha_innobase::truncate() will invoke innobase_rename_table() , which will commit the transaction prematurely. Because of this, the original table would be persistently renamed to a temporary name before the new empty table has been created.
            marko Marko Mäkelä made changes -
            Labels corruption recovery upstream recovery
            marko Marko Mäkelä made changes -
            Affects Version/s 10.3.10 [ 23140 ]
            Affects Version/s 10.2.2 [ 22013 ]
            Affects Version/s 10.3.0 [ 22127 ]
            marko Marko Mäkelä made changes -
            Priority Critical [ 2 ] Blocker [ 1 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.3 [ 22126 ]
            Fix Version/s 10.2 [ 14601 ]
            marko Marko Mäkelä made changes -
            Summary log_write_up_to() sometimes fails TRUNCATE is not atomic after MDEV-13564
            marko Marko Mäkelä made changes -
            Status Confirmed [ 10101 ] In Progress [ 3 ]
            marko Marko Mäkelä added a comment -

            It turned out that there were multiple InnoDB transaction commits inside ha_innobase::truncate() before the completion of ha_innobase::create(). Also the crash recovery had to be revised a bit, because the new empty file has to be deleted before the old non-empty table can be renamed back to the old file.

            marko Marko Mäkelä added a comment - It turned out that there were multiple InnoDB transaction commits inside ha_innobase::truncate() before the completion of ha_innobase::create() . Also the crash recovery had to be revised a bit, because the new empty file has to be deleted before the old non-empty table can be renamed back to the old file.
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2018-09-10 13:22:56.0 2018-09-10 13:22:56.38
            marko Marko Mäkelä made changes -
            Fix Version/s 10.3.10 [ 23140 ]
            Fix Version/s 10.3 [ 22126 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            Fix Version/s 10.2.19 [ 23207 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 89347 ] MariaDB v4 [ 154908 ]

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.