Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-25852

Orphan #sql*.ibd files are occasionally left behind after killed ALTER TABLE

    XMLWordPrintable

Details

    Description

      Even with the third fix related to MDEV-25506 (to never delete .ibd files before the DDL operation has been committed), we still have occasional result difference of the following test, which is based on innodb_fts.crash_recovery,release:

      --source include/have_innodb.inc
      --source include/not_embedded.inc
      FLUSH TABLES;
       
      --connect(ddl1, localhost, root,,)
      CREATE TABLE t1(a TEXT,b TEXT,INDEX(a(5))) ENGINE=InnoDB;
      send ALTER TABLE t1 ADD INDEX(b(5));
      --connect(ddl2, localhost, root,,)
      CREATE TABLE t2(a TEXT,b TEXT,INDEX(a(5))) ENGINE=InnoDB;
      send ALTER TABLE t2 DROP INDEX a, ADD INDEX(b(5)), FORCE;
      --connect(ddl3, localhost, root,,)
      CREATE TABLE t3(a TEXT,b TEXT,INDEX(a(5))) ENGINE=InnoDB;
      send ALTER TABLE t3 DROP INDEX a, ADD INDEX(b(5)), ALGORITHM=COPY;
       
      --connection default
      let $shutdown_timeout=0;
      --source include/shutdown_mysqld.inc
      --exec tar czf $datadir/../dd.tar.gz -C $datadir .
      --source include/start_mysqld.inc
       
      disconnect ddl1;
      disconnect ddl2;
      disconnect ddl3;
       
      DROP TABLE t1,t2,t3;
       
      SELECT * FROM information_schema.innodb_sys_tables WHERE name LIKE 'test/%';
      let $datadir=`select @@datadir`;
      list_files $datadir/test *.ibd;
      

      Occasionally, the list_files will list a file #sql-ib*.ibd or #sql-backup-*.ibd. If that last line is commented out, the test will continue, and the discrepancy will be reported by check-testcase (which does not count as a failure for mtr). Here are a couple of examples:

      --- /dev/shm/10.6/mysql-test/var/13/tmp/check-mysqld_1.result	2021-06-03 16:38:44.877394351 +0300
      +++ /dev/shm/10.6/mysql-test/var/13/tmp/check-mysqld_1.reject	2021-06-03 16:38:45.473401026 +0300
      @@ -1097,3 +1097,4 @@
       partition	1.0	DISABLED	STORAGE ENGINE	100602.0	NULL	NULL	Mikael Ronstrom, MySQL AB	Partition Storage Engine Helper	GPL	OFF	Stable	1.0
       VARIABLE_NAME	VARIABLE_VALUE
       DEBUG_SYNC	ON - current signal: ''
      +#sql-ib168.ibd
       
      mysqltest: Result length mismatch
      

      and

      --- /dev/shm/10.6/mysql-test/var/25/tmp/check-mysqld_1.result	2021-06-03 16:38:31.953249791 +0300
      +++ /dev/shm/10.6/mysql-test/var/25/tmp/check-mysqld_1.reject	2021-06-03 16:38:32.509256004 +0300
      @@ -1097,3 +1097,4 @@
       partition	1.0	DISABLED	STORAGE ENGINE	100602.0	NULL	NULL	Mikael Ronstrom, MySQL AB	Partition Storage Engine Helper	GPL	OFF	Stable	1.0
       VARIABLE_NAME	VARIABLE_VALUE
       DEBUG_SYNC	ON - current signal: ''
      +#sql-backup-387caf-a.ibd
       
      mysqltest: Result length mismatch
      

      Nothing else is failing. Attached is a copy of a data directory where we end up with an orphan file. The following patch would fix that, but as collateral damage, we would fail in a different way, by prematurely deleting a file t2.ibd during the test:

      innodb_fts.cr 'innodb'                   w2 [ fail ]  Found warnings/errors in server log file!
              Test ended at 2021-06-03 17:53:36
      line
      2021-06-03 17:53:36 3 [ERROR] InnoDB: Operating system error number 2 in a file operation.
      2021-06-03 17:53:36 3 [ERROR] InnoDB: The error means the system cannot find the path specified.
      2021-06-03 17:53:36 3 [ERROR] InnoDB: File ./test/t2.ibd: 'delete' returned OS error 71.
      

      Here is my incorrect patch:

      diff --git a/storage/innobase/log/log0recv.cc b/storage/innobase/log/log0recv.cc
      index 12ec2e29bfb..21f9ea86acc 100644
      --- a/storage/innobase/log/log0recv.cc
      +++ b/storage/innobase/log/log0recv.cc
      @@ -597,6 +597,8 @@ static struct
           lsn_t lsn;
           /** File name from the FILE_ record */
           std::string file_name;
      +    /** whether a FILE_DELETE record was encountered */
      +    mutable bool deleted;
         };
       
         using map= std::map<const uint32_t, item, std::less<const uint32_t>,
      @@ -638,7 +640,7 @@ static struct
       
           char *fil_path= fil_make_filepath(nullptr, {filename, strlen(filename)},
                                             IBD, false);
      -    const item defer= {lsn, fil_path};
      +    const item defer= {lsn, fil_path, false};
           auto p= defers.emplace(space, defer);
           if (!p.second && p.first->second.lsn <= defer.lsn)
             p.first->second= defer;
      @@ -1034,9 +1036,10 @@ fil_name_process(char* name, ulint len, ulint space_id,
       
       	if (deleted) {
       		/* Got FILE_DELETE */
      -
      -		deferred_spaces.remove(
      -			static_cast<uint32_t>(space_id));
      +		if (auto d = deferred_spaces.find(static_cast<uint32_t>(
      +							  space_id))) {
      +			d->deleted = true;
      +		}
       		if (!p.second && f.status != file_name_t::DELETED) {
       			f.status = file_name_t::DELETED;
       			if (f.space != NULL) {
      

      Attachments

        1. data_copy.tar.bz2
          957 kB
          Marko Mäkelä
        2. dd.tar.gz
          625 kB
          Marko Mäkelä
        3. dd-t2.tar.gz
          588 kB
          Marko Mäkelä

        Issue Links

          Activity

            People

              thiru Thirunarayanan Balathandayuthapani
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.