Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20320

Tablespace flags mismatch for full_crc32 format

Details

    Description

      Crash-recovery on 10.4 from crc32 to full_crc32 or vice versa frequently brings the error messages in the error log:

      10.4-enterprise 90fc8f11

      2019-08-05 19:22:52 0 [Note] InnoDB: Waiting for purge to start
      2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc035` in the cache. Attempting to load the tablespace with space id 39
      2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc011` in the cache. Attempting to load the tablespace with space id 15
      2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc031` in the cache. Attempting to load the tablespace with space id 35
      2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc020` in the cache. Attempting to load the tablespace with space id 24
      2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc032` in the cache. Attempting to load the tablespace with space id 36
      2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc010` in the cache. Attempting to load the tablespace with space id 14
      2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc024` in the cache. Attempting to load the tablespace with space id 28
      2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc022` in the cache. Attempting to load the tablespace with space id 26
      2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc036` in the cache. Attempting to load the tablespace with space id 40
      2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc004` in the cache. Attempting to load the tablespace with space id 8
      2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc007` in the cache. Attempting to load the tablespace with space id 11
      2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc030` in the cache. Attempting to load the tablespace with space id 34
      2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc033` in the cache. Attempting to load the tablespace with space id 37
      2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc028` in the cache. Attempting to load the tablespace with space id 32
      2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc017` in the cache. Attempting to load the tablespace with space id 21
      2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc006` in the cache. Attempting to load the tablespace with space id 10
      2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc008` in the cache. Attempting to load the tablespace with space id 12
      2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc034` in the cache. Attempting to load the tablespace with space id 38
      2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc019` in the cache. Attempting to load the tablespace with space id 23
      2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc021` in the cache. Attempting to load the tablespace with space id 25
      2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc018` in the cache. Attempting to load the tablespace with space id 22
      2019-08-05 19:22:52 0 [Note] InnoDB: 10.4.6 started; log sequence number 21412559; transaction id 66470
      

      As discussed with thiru, it is most likely a bogus error, and I indeed don't see any obvious problem with these tables (can SELECT, can ALTER, and the errors are only shown ones, don't re-appear after restart); but if the error message doesn't mean anything, it should be removed.

      To reproduce, if needed:

      git clone https://github.com/MariaDB/randgen --branch mdev20320 rqg-mdev20320
      cd rqg-mdev20320
      perl ./runall-new.pl  --basedir=<your basedir> --threads=1 --grammar=conf/mariadb/oltp-transactional.yy --gendata=conf/mariadb/innodb_upgrade_compression.zz --mysqld=--innodb-compression-algorithm=none --mysqld1=--innodb_checksum_algorithm=crc32 --mysqld2=--innodb_checksum_algorithm=full_crc32 --scenario=CrashUpgrade --duration=100 --vardir=/dev/shm/vardir
      

      Attachments

        Issue Links

          Activity

            elenst Elena Stepanova added a comment - - edited

            Another, scarier-looking variation of probably the same problem:

            2019-08-12  0:11:28 0 [ERROR] InnoDB: Refusing to load './test/tc001.ibd' (id=5, flags=0x15); dictionary contains id=5, flags=0x10021
            2019-08-12  0:11:28 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
            2019-08-12  0:11:28 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
            2019-08-12  0:11:28 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not create them.
            2019-08-12  0:11:28 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``test`.`tc001``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue.
            

            To reproduce:

            git clone https://github.com/MariaDB/randgen --branch mdev20320 rqg-mdev20320
            cd rqg-mdev20320
            perl ./runall-trials.pl --basedir=<your basedir> --vardir=<your vardir> --trials=5 --threads=4 --grammar=conf/mariadb/oltp-transactional.yy  --gendata=mdev20320.zz --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--innodb-lock-wait-timeout=5 --mysqld=--innodb-compression-algorithm=none --scenario=UndoLogUpgrade --duration=300 --mysqld1=--innodb_checksum_algorithm=crc32 --mysqld2=--innodb-checksum-algorithm=full_crc32
            

            It can take a while, as the command above runs the test 5 times (till the first failure) every run is ~4 min, and it doesn't always happen on the 1st attempt.

            elenst Elena Stepanova added a comment - - edited Another, scarier-looking variation of probably the same problem: 2019-08-12 0:11:28 0 [ERROR] InnoDB: Refusing to load './test/tc001.ibd' (id=5, flags=0x15); dictionary contains id=5, flags=0x10021 2019-08-12 0:11:28 0 [ERROR] InnoDB: Operating system error number 2 in a file operation. 2019-08-12 0:11:28 0 [ERROR] InnoDB: The error means the system cannot find the path specified. 2019-08-12 0:11:28 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not create them. 2019-08-12 0:11:28 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``test`.`tc001``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue. To reproduce: git clone https://github.com/MariaDB/randgen --branch mdev20320 rqg-mdev20320 cd rqg-mdev20320 perl ./runall-trials.pl --basedir=<your basedir> --vardir=<your vardir> --trials=5 --threads=4 --grammar=conf/mariadb/oltp-transactional.yy --gendata=mdev20320.zz --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--innodb-lock-wait-timeout=5 --mysqld=--innodb-compression-algorithm=none --scenario=UndoLogUpgrade --duration=300 --mysqld1=--innodb_checksum_algorithm=crc32 --mysqld2=--innodb-checksum-algorithm=full_crc32 It can take a while, as the command above runs the test 5 times (till the first failure) every run is ~4 min, and it doesn't always happen on the 1st attempt.

            Patch is in bb-10.4-MDEV-20320

            thiru Thirunarayanan Balathandayuthapani added a comment - Patch is in bb-10.4- MDEV-20320

            The value of innodb_compression_algorithm at PAGE_COMPRESSED table creation is stored within an innodb_checksum_algorithm=full_crc32 file. We must tolerate any difference in the algorithm setting when opening a full_crc32 file.

            A test case that would show the server error message after server restart would fail to trigger mtr failure. Because of that, I wrote a test case that uses IMPORT.

            marko Marko Mäkelä added a comment - The value of innodb_compression_algorithm at PAGE_COMPRESSED table creation is stored within an innodb_checksum_algorithm=full_crc32 file. We must tolerate any difference in the algorithm setting when opening a full_crc32 file. A test case that would show the server error message after server restart would fail to trigger mtr failure. Because of that, I wrote a test case that uses IMPORT .

            My enhanced test for IMPORT fails like this:

            10.4 13f740904a389f03d3c075fee0b91ecfcc0de60b

            CURRENT_TEST: innodb.full_crc32_import
            mysqltest: At line 168: query 'ALTER TABLE t1 IMPORT TABLESPACE' failed: 1296: Got error 39 'Data structure corruption' from ./test/t1.ibd
            

            marko Marko Mäkelä added a comment - My enhanced test for IMPORT fails like this: 10.4 13f740904a389f03d3c075fee0b91ecfcc0de60b CURRENT_TEST: innodb.full_crc32_import mysqltest: At line 168: query 'ALTER TABLE t1 IMPORT TABLESPACE' failed: 1296: Got error 39 'Data structure corruption' from ./test/t1.ibd

            People

              thiru Thirunarayanan Balathandayuthapani
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.