[MDEV-20320] Tablespace flags mismatch for full_crc32 format Created: 2019-08-05  Updated: 2019-09-10  Resolved: 2019-09-10

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.4
Fix Version/s: 10.4.9

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Thirunarayanan Balathandayuthapani
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-19534 Make innodb_checksum_algorithm=full_c... Closed

 Description   

Crash-recovery on 10.4 from crc32 to full_crc32 or vice versa frequently brings the error messages in the error log:

10.4-enterprise 90fc8f11

 
2019-08-05 19:22:52 0 [Note] InnoDB: Waiting for purge to start
 
2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc035` in the cache. Attempting to load the tablespace with space id 39
 
2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc011` in the cache. Attempting to load the tablespace with space id 15
 
2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc031` in the cache. Attempting to load the tablespace with space id 35
 
2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc020` in the cache. Attempting to load the tablespace with space id 24
 
2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc032` in the cache. Attempting to load the tablespace with space id 36
 
2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc010` in the cache. Attempting to load the tablespace with space id 14
 
2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc024` in the cache. Attempting to load the tablespace with space id 28
 
2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc022` in the cache. Attempting to load the tablespace with space id 26
 
2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc036` in the cache. Attempting to load the tablespace with space id 40
 
2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc004` in the cache. Attempting to load the tablespace with space id 8
 
2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc007` in the cache. Attempting to load the tablespace with space id 11
 
2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc030` in the cache. Attempting to load the tablespace with space id 34
 
2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc033` in the cache. Attempting to load the tablespace with space id 37
 
2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc028` in the cache. Attempting to load the tablespace with space id 32
 
2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc017` in the cache. Attempting to load the tablespace with space id 21
 
2019-08-05 19:22:52 3 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc006` in the cache. Attempting to load the tablespace with space id 10
 
2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc008` in the cache. Attempting to load the tablespace with space id 12
 
2019-08-05 19:22:52 2 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc034` in the cache. Attempting to load the tablespace with space id 38
 
2019-08-05 19:22:52 1 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc019` in the cache. Attempting to load the tablespace with space id 23
 
2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc021` in the cache. Attempting to load the tablespace with space id 25
 
2019-08-05 19:22:52 4 [ERROR] InnoDB: Failed to find tablespace for table `test`.`tc018` in the cache. Attempting to load the tablespace with space id 22
 
2019-08-05 19:22:52 0 [Note] InnoDB: 10.4.6 started; log sequence number 21412559; transaction id 66470

As discussed with thiru, it is most likely a bogus error, and I indeed don't see any obvious problem with these tables (can SELECT, can ALTER, and the errors are only shown ones, don't re-appear after restart); but if the error message doesn't mean anything, it should be removed.

To reproduce, if needed:

git clone https://github.com/MariaDB/randgen --branch mdev20320 rqg-mdev20320
 
cd rqg-mdev20320
 
perl ./runall-new.pl  --basedir=<your basedir> --threads=1 --grammar=conf/mariadb/oltp-transactional.yy --gendata=conf/mariadb/innodb_upgrade_compression.zz --mysqld=--innodb-compression-algorithm=none --mysqld1=--innodb_checksum_algorithm=crc32 --mysqld2=--innodb_checksum_algorithm=full_crc32 --scenario=CrashUpgrade --duration=100 --vardir=/dev/shm/vardir



 Comments   
Comment by Elena Stepanova [ 2019-08-11 ]

Another, scarier-looking variation of probably the same problem:

2019-08-12  0:11:28 0 [ERROR] InnoDB: Refusing to load './test/tc001.ibd' (id=5, flags=0x15); dictionary contains id=5, flags=0x10021
 
2019-08-12  0:11:28 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
 
2019-08-12  0:11:28 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
 
2019-08-12  0:11:28 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not create them.
 
2019-08-12  0:11:28 0 [ERROR] InnoDB: Could not find a valid tablespace file for ``test`.`tc001``. Please refer to https://mariadb.com/kb/en/innodb-data-dictionary-troubleshooting/ for how to resolve the issue.

To reproduce:

git clone https://github.com/MariaDB/randgen --branch mdev20320 rqg-mdev20320
 
cd rqg-mdev20320
 
perl ./runall-trials.pl --basedir=<your basedir> --vardir=<your vardir> --trials=5 --threads=4 --grammar=conf/mariadb/oltp-transactional.yy  --gendata=mdev20320.zz --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--innodb-lock-wait-timeout=5 --mysqld=--innodb-compression-algorithm=none --scenario=UndoLogUpgrade --duration=300 --mysqld1=--innodb_checksum_algorithm=crc32 --mysqld2=--innodb-checksum-algorithm=full_crc32

It can take a while, as the command above runs the test 5 times (till the first failure) every run is ~4 min, and it doesn't always happen on the 1st attempt.

Comment by Thirunarayanan Balathandayuthapani [ 2019-09-10 ]

Patch is in bb-10.4-MDEV-20320

Comment by Marko Mäkelä [ 2019-09-10 ]

The value of innodb_compression_algorithm at PAGE_COMPRESSED table creation is stored within an innodb_checksum_algorithm=full_crc32 file. We must tolerate any difference in the algorithm setting when opening a full_crc32 file.

A test case that would show the server error message after server restart would fail to trigger mtr failure. Because of that, I wrote a test case that uses IMPORT.

Comment by Marko Mäkelä [ 2019-09-10 ]

My enhanced test for IMPORT fails like this:

10.4 13f740904a389f03d3c075fee0b91ecfcc0de60b

 
CURRENT_TEST: innodb.full_crc32_import
 
mysqltest: At line 168: query 'ALTER TABLE t1 IMPORT TABLESPACE' failed: 1296: Got error 39 'Data structure corruption' from ./test/t1.ibd

Generated at Thu Feb 08 08:58:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.