[MDEV-19016] Table in InnoDB data dictionary contains invalid flags upon undo-upgrade from 10.1.8 Created: 2019-03-21  Updated: 2019-03-22  Resolved: 2019-03-22

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB, Storage Engine - XtraDB
Affects Version/s: 10.2, 10.3, 10.4
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Marko Mäkelä
Resolution: Not a Bug Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MDEV-6075 Allow > 16K pages on InnoDB Closed
Relates
relates to MDEV-9040 10.1.8 fails after upgrade from 10.0.21 Closed

 Description   

10.3 117291db

 
2019-03-22  0:36:36 4 [ERROR] InnoDB: Table `test`.`t016` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=35 SYS_TABLES.N_COLS=2147483660
 
2019-03-22  0:36:36 1 [ERROR] InnoDB: Table `test`.`t018` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=39 SYS_TABLES.N_COLS=2147483660
 
2019-03-22  0:36:36 3 [ERROR] InnoDB: Table `test`.`t007` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=37 SYS_TABLES.N_COLS=2147483660
 
2019-03-22  0:36:36 3 [ERROR] InnoDB: Table `test`.`t008` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=39 SYS_TABLES.N_COLS=2147483660
 
2019-03-22  0:36:36 1 [ERROR] InnoDB: Table `test`.`t008` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=39 SYS_TABLES.N_COLS=2147483660
 
2019-03-22  0:36:36 3 [Warning] InnoDB: adjusting FSP_SPACE_FLAGS of file './test/t013.ibd' from 0xc000 to 0x180
 
2019-03-22  0:36:36 1 [ERROR] InnoDB: Table `test`.`t019` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=41 SYS_TABLES.N_COLS=2147483660
 
2019-03-22  0:36:36 1 [ERROR] InnoDB: Table `test`.`t007` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=37 SYS_TABLES.N_COLS=2147483660
 
...

mysql_upgrade fails saying that tables don't exist in the engine.

Possibly such upgrade from 10.1.8 is not supported (as it was a fairly broken version), but then maybe InnoDB could just say so and refuse to start?

To reproduce:

 Comments   
Comment by Marko Mäkelä [ 2019-03-22 ]

The SYS_TABLES.N_COLS value 2147483660 is 1<<31|8, meaning that the ROW_FORMAT is not REDUNDANT, and the number of user-defined stored columns is 8.

There was a problem related to SYS_TABLES.TYPE in MariaDB between versions 10.2.2 and 10.2.6, which was fixed in MDEV-12873.
Let us decode the distinct SYS_TABLES.TYPE values:

SYS_TABLES.TYPE interpretation
0x23 (35) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=1
0x25 (37) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2
0x27 (39) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4
0x29 (41) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8

The reason why these are invalid is that ROW_FORMAT=COMPRESSED only supports innodb_page_size up to 16k, but the dataset uses innodb_page_size=32k. In ROW_FORMAT=COMPRESSED, the page directory at the end of the compressed page reserves the 2 most significant bits of the 16-bit byte offsets for flags, hence limiting innodb_page_size to at most 16k.

The creation of invalid ROW_FORMAT=COMPRESSED tables was prevented in MariaDB 10.1.9 as part of fixing MDEV-9040. The bug was introduced by MDEV-6075 in MariaDB 10.1.0, which allowed innodb_page_size=32k and innodb_page_size=64k.

Note: there were other corruption bugs related to the larger page sizes. Examples include MDEV-11828 and MDEV-13227, which were not fixed until 10.1.26 and 10.2.8.

Comment by Marko Mäkelä [ 2019-03-22 ]

This is a GIGO (garbage in, garbage out) situation. I would say that such unconstrained upgrade testing is not feasible before 10.1.9, and with innodb_page_size=64k not before 10.1.26 or 10.2.8.

Generated at Thu Feb 08 08:48:26 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.