[MDEV-19016] Table in InnoDB data dictionary contains invalid flags upon undo-upgrade from 10.1.8 - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Not a Bug
Affects Version/s: 10.2(EOL), 10.3(EOL), 10.4(EOL)
Fix Version/s: N/A
Component/s: Storage Engine - InnoDB, Storage Engine - XtraDB
Labels:
None

Description

10.3 117291db
2019-03-22 0:36:36 4 [ERROR] InnoDB: Table `test`.`t016` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=35 SYS_TABLES.N_COLS=2147483660
2019-03-22 0:36:36 1 [ERROR] InnoDB: Table `test`.`t018` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=39 SYS_TABLES.N_COLS=2147483660
2019-03-22 0:36:36 3 [ERROR] InnoDB: Table `test`.`t007` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=37 SYS_TABLES.N_COLS=2147483660
2019-03-22 0:36:36 3 [ERROR] InnoDB: Table `test`.`t008` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=39 SYS_TABLES.N_COLS=2147483660
2019-03-22 0:36:36 1 [ERROR] InnoDB: Table `test`.`t008` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=39 SYS_TABLES.N_COLS=2147483660
2019-03-22 0:36:36 3 [Warning] InnoDB: adjusting FSP_SPACE_FLAGS of file './test/t013.ibd' from 0xc000 to 0x180
2019-03-22 0:36:36 1 [ERROR] InnoDB: Table `test`.`t019` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=41 SYS_TABLES.N_COLS=2147483660
2019-03-22 0:36:36 1 [ERROR] InnoDB: Table `test`.`t007` in InnoDB data dictionary contains invalid flags. SYS_TABLES.TYPE=37 SYS_TABLES.N_COLS=2147483660
...

mysql_upgrade fails saying that tables don't exist in the engine.

Possibly such upgrade from 10.1.8 is not supported (as it was a fairly broken version), but then maybe InnoDB could just say so and refuse to start?

To reproduce:

download ftp://perro.askmonty.org/public/innodb_upgrade_data/10.1.8/format-Barracuda/innodb-builtin/32K/compression-none/encryption-off/undo.tar.gz
unpack (it creates data dir);

start the new server with

--innodb-page-size=32K --innodb-compression-algorithm=none

observe the failures

Attachments

Issue Links

is caused by

MDEV-6075 Allow > 16K pages on InnoDB

Closed

relates to

MDEV-9040 10.1.8 fails after upgrade from 10.0.21

Closed

Activity

Ascending order - Click to sort in descending order

Marko Mäkelä added a comment - 2019-03-22 07:24

The SYS_TABLES.N_COLS value 2147483660 is 1<<31|8, meaning that the ROW_FORMAT is not REDUNDANT, and the number of user-defined stored columns is 8.

There was a problem related to SYS_TABLES.TYPE in MariaDB between versions 10.2.2 and 10.2.6, which was fixed in ~~MDEV-12873~~.
Let us decode the distinct SYS_TABLES.TYPE values:

SYS_TABLES.TYPE	interpretation
0x23 (35)	ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=1
0x25 (37)	ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2
0x27 (39)	ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4
0x29 (41)	ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8

The reason why these are invalid is that ROW_FORMAT=COMPRESSED only supports innodb_page_size up to 16k, but the dataset uses innodb_page_size=32k. In ROW_FORMAT=COMPRESSED, the page directory at the end of the compressed page reserves the 2 most significant bits of the 16-bit byte offsets for flags, hence limiting innodb_page_size to at most 16k.

The creation of invalid ROW_FORMAT=COMPRESSED tables was prevented in MariaDB 10.1.9 as part of fixing ~~MDEV-9040~~. The bug was introduced by ~~MDEV-6075~~ in MariaDB 10.1.0, which allowed innodb_page_size=32k and innodb_page_size=64k.

Note: there were other corruption bugs related to the larger page sizes. Examples include ~~MDEV-11828~~ and ~~MDEV-13227~~, which were not fixed until 10.1.26 and 10.2.8.

Marko Mäkelä added a comment - 2019-03-22 07:24 The SYS_TABLES.N_COLS value 2147483660 is 1<<31|8, meaning that the ROW_FORMAT is not REDUNDANT, and the number of user-defined stored columns is 8. There was a problem related to SYS_TABLES.TYPE in MariaDB between versions 10.2.2 and 10.2.6, which was fixed in MDEV-12873 . Let us decode the distinct SYS_TABLES.TYPE values: SYS_TABLES.TYPE interpretation 0x23 (35) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=1 0x25 (37) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 0x27 (39) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=4 0x29 (41) ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8 The reason why these are invalid is that ROW_FORMAT=COMPRESSED only supports innodb_page_size up to 16k , but the dataset uses innodb_page_size=32k . In ROW_FORMAT=COMPRESSED , the page directory at the end of the compressed page reserves the 2 most significant bits of the 16-bit byte offsets for flags, hence limiting innodb_page_size to at most 16k. The creation of invalid ROW_FORMAT=COMPRESSED tables was prevented in MariaDB 10.1.9 as part of fixing MDEV-9040 . The bug was introduced by MDEV-6075 in MariaDB 10.1.0 , which allowed innodb_page_size=32k and innodb_page_size=64k . Note: there were other corruption bugs related to the larger page sizes. Examples include MDEV-11828 and MDEV-13227 , which were not fixed until 10.1.26 and 10.2.8.

Marko Mäkelä added a comment - 2019-03-22 07:27

This is a GIGO (garbage in, garbage out) situation. I would say that such unconstrained upgrade testing is not feasible before 10.1.9, and with innodb_page_size=64k not before 10.1.26 or 10.2.8.

Marko Mäkelä added a comment - 2019-03-22 07:27 This is a GIGO (garbage in, garbage out) situation. I would say that such unconstrained upgrade testing is not feasible before 10.1.9, and with innodb_page_size=64k not before 10.1.26 or 10.2.8.

People

Assignee:: Marko Mäkelä

Reporter:: Elena Stepanova

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 2019-03-21 22:49

Updated:: 2019-03-22 07:27

Resolved:: 2019-03-22 07:27

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server