[MDEV-11757] KEY_BLOCK_SIZE strangeness when UNCOMPRESSing COMPRESSed InnoDB tables Created: 2017-01-10  Updated: 2021-08-18  Resolved: 2017-02-28

Status: Closed
Project: MariaDB Server
Component/s: Data Definition - Alter Table
Affects Version/s: 10.1.18, 10.0.28, 10.2
Fix Version/s: 10.0.30, 10.1.22, 10.2.5

Type: Bug Priority: Major
Reporter: Chris Calender (Inactive) Assignee: Sergei Golubchik
Resolution: Fixed Votes: 2
Labels: upstream

Issue Links:
Relates
relates to MDEV-12152 KEY_BLOCK_SIZE strangeness in ALTER T... Open
relates to MDEV-26400 ALTER TABLE does not remove KEY_BLOCK... Stalled
Sprint: 10.0.30

 Description   

Summary: If you uncompress a compressed InnoDB table, that you originally specified KEY_BLOCK_SIZE, you cannot eliminate KEY_BLOCK_SIZE from the table's structure. (It may indeed be ignored, but you cannot remove it, unles you re-create the table.)

Say you create the following compressed table:

CREATE TABLE `t1` (
`id1` bigint(20) NOT NULL,
`id2` bigint(20) NOT NULL,
PRIMARY KEY (`id1`),
UNIQUE KEY `id2` (`id2`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8;

ALTER TABLE t1 ROW_FORMAT=DYNAMIC;

This throws a warning:

mysql> ALTER TABLE t1 ROW_FORMAT=DYNAMIC;
Query OK, 0 rows affected, 1 warning (0.21 sec)
Records: 0  Duplicates: 0  Warnings: 1

mysql> SHOW WARNINGS;
+---------+------+-----------------------------------------------------------------+
| Level   | Code | Message                                                         |
+---------+------+-----------------------------------------------------------------+
| Warning | 1478 | InnoDB: ignoring KEY_BLOCK_SIZE=8 unless ROW_FORMAT=COMPRESSED. |
+---------+------+-----------------------------------------------------------------+
1 row in set (0.00 sec)

So now try the following instead to eliminate the warning (start with fresh `t1`):

ALTER TABLE t1 ROW_FORMAT=DYNAMIC KEY_BLOCK_SIZE=0;

It runs fine, with no warnings:

mysql> ALTER TABLE t1 ROW_FORMAT=DYNAMIC KEY_BLOCK_SIZE=0;
Query OK, 0 rows affected (0.11 sec)
Records: 0  Duplicates: 0  Warnings: 0

But, now look at the SHOW CREATE TABLE output:

mysql> SHOW CREATE TABLE `t1`\G
*************************** 1. row ***************************
       Table: t1
Create Table: CREATE TABLE `t1` (
  `id1` bigint(20) NOT NULL,
  `id2` bigint(20) NOT NULL,
  PRIMARY KEY (`id1`) KEY_BLOCK_SIZE=8,
  UNIQUE KEY `id2` (`id2`) KEY_BLOCK_SIZE=8
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC
1 row in set (0.00 sec)

This removed KEY_BLOCK_SIZE from the end of the table structure, but instead it now added KEY_BLOCK_SIZE=8 to both KEY definitions.

So no matter what you do when uncompressing the table, it seems to retain the original KEY_BLOCK_SIZE somewhere, and I cannot seem to eliminate it.



 Comments   
Comment by Elena Stepanova [ 2017-01-11 ]

Reproducible on MySQL 5.7.

Comment by Marko Mäkelä [ 2017-01-11 ]

This is a known issue, ‘submarine attributes’. A similar thing happens with ALTER TABLE…ENGINE when some table attributes that are stored in the .frm file are ignored by certain storage engines. An example could be

CREATE TABLE t (a INT) ENGINE=MyISAM ROW_FORMAT=FIXED;
ALTER TABLE t ENGINE=InnoDB;
ALTER TABLE t ENGINE=MyISAM;

In InnoDB, the table would actually get some different ROW_FORMAT. When converting back to MyISAM, what happens might depend on the server version. The table could get the original ROW_FORMAT=FIXED from the .frm file, or the SQL layer could use the ROW_FORMAT that it got from the storage engine. It is possible that SHOW CREATE TABLE displays what the .frm file says, while SHOW TABLES or SHOW TABLE STATUS would display what the storage engine says.

MySQL 8.0 might fix this in the Global Data Dictionary (at least it was discussed a couple of years ago, and WL#7811 where I proposed a fix was design-approved more than 2 years ago). We will soon see how much they fixed this, as more 8.0 releases are coming out. A related thing is the need to distinguish implicit and explicit attributes. The SQL layer developers of MySQL would not want SHOW CREATE TABLE to display something that was implied by the storage engine (but they are happy to display NOT NULL that was implied for PRIMARY KEY columns). So, they would want to distinguish if an attribute was specified by the user.

For MariaDB, I have proposed a design fix in MDEV-11655. That is, we would basically write the CREATE TABLE string into a table, and this string would be the source of metadata. .frm files could still be used when manually copying files into the data directory.

In any case, the reported bug is not something that can be fixed in storage engines such as InnoDB. It must be fixed in the SQL layer, in the code that writes the .frm file in ALTER TABLE.

Comment by Sergei Golubchik [ 2017-02-22 ]

This is kind of intentional. KEY_BLOCK_SIZE is per-key attribute, specifying it on the table level is just a shortcut for setting it for every key that doesn't have it otherwise set.

When you create a table with KEY_BLOCK_SIZE=8, all keys get KEY_BLOCK_SIZE=8. When you do ALTER TABLE ... KEY_BLOCK_SIZE=0, it sets it for "every key that doesn't have it otherwise set", but all keys do, so this doesn't change anything per-key. You also need to recreate keys

alter table t1 drop primary key, add primary key (id1) key_block_size=0,
               drop key id2, add unique (id2) key_block_size=0;

After that KEY_BLOCK_SIZE will be completely gone from your SHOW CREATE TABLE.

There's an issue, though. With such an ALTER TABLE, MariaDB doesn't notice that the keys are changed (it compares the old and the new key definitions, but not key block sizes), so it treats the whole ALTER as no-op.
I've fixed it now, in new 10.0 (and 10.1, 10.2) this ALTER will work.

Comment by Jean-François Gagné [ 2017-02-23 ]

I know it might be intentional to have different KEY_BLOCK_SIZE on different KEYs.
So what is the good way to uncompress a table ? Is that what you share above Sergei (drop PK, add PK, drop KEY1, add KEY1, ...) ?
Thanks.

Comment by Sergei Golubchik [ 2017-02-23 ]

First, I thought that the first command ALTER TABLE ... ROW_FORMAT=DYNAMIC did uncompress the table, it only left a (harmless) warning for the ignored KEY_BLOCK_SIZE value. Was I wrong?

Second, I don't quite like the asymmetry in setting and clearing key block size attributes — one can set it with one ALTER TABLE ...KEY_BLOCK_SIZE=8, but needs to clear it on the table level and for every single index. I believe we can fix that, but not in 10.0.

Comment by Jean-François Gagné [ 2017-02-23 ]

I am ok with a fix in 10.1.

Comment by Marko Mäkelä [ 2017-02-28 ]

InnoDB is only looking at the table-wide KEY_BLOCK_SIZE attribute. Each index of the table will use the same ROW_FORMAT=COMPRESSED page size. MariaDB could support per-index KEY_BLOCK_SIZE for InnoDB, if there is demand for it, but I do not think it can happen before 10.3. Inside InnoDB, it would likely require having one file for each distinct KEY_BLOCK_SIZE, or for each index, so it would not be a simple change.

SHOW TABLE STATUS or SELECT * FROM INFORMATION_SCHEMA.INNODB_SYS_TABLES should reflect the actual ROW_FORMAT and page size that are used in InnoDB. The .frm file can contain conflicting information.

Comment by Sergei Golubchik [ 2017-02-28 ]

So, I've fixed ALTER TABLE to rebuild the index if KEY_BLOCK_SIZE changes.

The rest of the issue is moved to MDEV-12152, it cannot be fixed in 10.0

Comment by Jean-François Gagné [ 2017-03-01 ]

The way this is "fixed" is unclear to me. Which ALTER TABLE will rebuild the index ?

Does that mean that ALTER TABLE t1 ROW_FORMAT=DYNAMIC KEY_BLOCK_SIZE=0; does not rebuild indexes for InnoDB in "old" versions (so does not fully uncompress the table ? If this is the case, this should be considered as an InnoDB bug (and what is the "clean" way of uncompressing an InnoDB table) ?

Wouldn't ALTER TABLE t1 ROW_FORMAT=DYNAMIC; be the right way of uncompressing an InnoDB table ? In this case, it should not produce a warning and it should not "push" the KEY_BLOCK_SIZE in KEYs (I am not sure if it does, I did not test).

Note: in MDEV-12152, I suggested that ALTER TABLE t1 ROW_FORMAT=DYNAMIC KEY_BLOCK_SIZE=0; should fail for InnoDB tables.

Comment by Sergei Golubchik [ 2017-03-01 ]

This is "fixed" in a way that

alter table t1 drop primary key, add primary key (id1) key_block_size=0,
               drop key id2, add unique (id2) key_block_size=0;

would not be treated as a no-op, the server would actually notice that the old and new index definitions are different. That was clearly a bug and it's fixed in 10.0.

The rest of this issue is more like a change in behavior, not a bug fix, so we cannot do it in 10.0. It's now in MDEV-12152.

Generated at Thu Feb 08 07:52:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.