[#MDEV-26203] CREATE INDEX may check the wrong maximum column length

In ha_innobase::prepare_inplace_alter_table() when the table is not being rebuilt, max_col_len must be determined based on the current table:

diff --git a/storage/innobase/handler/handler0alter.cc b/storage/innobase/handler/handler0alter.cc

index 3ac4f4fe4c1..1b4b48e90f7 100644

--- a/storage/innobase/handler/handler0alter.cc

+++ b/storage/innobase/handler/handler0alter.cc

@@ -5601,7 +5601,14 @@ ha_innobase::prepare_inplace_alter_table(

 			       & 1U << DICT_TF_POS_DATA_DIR);

-	max_col_len = DICT_MAX_FIELD_LEN_BY_FORMAT_FLAG(info.flags());

+	/* ALGORITHM=COPY uses the default row format while

+	ALGORITHM=INPLACE uses the current format. Find the limit for

+	the resulting format.*/

+	max_col_len = DICT_MAX_FIELD_LEN_BY_FORMAT_FLAG(

+		innobase_need_rebuild(ha_alter_info, this->table)

+		? info.flags()

+		: m_prebuilt->table->flags);

 	/* Check each index's column length to make sure they do not

 	exceed limit */

When no ROW_FORMAT is explicitly specified and the value of innodb_default_row_format is changed between CREATE TABLE and ALTER TABLE…ADD INDEX (or CREATE INDEX), we could enforce the wrong maximum column length limitation. For ROW_FORMAT=REDUNDANT and ROW_FORMAT=COMPACT, the maximum column length for a full-column index is 767 bytes.

[MDEV-26203] CREATE INDEX may check the wrong maximum column length Created: 2021-07-21 Updated: 2021-07-23 Resolved: 2021-07-23
Status:	Closed
Project:	MariaDB Server
Component/s:	Storage Engine - InnoDB
Affects Version/s:	10.2.2, 10.3.0, 10.4.0, 10.5.0, 10.6.0
Fix Version/s:	10.2.40, 10.3.31, 10.4.21, 10.5.12, 10.6.4

[MDEV-26203] CREATE INDEX may check the wrong maximum column length Created: 2021-07-21 Updated: 2021-07-23 Resolved: 2021-07-23