[MDEV-29742] Assertion `((val << shift) & mask) == (val << shift)' failed in rec_set_bit_field_2 - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.7(EOL), 10.8(EOL), 10.9(EOL), 10.10(EOL)
Fix Version/s: 10.3.37, 10.4.27, 10.5.18, 10.6.11, 10.7.7, 10.8.6, 10.9.4, 10.10.2, 10.11.1
Component/s: Storage Engine - InnoDB
Labels:
None

Description

Set to Minor because I can only reproduce it on 10.3 debug.

Run with

--mysqld=--innodb-buffer-pool-size=128M --mysqld=--innodb-page-size=64K

--source include/have_innodb.inc

--source include/have_innodb_64k.inc

--source include/have_sequence.inc

CREATE TABLE t (id SMALLINT PRIMARY KEY, KEY(id)) ENGINE=InnoDB;

INSERT INTO t SELECT seq from seq_1_to_8200;

ALTER TABLE t ADD f BINARY FIRST;

# Cleanup

DROP TABLE t;

10.3 d099bcad
mysqld: /data/src/10.3/storage/innobase/include/rem0rec.inl:218: void rec_set_bit_field_2(rec_t*, ulint, ulint, ulint, ulint): Assertion `((val << shift) & mask) == (val << shift)' failed.
221008 3:05:32 [ERROR] mysqld got signal 6 ;

#7 0x00007f2fb747c662 in __GI___assert_fail (assertion=0x55a4d6cd9d60 "((val << shift) & mask) == (val << shift)", file=0x55a4d6cd9b20 "/data/src/10.3/storage/innobase/include/rem0rec.inl", line=218, function=0x55a4d6cd9e20 "void rec_set_bit_field_2(rec_t*, ulint, ulint, ulint, ulint)") at assert.c:101
#8 0x000055a4d5c65881 in rec_set_bit_field_2 (rec=0x7f2fa6ece06f "\237\377", val=8192, offs=4, mask=65528, shift=3) at /data/src/10.3/storage/innobase/include/rem0rec.inl:218
#9 0x000055a4d5c65ef3 in rec_set_heap_no_new (rec=0x7f2fa6ece06f "\237\377", heap_no=8192) at /data/src/10.3/storage/innobase/include/rem0rec.inl:780
#10 0x000055a4d5c6b685 in PageBulk::insert (this=0x61a00001da80, rec=0x62500071159d "\237\377", offsets=0x6250007115a8) at /data/src/10.3/storage/innobase/btr/btr0bulk.cc:220
#11 0x000055a4d5c70065 in BtrBulk::insert (this=0x7f2f98587e30, tuple=0x63100026c8a8, level=0) at /data/src/10.3/storage/innobase/btr/btr0bulk.cc:985
#12 0x000055a4d59cb72a in BtrBulk::insert (this=0x7f2f98587e30, tuple=0x63100026c8a8) at /data/src/10.3/storage/innobase/include/btr0bulk.h:315
#13 0x000055a4d5a355cf in row_merge_insert_index_tuples (index=0x61800003b108, old_table=0x6190000d5d08, fd=..., block=0x0, row_buf=0x615000032388, btr_bulk=0x7f2f98587e30, table_total_rows=6974, pct_progress=33.333333333333336, pct_cost=33.333333333333336, crypt_block=0x0, space=6, stage=0x0) at /data/src/10.3/storage/innobase/row/row0merge.cc:3682
#14 0x000055a4d5a2dcdd in row_merge_read_clustered_index (trx=0x7f2fadade8b8, table=0x61f000046088, old_table=0x6190000d5d08, new_table=0x6190014e7008, online=true, index=0x6190014e6da8, fts_sort_idx=0x0, psort_info=0x0, files=0x606000ae0980, key_numbers=0x6190014e6dc0, n_index=2, defaults=0x61d0001feb08, add_v=0x0, col_map=0x61d0001febc0, add_autoinc=18446744073709551615, sequence=..., block=0x7f2f97f80000 "", skip_pk_sort=true, tmpfd=0x7f2f985889a0, stage=0x60700001ed30, pct_cost=33.333333333333336, crypt_block=0x0, eval_table=0x61f000046088, allow_not_null=false) at /data/src/10.3/storage/innobase/row/row0merge.cc:2628
#15 0x000055a4d5a3c040 in row_merge_build_indexes (trx=0x7f2fadade8b8, old_table=0x6190000d5d08, new_table=0x6190014e7008, online=true, indexes=0x6190014e6da8, key_numbers=0x6190014e6dc0, n_indexes=2, table=0x61f000046088, defaults=0x61d0001feb08, col_map=0x61d0001febc0, add_autoinc=18446744073709551615, sequence=..., skip_pk_sort=true, stage=0x60700001ed30, add_v=0x0, eval_table=0x61f000046088, allow_not_null=false) at /data/src/10.3/storage/innobase/row/row0merge.cc:4736
#16 0x000055a4d57d142d in ha_innobase::inplace_alter_table (this=0x61c0000530a8, altered_table=0x61f000046088, ha_alter_info=0x7f2f98589e70) at /data/src/10.3/storage/innobase/handler/handler0alter.cc:7209
#17 0x000055a4d4cbd21e in handler::ha_inplace_alter_table (this=0x61c0000530a8, altered_table=0x61f000046088, ha_alter_info=0x7f2f98589e70) at /data/src/10.3/sql/handler.h:4147
#18 0x000055a4d4c9ffc1 in mysql_inplace_alter_table (thd=0x62a00009c208, table_list=0x62b000000340, table=0x61f000042888, altered_table=0x61f000046088, ha_alter_info=0x7f2f98589e70, target_mdl_request=0x7f2f9858a050, alter_ctx=0x7f2f9858a8b0) at /data/src/10.3/sql/sql_table.cc:7774
#19 0x000055a4d4cb14dc in mysql_alter_table (thd=0x62a00009c208, new_db=0x62a0000a0918, new_name=0x62a0000a0d00, create_info=0x7f2f9858b5d0, table_list=0x62b000000340, alter_info=0x7f2f9858b4d0, order_num=0, order=0x0, ignore=false) at /data/src/10.3/sql/sql_table.cc:10159
#20 0x000055a4d4e1a198 in Sql_cmd_alter_table::execute (this=0x62b000000b10, thd=0x62a00009c208) at /data/src/10.3/sql/sql_alter.cc:512
#21 0x000055a4d4a667d6 in mysql_execute_command (thd=0x62a00009c208) at /data/src/10.3/sql/sql_parse.cc:6076
#22 0x000055a4d4a72608 in mysql_parse (thd=0x62a00009c208, rawbuf=0x62b000000228 "ALTER TABLE t ADD f BINARY FIRST", length=32, parser_state=0x7f2f9858d9d0, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:7855
#23 0x000055a4d4a49c56 in dispatch_command (command=COM_QUERY, thd=0x62a00009c208, packet=0x6290000fa209 "ALTER TABLE t ADD f BINARY FIRST", packet_length=32, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:1852
#24 0x000055a4d4a4680e in do_command (thd=0x62a00009c208) at /data/src/10.3/sql/sql_parse.cc:1398
#25 0x000055a4d4e09a82 in do_handle_one_connection (connect=0x6080000010a8) at /data/src/10.3/sql/sql_connect.cc:1403
#26 0x000055a4d4e0937e in handle_one_connection (arg=0x6080000010a8) at /data/src/10.3/sql/sql_connect.cc:1308
#27 0x000055a4d63b4e9f in pfs_spawn_thread (arg=0x615000006c08) at /data/src/10.3/storage/perfschema/pfs.cc:1869
#28 0x00007f2fb7627ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#29 0x00007f2fb7547aef in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Attachments

Issue Links

relates to

MDEV-19526 Assertion `((val << shift) & mask) == (val << shift)' failed in rec_set_bit_field_2

Closed

Activity

Ascending order - Click to sort in descending order

Marko Mäkelä added a comment - 2022-10-10 05:58

ALTER TABLE t FORCE;

will cause the test to crash just fine in any version. Alternatively, if you disable ~~MDEV-15562~~ by the parameter that was introduced in ~~MDEV-20590~~, it should crash in any version.

In InnoDB index pages, index records are identified not only by the key, but also by "heap numbers" which reflect the order in which the records were inserted into the page. The heap numbers 0 and 1 are reserved for the predetermined "infimum" and "supremum" records, and in total there may be up to 8190 user records in a page, because 13 bits are being reserved for the heap number.

In the test, the fields of the PRIMARY KEY records are (id,DB_TRX_ID,DB_ROLL_PTR), total 5+2+6+7=15 bytes. Something like 3,200 such records can fit in a page when using innodb_page_size=64k. The fields of the secondary index are just (id), occupying 5+2 bytes each (5 bytes record header and 2 bytes payload). One could fit over 9,000 such records in a page with innodb_page_size=64k if we did not have the limitation with regard to the heap number.

In the INSERT…SELECT code path, the following check in btr_cur_optimistic_insert() will prevent this problem, triggering a page split.

	const ulint n_recs = page_get_n_recs(page);

	if (UNIV_UNLIKELY(n_recs >= 8189)) {

		ut_ad(srv_page_size == 65536);

		goto fail;

In other words, the fix ~~MDEV-19526~~ was incomplete.

Marko Mäkelä added a comment - 2022-10-10 05:58 ALTER TABLE t FORCE ; will cause the test to crash just fine in any version. Alternatively, if you disable MDEV-15562 by the parameter that was introduced in MDEV-20590 , it should crash in any version. In InnoDB index pages, index records are identified not only by the key, but also by "heap numbers" which reflect the order in which the records were inserted into the page. The heap numbers 0 and 1 are reserved for the predetermined "infimum" and "supremum" records, and in total there may be up to 8190 user records in a page, because 13 bits are being reserved for the heap number. In the test, the fields of the PRIMARY KEY records are (id,DB_TRX_ID,DB_ROLL_PTR), total 5+2+6+7=15 bytes. Something like 3,200 such records can fit in a page when using innodb_page_size=64k . The fields of the secondary index are just (id), occupying 5+2 bytes each (5 bytes record header and 2 bytes payload). One could fit over 9,000 such records in a page with innodb_page_size=64k if we did not have the limitation with regard to the heap number. In the INSERT…SELECT code path, the following check in btr_cur_optimistic_insert() will prevent this problem, triggering a page split. const ulint n_recs = page_get_n_recs(page); if (UNIV_UNLIKELY(n_recs >= 8189)) { ut_ad(srv_page_size == 65536); goto fail; } In other words, the fix MDEV-19526 was incomplete.

Marko Mäkelä added a comment - 2022-10-14 07:21

If the secondary index payload size is at least 4 bytes (say, if the redundant KEY(id) that is duplicating PRIMARY KEY(id) was defined on INT instead of SMALLINT), then less than 7,300 records could fit in a page. An upper bound estimate that pretends that the page header and footer are empty would be 65536/(5+4).

The definition id SMALLINT PRIMARY KEY allows us to insert at most 65536 rows into the entire table. If we had a non-redundant index, say, b TINYINT NOT NULL, INDEX(b), then the secondary index record would be (b,id) with a size of 5+1+2=8 bytes. In this case, 65536/8 would yield 8192. If we subtract the page header and footer, we can easily see that we can fit less than 8192 records in a 64-kilobyte page.

If we had id TINYINT PRIMARY KEY, then there would be at most 256 records in the entire table, and the 8192-byte heap number limit cannot possibly be exceeded during ALTER TABLE.

I do not think that this bug has any practical impact. ~~MDEV-19526~~ had more impact, because during DML, secondary index pages can contain delete-marked records.

Marko Mäkelä added a comment - 2022-10-14 07:21 If the secondary index payload size is at least 4 bytes (say, if the redundant KEY(id) that is duplicating PRIMARY KEY(id) was defined on INT instead of SMALLINT ), then less than 7,300 records could fit in a page. An upper bound estimate that pretends that the page header and footer are empty would be 65536/(5+4). The definition id SMALLINT PRIMARY KEY allows us to insert at most 65536 rows into the entire table. If we had a non-redundant index, say, b TINYINT NOT NULL, INDEX(b) , then the secondary index record would be (b,id) with a size of 5+1+2=8 bytes. In this case, 65536/8 would yield 8192. If we subtract the page header and footer, we can easily see that we can fit less than 8192 records in a 64-kilobyte page. If we had id TINYINT PRIMARY KEY , then there would be at most 256 records in the entire table, and the 8192-byte heap number limit cannot possibly be exceeded during ALTER TABLE . I do not think that this bug has any practical impact. MDEV-19526 had more impact, because during DML, secondary index pages can contain delete-marked records.

MariaDB Server

Assertion `((val << shift) & mask) == (val << shift)' failed in rec_set_bit_field_2

Details

Description

Attachments

Issue Links

Activity

People

Dates

Git Integration