Marko Mäkelä added a comment - 2022-06-10 13:10 This is a table rebuild into the ROW_FORMAT=COMPRESSED format. ALTER /* QNO 14785 CON_ID 10 */ TABLE `table10_innodb_int_autoinc` ROW_FORMAT=COMPRESSED; The assumption looks invalid in a ROW_FORMAT=COMPRESSED leaf page. Those pages are allowed to contain only one record. 639 /* Keep at least one record on left page */ 640 if (page_rec_is_second(rec, m_page)) { 641 rec = page_rec_get_next(rec); 642 ut_ad(page_rec_is_user_rec(rec)); 643 } I did recently touch this code in MDEV-28457 , but even before that, the incorrect assumption was there: /* Keep at least one record on left page */ if (page_rec_is_infimum(page_rec_get_prev(rec))) { rec = page_rec_get_next(rec); ut_ad(page_rec_is_user_rec(rec)); } I only replaced the more expensive and crash-prone if condition with an equivalent one that simply dereferences the next-record pointer of the page infimum twice. While I know that the assumption is invalid, it is not obvious to me how this should be fixed.

Elena Stepanova added a comment - 2022-06-12 18:30 - edited I suppose this is related to the originally reported failure: 10.6 51ca5d51 mariadbd: /data/src/10.6/storage/innobase/btr/btr0bulk.cc:699: void PageBulk::copyOut(rec_t*): Assertion `n > 0' failed. 220612 21:23:51 [ERROR] mysqld got signal 6 ;   #6 0x00007f04c063b40f in __assert_fail_base (fmt=0x7f04c07a4128 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x560f36424caa "n > 0", file=0x560f364248c0 "/data/src/10.6/storage/innobase/btr/btr0bulk.cc", line=699, function=<optimized out>) at assert.c:92 #7 0x00007f04c064a662 in __GI___assert_fail (assertion=0x560f36424caa "n > 0", file=0x560f364248c0 "/data/src/10.6/storage/innobase/btr/btr0bulk.cc", line=699, function=0x560f36424cb0 "void PageBulk::copyOut(rec_t*)") at assert.c:101 #8 0x0000560f35d41b40 in PageBulk::copyOut (this=0x7f0464058468, split_rec=0x7f04b6174082 "\200") at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:699 #9 0x0000560f35d42095 in BtrBulk::pageSplit (this=0x7f04641f9408, page_bulk=0x7f0464058468, next_page_bulk=0x7f0464c059b8) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:897 #10 0x0000560f35d423f3 in BtrBulk::pageCommit (this=0x7f04641f9408, page_bulk=0x7f0464058468, next_page_bulk=0x7f0464c059b8, insert_father=true) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:948 #11 0x0000560f35d42b64 in BtrBulk::insert (this=0x7f04641f9408, tuple=0x7f0464c2c358, level=0) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:1082 #12 0x0000560f35c25b58 in BtrBulk::insert (this=0x7f04641f9408, tuple=0x7f0464c2c358) at /data/src/10.6/storage/innobase/include/btr0bulk.h:301 #13 0x0000560f35c5d5c7 in row_merge_insert_index_tuples (index=0x7f046405fa98, old_table=0x7f04641fcb58, fd=..., block=0x0, row_buf=0x7f046405f4f8, btr_bulk=0x7f04641f9408, table_total_rows=5, pct_progress=0, pct_cost=50, crypt_block=0x0, space=6, stage=0x0) at /data/src/10.6/storage/innobase/row/row0merge.cc:3675 #14 0x0000560f35c59821 in row_merge_read_clustered_index (trx=0x7f04b6c60b80, table=0x7f04b59c42f0, old_table=0x7f04641fcb58, new_table=0x7f046405a8e8, online=true, index=0x7f046405a628, fts_sort_idx=0x0, psort_info=0x0, files=0x7f046420a628, key_numbers=0x7f046405a630, n_index=1, defaults=0x0, add_v=0x0, col_map=0x7f046405a6a8, add_autoinc=18446744073709551615, sequence=..., block=0x7f04b4153000 "", skip_pk_sort=true, tmpfd=0x7f04b59c3480, stage=0x7f0464043868, pct_cost=50, crypt_block=0x0, eval_table=0x7f04b59c42f0, allow_not_null=false) at /data/src/10.6/storage/innobase/row/row0merge.cc:2545 #15 0x0000560f35c607f4 in row_merge_build_indexes (trx=0x7f04b6c60b80, old_table=0x7f04641fcb58, new_table=0x7f046405a8e8, online=true, indexes=0x7f046405a628, key_numbers=0x7f046405a630, n_indexes=1, table=0x7f04b59c42f0, defaults=0x0, col_map=0x7f046405a6a8, add_autoinc=18446744073709551615, sequence=..., skip_pk_sort=true, stage=0x7f0464043868, add_v=0x0, eval_table=0x7f04b59c42f0, allow_not_null=false) at /data/src/10.6/storage/innobase/row/row0merge.cc:4591 #16 0x0000560f35afa1ec in ha_innobase::inplace_alter_table (this=0x7f0464238170, altered_table=0x7f04b59c42f0, ha_alter_info=0x7f04b59c4230) at /data/src/10.6/storage/innobase/handler/handler0alter.cc:8508 #17 0x0000560f35411ea1 in handler::ha_inplace_alter_table (this=0x7f0464238170, altered_table=0x7f04b59c42f0, ha_alter_info=0x7f04b59c4230) at /data/src/10.6/sql/handler.h:4676 #18 0x0000560f354033c1 in mysql_inplace_alter_table (thd=0x7f0464000db8, table_list=0x7f0464015690, table=0x7f0464237cc8, altered_table=0x7f04b59c42f0, ha_alter_info=0x7f04b59c4230, target_mdl_request=0x7f04b59c4b30, ddl_log_state=0x7f04b59c41d0, trigger_param=0x7f04b59c46e0, alter_ctx=0x7f04b59c5690) at /data/src/10.6/sql/sql_table.cc:7410 #19 0x0000560f3540c1f4 in mysql_alter_table (thd=0x7f0464000db8, new_db=0x7f04640059b8, new_name=0x7f0464005dd0, create_info=0x7f04b59c64a0, table_list=0x7f0464015690, alter_info=0x7f04b59c63b0, order_num=0, order=0x0, ignore=false, if_exists=false) at /data/src/10.6/sql/sql_table.cc:10278 #20 0x0000560f354bfcac in Sql_cmd_alter_table::execute (this=0x7f0464015d78, thd=0x7f0464000db8) at /data/src/10.6/sql/sql_alter.cc:542 #21 0x0000560f35309a62 in mysql_execute_command (thd=0x7f0464000db8, is_called_from_prepared_stmt=false) at /data/src/10.6/sql/sql_parse.cc:5996 #22 0x0000560f3530fab0 in mysql_parse (thd=0x7f0464000db8, rawbuf=0x7f04640155c0 "ALTER TABLE t FORCE", length=19, parser_state=0x7f04b59c7500) at /data/src/10.6/sql/sql_parse.cc:8029 #23 0x0000560f352fc113 in dispatch_command (command=COM_QUERY, thd=0x7f0464000db8, packet=0x7f046400b879 "ALTER TABLE t FORCE", packet_length=19, blocking=true) at /data/src/10.6/sql/sql_parse.cc:1896 #24 0x0000560f352fab0e in do_command (thd=0x7f0464000db8, blocking=true) at /data/src/10.6/sql/sql_parse.cc:1409 #25 0x0000560f354b4c68 in do_handle_one_connection (connect=0x560f37edbad8, put_in_cache=true) at /data/src/10.6/sql/sql_connect.cc:1418 #26 0x0000560f354b4907 in handle_one_connection (arg=0x560f37fa1628) at /data/src/10.6/sql/sql_connect.cc:1312 #27 0x0000560f359d012c in pfs_spawn_thread (arg=0x560f37edb648) at /data/src/10.6/storage/perfschema/pfs.cc:2201 #28 0x00007f04c0b16ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #29 0x00007f04c0713def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 The test case for this is simple, but it requires extra data files, I couldn't replace them with any obvious generations (REPEAT or HEX(REPEAT..)): --source include/have_innodb.inc   if (` select @@secure_file_priv is not null `) { --die # The test should be run with --secure-file-priv= (empty value) }   CREATE TABLE t (pk int , a blob, b blob, primary key (pk)) ENGINE=InnoDB ROW_FORMAT=COMPRESSED; INSERT INTO t VALUES (1, HEX(REPEAT( 'a' ,2048)), '' ), (2, LOAD_FILE( '/tmp/ja-4kb.txt' ), REPEAT( 'b' ,4096)), (3, LOAD_FILE( '/tmp/earth2kb.jpg' ), LOAD_FILE( '/tmp/ja-4kb.txt' )), (4, LOAD_FILE( '/tmp/earth5kb.jpg' ), '' ), (5, LOAD_FILE( '/tmp/earth5kb.jpg' ), '' ); ALTER TABLE t FORCE ;   # Cleanup DROP TABLE t; The files are attached, place them into /tmp/ directory and run the test case with --mysqld=--secure-file-priv='' . <= This is important , because if LOAD_FILE doesn't find the file or doesn't want to load it due to secure-file-priv restriction, it doesn't fail, it simply returns NULL, so it will look like the failure is not reproducible. earth2kb.jpg earth5kb.jpg ja-4kb.txt Reproducible on 10.6+. Non-debug build fails with the same test case on a different assertion: 10.6 51ca5d51 2022-06-12 21:33:44 0x7fe7f1453700 InnoDB: Assertion failure in file /data/src/10.6/storage/innobase/page/page0zip.cc line 761 InnoDB: Failing assertion: i + PAGE_HEAP_NO_USER_LOW == n_heap   #6 0x000055676b6d0058 in ut_dbg_assertion_failed (expr=expr@entry=0x55676c30f328 "i + PAGE_HEAP_NO_USER_LOW == n_heap", file=file@entry=0x55676c30efa0 "/data/src/10.6/storage/innobase/page/page0zip.cc", line=line@entry=761) at /data/src/10.6/storage/innobase/ut/ut0dbg.cc:60 #7 0x000055676b6b91b0 in page_zip_dir_encode (recs=0x7fe7acbcfe90, buf=<optimized out>, page=0x7fe7f1bf8000 "") at /data/src/10.6/storage/innobase/page/page0zip.cc:761 #8 page_zip_compress (block=0x7fe7f16f66c0, index=0x7fe7ac1a15a0, level=<optimized out>, mtr=mtr@entry=0x7fe7ac071798) at /data/src/10.6/storage/innobase/page/page0zip.cc:1432 #9 0x000055676be79646 in PageBulk::compress (this=0x7fe7ac071788) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:582 #10 BtrBulk::pageCommit (this=0x7fe7ac031fb8, page_bulk=0x7fe7ac071788, next_page_bulk=0x7fe7f144d020, insert_father=<optimized out>) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:947 #11 0x000055676be79434 in BtrBulk::pageSplit (this=0x7fe7ac031fb8, page_bulk=0x7fe7ac071788, next_page_bulk=0x7fe7ac1a41b8) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:900 #12 0x000055676be79e48 in BtrBulk::insert (this=this@entry=0x7fe7ac031fb8, tuple=tuple@entry=0x7fe7ac1bad20, level=level@entry=0) at /data/src/10.6/storage/innobase/btr/btr0bulk.cc:1082 #13 0x000055676be03781 in BtrBulk::insert (tuple=0x7fe7ac1bad20, this=0x7fe7ac031fb8) at /data/src/10.6/storage/innobase/include/btr0bulk.h:301 #14 row_merge_insert_index_tuples (index=0x7fe7ac1a15a0, old_table=0x7fe7ac11a910, fd=..., block=0x0, row_buf=0x7fe7ac114590, btr_bulk=0x7fe7ac031fb8, table_total_rows=5, pct_progress=0, pct_cost=50, crypt_block=0x0, space=6, stage=0x0) at /data/src/10.6/storage/innobase/row/row0merge.cc:3675 #15 0x000055676be09c02 in row_merge_read_clustered_index (trx=0x7fe7f2705b80, table=0x7fe7f144ee80, old_table=0x7fe7ac11a910, new_table=<optimized out>, online=true, index=0x7fe7ac037a68, fts_sort_idx=0x0, psort_info=0x0, files=0x7fe7ac104d68, key_numbers=0x7fe7ac037a70, n_index=1, defaults=0x0, add_v=0x0, col_map=0x7fe7ac037ae8, add_autoinc=18446744073709551615, sequence=..., block=0x7fe7f03df000 "", skip_pk_sort=true, tmpfd=0x7fe7f144e710, stage=0x7fe7ac038928, pct_cost=50, crypt_block=0x0, eval_table=0x7fe7f144ee80, allow_not_null=false) at /data/src/10.6/storage/innobase/row/row0merge.cc:2545 #16 0x000055676be0abdd in row_merge_build_indexes (trx=0x7fe7f2705b80, old_table=0x7fe7ac11a910, new_table=0x7fe7ac037d20, online=true, indexes=0x7fe7ac037a68, key_numbers=0x7fe7ac037a70, n_indexes=1, table=0x7fe7f144ee80, defaults=0x0, col_map=0x7fe7ac037ae8, add_autoinc=18446744073709551615, sequence=..., skip_pk_sort=true, stage=0x7fe7ac038928, add_v=0x0, eval_table=0x7fe7f144ee80, allow_not_null=false) at /data/src/10.6/storage/innobase/row/row0merge.cc:4591 #17 0x000055676bd7cbff in ha_innobase::inplace_alter_table (this=0x7fe7ac0ea100, altered_table=0x7fe7f144ee80, ha_alter_info=0x7fe7f144edc0) at /data/src/10.6/storage/innobase/handler/handler0alter.cc:8508 #18 0x000055676b8c3eb9 in handler::ha_inplace_alter_table (ha_alter_info=0x7fe7f144edc0, altered_table=0x7fe7f144ee80, this=<optimized out>) at /data/src/10.6/sql/handler.h:4676 #19 mysql_inplace_alter_table (target_mdl_request=0x7fe7f144f6c0, alter_ctx=0x7fe7f1450810, trigger_param=0x7fe7f144f270, ddl_log_state=0x7fe7f144ed60, ha_alter_info=0x7fe7f144edc0, altered_table=0x7fe7f144ee80, table=0x7fe7ac11b498, table_list=0x7fe7ac010a10, thd=0x7fe7ac000c58) at /data/src/10.6/sql/sql_table.cc:7410 #20 mysql_alter_table (thd=thd@entry=0x7fe7ac000c58, new_db=new_db@entry=0x7fe7ac005698, new_name=new_name@entry=0x7fe7ac005ab0, create_info=create_info@entry=0x7fe7f1451610, table_list=<optimized out>, table_list@entry=0x7fe7ac010a10, alter_info=alter_info@entry=0x7fe7f1451520, order_num=0, order=0x0, ignore=false, if_exists=false) at /data/src/10.6/sql/sql_table.cc:10278 #21 0x000055676b92d707 in Sql_cmd_alter_table::execute (this=<optimized out>, thd=0x7fe7ac000c58) at /data/src/10.6/sql/sql_alter.cc:542 #22 0x000055676b81850e in mysql_execute_command (thd=0x7fe7ac000c58, is_called_from_prepared_stmt=<optimized out>) at /data/src/10.6/sql/sql_parse.cc:5996 #23 0x000055676b8196e7 in mysql_parse (thd=0x7fe7ac000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /data/src/10.6/sql/sql_parse.cc:8029 #24 0x000055676b81bfad in dispatch_command (command=COM_QUERY, thd=0x7fe7ac000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /data/src/10.6/sql/sql_class.h:234 #25 0x000055676b81dc62 in do_command (thd=0x7fe7ac000c58, blocking=blocking@entry=true) at /data/src/10.6/sql/sql_parse.cc:1409 #26 0x000055676b928a9f in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /data/src/10.6/sql/sql_connect.cc:1418 #27 0x000055676b928ded in handle_one_connection (arg=arg@entry=0x55676f61f638) at /data/src/10.6/sql/sql_connect.cc:1312 #28 0x000055676bcaa132 in pfs_spawn_thread (arg=0x55676f5daee8) at /data/src/10.6/storage/perfschema/pfs.cc:2201 #29 0x00007fe7f858fea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #30 0x00007fe7f81a6def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Elena Stepanova added a comment - 2022-11-04 17:22 Raising the priority because there is now MDEV-29589 with the same assertion failure as in the above comment, reported by a user.

Marko Mäkelä added a comment - 2022-11-07 06:50 The record heap number overflow may have been fixed in MDEV-29742 .

Thirunarayanan Balathandayuthapani added a comment - 2022-11-07 09:47 Problem is caused because PageBulk::PageSplit() chooses the first record to be split record. It leads to an empty left side page. In debug build, CopyOut() fails with n> 0 that means at least one record should present in left page. In release build, while doing pageCommit(), we populate the dense page directory and it iterates through the empty page. This leads to failure for the assert ut_a(i + PAGE_HEAP_NO_USER_LOW == n_heap); This is a regression of MDEV-28457 .

Thirunarayanan Balathandayuthapani added a comment - 2022-11-07 11:46 Patch is in bb-10.6- MDEV-29518

Marko Mäkelä added a comment - 2022-11-07 14:27 Thank you. I see that you also removed a couple unused functions.