Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.11, 11.4, 11.8
Description
MySQL 8.0.42 includes the following change:
Bug #36993445 virtual index unstable after rollback when index_id is greater than max uint32 (fixup)
This fixes an old bug that had been introduced in MySQL 5.7. I knew about that at least since 2016 (and in 2020 as part of fixing MDEV-23672 we added FIXME comments about it) but I was reluctant to fix it, because I wanted to preserve compatibility with MySQL 5.7 data files.
The fix in MySQL 8.0.42 is changing the encoding, The original variable-length encoding was as follows:
| encoding | range |
| 0b0xxxxxxx | 0‥(1<<7)-1 |
| 0b10xxxxxxxxxxxxxx | 1<<8…(1<<14)-1 |
| 0b110xxxxxxxxxxxxxxxxxxxxx | 1<<14…(1<<21)-1 |
| 0b1110xxxxxxxxxxxxxxxxxxxxxxxxxxxx | 1<<21‥(1<<28)-1 |
| 0b11110000xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | 1<<28‥(1<<32)-1 |
The fix is replacing mach_write_compressed() with mach_u64_write_much_compressed(), which produces an identical encoding for 32-bit unsigned values. Any 64-bit unsigned integer that does not fit in 32 bits would be encoded as the octet 0xff followed by two the variable-length encoded 32-bit halves of the integer. This encoding would occupy 1+2·3 to 1+2·5 bytes (4 to 11 bytes). So, instead of the original 1‥5-byte encoding we would encode 1‥11 bytes.
Attachments
Issue Links
- blocks
-
-
- Closed
-
- relates to
-
MDEV-5800 indexes on virtual (not materialized) columns
-
- Closed
-
- links to
Activity
On a CMAKE_BUILD_TYPE=RelWithDebInfo executable without the fix, the modified test case would us a little further before crashing:
|
10.5 08f902e9ce525e0512c5d6d43b6548200c6fc673 |
CURRENT_TEST: gcol.innodb_virtual_basic
|
mysqltest: At line 521: query 'ROLLBACK' failed: 2013: Lost connection to MySQL server during query
|
…
|
2025-04-16 16:26:12 3 [ERROR] InnoDB: Clustered record for sec rec not found index `idx` of table `test`.`t`
|
InnoDB: sec index record PHYSICAL RECORD: n_fields 2; compact format; info bits 0
|
0: len 30; hex 616161616161616161616161616161616161616161616161616161616161; asc aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa; (total 100 bytes);
|
1: len 6; hex 000000000301; asc ;;
|
|
|
InnoDB: clust index record PHYSICAL RECORD: n_fields 6; compact format; info bits 0
|
0: len 6; hex 000000000300; asc ;;
|
1: len 6; hex 000000000000; asc ;;
|
2: len 7; hex 80000000000000; asc ;;
|
3: len 20; hex 0000000800000004000000260000000000003e80; asc & > ;;
|
4: len 20; hex 0000000800000005000000260000000000003e80; asc & > ;;
|
5: len 2; hex 6b6b; asc kk;;
|
|
|
TRANSACTION 421502838587544, ACTIVE 0 sec starting index read
|
mysql tables in use 1, locked 0
|
0 lock struct(s), heap size 1128, 0 row lock(s)
|
MySQL thread id 3, OS thread handle 140027818358464, query id 17 localhost root Sending data
|
SELECT COUNT(*) FROM t WHERE c like "aaa%"
|
|
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
2025-04-16 16:26:12 0x7f5ac45f36c0 InnoDB: Assertion failure in file /mariadb/10.5/storage/innobase/row/row0umod.cc line 1121
|
InnoDB: Failing assertion: thr_get_trx(thr)->is_recovered
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mariadbd startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
InnoDB: about forcing recovery.
|
250416 16:26:12 [ERROR] /dev/shm/10.5/sql/mariadbd got signal 6 ;
|
Sorry, we probably made a mistake, and this is a bug.
|
…
|
#7 0x00005632b7de24ab in ut_dbg_assertion_failed (expr=expr@entry=0x5632b86d48f0 "thr_get_trx(thr)->is_recovered", file=file@entry=0x5632b86d47a0 "/mariadb/10.5/storage/innobase/row/row0umod.cc", line=line@entry=0x461) at /mariadb/10.5/storage/innobase/ut/ut0dbg.cc:60
|
#8 0x00005632b7e00d4e in row_undo_mod_upd_exist_sec (node=0x0, thr=0x7f5a70136d38) at /mariadb/10.5/storage/innobase/row/row0umod.cc:1121
|
#9 row_undo_mod (node=node@entry=0x7f5a700dda68, thr=thr@entry=0x7f5a70136d38) at /mariadb/10.5/storage/innobase/row/row0umod.cc:1358
|
#10 0x00005632b8447e4a in row_undo (node=0x7f5a700dda68, thr=0x7f5a70136d38) at /mariadb/10.5/storage/innobase/row/row0undo.cc:424
|
#11 row_undo_step (thr=thr@entry=0x7f5a70136d38) at /mariadb/10.5/storage/innobase/row/row0undo.cc:476
|
#12 0x00005632b8403f8d in que_thr_step (thr=0x7f5a70136d38) at /mariadb/10.5/storage/innobase/que/que0que.cc:863
|
#13 que_run_threads_low (thr=0x7f5a70136d38) at /mariadb/10.5/storage/innobase/que/que0que.cc:927
|
#14 que_run_threads (thr=<optimized out>) at /mariadb/10.5/storage/innobase/que/que0que.cc:967
|
#15 0x00005632b84706f6 in trx_t::rollback_low (this=this@entry=0x7f5ac6f74098, savept=savept@entry=0x0) at /mariadb/10.5/storage/innobase/trx/trx0roll.cc:124
|
#16 0x00005632b846f738 in trx_rollback_for_mysql_low (trx=0x7f5ac6f74098) at /mariadb/10.5/storage/innobase/trx/trx0roll.cc:192
|
#17 trx_rollback_for_mysql (trx=trx@entry=0x7f5ac6f74098) at /mariadb/10.5/storage/innobase/trx/trx0roll.cc:263
|
#18 0x00005632b836d65e in innobase_rollback (hton=<optimized out>, thd=0x7f5a70000c68, rollback_trx=0x1) at /mariadb/10.5/storage/innobase/handler/ha_innodb.cc:4472
|
#19 0x00005632b81158fc in ha_rollback_trans (thd=thd@entry=0x7f5a70000c68, all=all@entry=0x1) at /mariadb/10.5/sql/handler.cc:2157
|
#20 0x00005632b8011901 in trans_rollback (thd=thd@entry=0x7f5a70000c68) at /mariadb/10.5/sql/transaction.cc:372
|
#21 0x00005632b7f058e3 in mysql_execute_command (thd=thd@entry=0x7f5a70000c68) at /mariadb/10.5/sql/sql_parse.cc:5854
|
#22 0x00005632b7f07ce5 in mysql_parse (thd=0x7f5a70000c68, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /mariadb/10.5/sql/sql_parse.cc:8252
|
#23 0x00005632b7f0a693 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7f5a70000c68, packet=packet@entry=0x7f5a70008119 "ROLLBACK", packet_length=packet_length@entry=0x8, is_com_multi=is_com_multi@entry=0x0, is_next_command=is_next_command@entry=0x0) at /mariadb/10.5/sql/sql_parse.cc:1992
|
#24 0x00005632b7f0b9ba in do_command (thd=thd@entry=0x7f5a70000c68) at /mariadb/10.5/sql/sql_parse.cc:1375
|
#25 0x00005632b8003060 in do_handle_one_connection (connect=<optimized out>, put_in_cache=0x1) at /mariadb/10.5/sql/sql_connect.cc:1386
|
#26 0x00005632b800334d in handle_one_connection (arg=0x5632d6e15828) at /mariadb/10.5/sql/sql_connect.cc:1298
|
#27 0x00007f5ac769cb7b in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:448
|
#28 0x00007f5ac771a7b8 in __GI___clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
|
I can reproduce a crash with the following test change:
diff --git a/mysql-test/suite/gcol/t/innodb_virtual_basic.test b/mysql-test/suite/gcol/t/innodb_virtual_basic.testindex b64daa2bcdb..69f9f89ccee 100644--- a/mysql-test/suite/gcol/t/innodb_virtual_basic.test+++ b/mysql-test/suite/gcol/t/innodb_virtual_basic.test@@ -1,6 +1,6 @@--source include/have_innodb.inc--source include/have_partition.inc---source include/big_test.inc+--source include/not_embedded.inccall mtr.add_suppression("\\[Warning\\] InnoDB: Compute virtual");@@ -66,6 +66,41 @@ DROP INDEX idx1 ON t;DROP INDEX idx2 ON t;DROP TABLE t;+let MYSQLD_DATADIR=`select @@datadir`;+let PAGE_SIZE=`select @@innodb_page_size`;+--source include/shutdown_mysqld.inc+perl;+do "$ENV{MTR_SUITE_DIR}/../innodb/include/crc32.pl";+my $file = "$ENV{MYSQLD_DATADIR}/ibdata1";+open(FILE, "+<$file") || die "Unable to open $file";+binmode FILE;+my $ps= $ENV{PAGE_SIZE};+my $page;+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;+my $full_crc32 = unpack("N",substr($page,54,4)) & 0x10; # FIL_SPACE_FLAGS+sysseek(FILE, 7*$ps, 0) || die "Unable to seek $file\n";+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;+substr($page,54,4)=pack("N",0xc001cafe); # 32 MSB of 64-bit DICT_HDR_INDEX_ID+my $polynomial = 0x82f63b78; # CRC-32C+if ($full_crc32)+{+ my $ck = mycrc32(substr($page, 0, $ps-4), 0, $polynomial);+ substr($page, $ps-4, 4) = pack("N", $ck);+}+else+{+ my $ck= pack("N",mycrc32(substr($page, 4, 22), 0, $polynomial) ^+ mycrc32(substr($page, 38, $ps - 38 - 8), 0, $polynomial));+ substr($page,0,4)=$ck;+ substr($page,$ps-8,4)=$ck;+}+sysseek(FILE, 7*$ps, 0) || die "Unable to rewind $file\n";+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";+close(FILE) || die "Unable to close $file";+EOF+--source include/start_mysqld.inc+set default_storage_engine=innodb;+/* Test large BLOB data */CREATE TABLE `t` (10.5 08f902e9ce525e0512c5d6d43b6548200c6fc673
Version: '10.5.29-MariaDB-debug-log' socket: '/dev/shm/10.5/mysql-test/var/tmp/mysqld.1.sock' port: 19000 Source distribution2025-04-16 15:32:13 3 [ERROR] InnoDB: Clustered record for sec rec not found index `idx` of table `test`.`t`InnoDB: sec index record PHYSICAL RECORD: n_fields 2; compact format; info bits 00: len 30; hex 616161616161616161616161616161616161616161616161616161616161; asc aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa; (total 100 bytes);1: len 6; hex 000000000301; asc ;;InnoDB: clust index record PHYSICAL RECORD: n_fields 6; compact format; info bits 00: len 6; hex 000000000300; asc ;;1: len 6; hex 000000000000; asc ;;2: len 7; hex 80000000000000; asc ;;3: len 20; hex 0000000800000004000000260000000000003e80; asc & > ;;4: len 20; hex 0000000800000005000000260000000000003e80; asc & > ;;5: len 2; hex 6b6b; asc kk;;TRANSACTION 421265987068384, ACTIVE 0 sec starting index readmysql tables in use 1, locked 00 lock struct(s), heap size 1152, 0 row lock(s)MySQL thread id 3, OS thread handle 139790987351744, query id 17 localhost root Sending dataSELECT COUNT(*) FROM t WHERE c like "aaa%"InnoDB: Submit a detailed bug report to https://jira.mariadb.org/mariadbd: /mariadb/10.5/storage/innobase/row/row0sel.cc:3438: dberr_t Row_sel_get_clust_rec_for_mysql::operator()(row_prebuilt_t*, dict_index_t*, const rec_t*, que_thr_t*, const rec_t**, rec_offs**, mem_heap_t**, dtuple_t**, mtr_t*): Assertion `0' failed.