[MDEV-33957] Slave index corruption: InnoDB: Record in index was not found on update/rollback: TUPLE ... at: COMPACT RECORD - Jira

Details

Type: Bug
Status: Confirmed (View Workflow)
Priority: Critical
Resolution: Unresolved
Affects Version/s: 10.5, 10.6, 10.11, 11.1(EOL), 11.2(EOL), 11.3(EOL)
Fix Version/s: 10.5, 10.6, 10.11
Component/s: Performance Schema, Replication, Storage Engine - InnoDB, Virtual Columns
Labels:
- corruption
- sporadic

Description

# Requires standard m/s replication. Use RBR or MBR.

SET binlog_row_image='NOBLOB';

CREATE TABLE t (c INT PRIMARY KEY, d INT, i BLOB GENERATED ALWAYS AS (c), KEY k(i)) ENGINE=InnoDB;

CREATE TEMPORARY TABLE t2 (c INT) ENGINE=InnoDB;

INSERT INTO t (c) VALUES (1);

SELECT * FROM performance_schema.global_status;

UPDATE t SET d=0;

Leads to:

10.6.18 33af5575a976c86163752cce77dddb640ffeef00 (Optimized, Slave)
2024-04-23 10:31:44 6 [ERROR] InnoDB: Record in index `k` of table `test`.`t` was not found on update: TUPLE (info_bits=0, 2 fields): {NULL,[4] (0x80000001)} at: COMPACT RECORD(info_bits=0, 1 fields): {[8]infimum (0x696E66696D756D00)}
2024-04-23 10:31:44 6 [ERROR] InnoDB: We detected index corruption in an InnoDB type table. You have to dump + drop + reimport the table or, in a case of widespread corruption, dump all InnoDB tables and recreate the whole tablespace. If the mariadbd server crashes after the startup or when you dump the tables. Please refer to https://mariadb.com/kb/en/library/innodb-recovery-modes/ for information about forcing recovery.
2024-04-23 10:31:44 6 [Warning] InnoDB: Record in index `k` of table `test`.`t` was not found on rollback, trying to insert: TUPLE (info_bits=0, 2 fields): {NULL,[4] (0x80000001)} at: COMPACT RECORD(info_bits=0, 1 fields): {[8]infimum (0x696E66696D756D00)}
2024-04-23 10:31:44 6 [ERROR] mariadbd: Index for table 't' is corrupt; try to repair it
2024-04-23 10:31:44 6 [ERROR] Slave SQL: Could not execute Update_rows_v1 event on table test.t; Index for table 't' is corrupt; try to repair it, Error_code: 1034; handler error HA_ERR_CRASHED; the event's master log binlog.000001, end_log_pos 1595, Gtid 0-1-7, Internal MariaDB error code: 1034
2024-04-23 10:31:44 6 [Warning] Slave: Index for table 't' is corrupt; try to repair it Error_code: 1034
2024-04-23 10:31:44 6 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'binlog.000001' position 1416; GTID position '0-1-6'

May seem lightly sporadic, but is more likely timing-based (i.e. SLEEP's etc. may help to reduce any light sporadicity).
Both dbg+opt affected. PS required. The specific PS table queried does not seem to matter.
The issue has been reproduced in the versions listed above, but not in 11.4, 11.5.

Attachments

Issue Links

relates to

MDEV-34344 Replica index corruption and InnoDB: Failing assertion: !cursor->index->is_committed() in row_ins_sec_index_entry_by_modify when using generated columns

Confirmed

MDEV-34572 [ERROR] InnoDB: Record in index of table was not found on update

Open

Activity

Ascending order - Click to sort in descending order

Marko Mäkelä added a comment - 2024-04-22 04:58

I would like to see a self-contained, minimal test case for this. I would suggest to test an AddressSanitizer instrumented build, in case the error occurs due to a memory overrun. I tested the following, without the SELECT statement that would attempt to access performance_schema, because I usually build cmake -DPLUGIN_PERFSCHEMA=NO:

--source include/have_innodb.inc

--source include/have_partition.inc

SET default_storage_engine=innodb;

CREATE DATABASE mysqltest_lc2;

CREATE TABLE t1 (a INT,b INT,PRIMARY KEY(a)) PARTITION BY HASH (a) PARTITIONS 3;

SET SESSION binlog_row_image=1;

ALTER TABLE t1 ADD COLUMN x1 BLOB GENERATED ALWAYS AS (CONCAT (a,b)) VIRTUAL,ADD COLUMN x2 BLOB GENERATED ALWAYS AS (CONCAT (a,b)) VIRTUAL,ADD INDEX (x1 (100),x2 (120)),ADD INDEX (x1 (20));

CREATE TEMPORARY TABLE mysqltest_lc2.t1 (a INT);

INSERT INTO t1 (a) VALUES (8),(10),(12),(14),(16),(18),(20),(22),(24),(26),(23),(27);

--error ER_WARNING_NON_DEFAULT_VALUE_FOR_GENERATED_COLUMN

INSERT INTO t1 VALUES (6559,6559,6559,6559);

--error ER_TRUNCATED_WRONG_VALUE_FOR_FIELD

UPDATE t1 SET b='';

There was no crash on 10.6 bb2e125d076d9475cbe04fc8c24fbaee39d76bda.

Marko Mäkelä added a comment - 2024-04-22 04:58 I would like to see a self-contained, minimal test case for this. I would suggest to test an AddressSanitizer instrumented build, in case the error occurs due to a memory overrun. I tested the following, without the SELECT statement that would attempt to access performance_schema , because I usually build cmake -DPLUGIN_PERFSCHEMA=NO : --source include/have_innodb.inc --source include/have_partition.inc SET default_storage_engine=innodb; CREATE DATABASE mysqltest_lc2; CREATE TABLE t1 (a INT ,b INT , PRIMARY KEY (a)) PARTITION BY HASH (a) PARTITIONS 3; SET SESSION binlog_row_image=1; ALTER TABLE t1 ADD COLUMN x1 BLOB GENERATED ALWAYS AS (CONCAT (a,b)) VIRTUAL, ADD COLUMN x2 BLOB GENERATED ALWAYS AS (CONCAT (a,b)) VIRTUAL, ADD INDEX (x1 (100),x2 (120)), ADD INDEX (x1 (20)); CREATE TEMPORARY TABLE mysqltest_lc2.t1 (a INT ); INSERT INTO t1 (a) VALUES (8),(10),(12),(14),(16),(18),(20),(22),(24),(26),(23),(27); --error ER_WARNING_NON_DEFAULT_VALUE_FOR_GENERATED_COLUMN INSERT INTO t1 VALUES (6559,6559,6559,6559); --error ER_TRUNCATED_WRONG_VALUE_FOR_FIELD UPDATE t1 SET b= '' ; There was no crash on 10.6 bb2e125d076d9475cbe04fc8c24fbaee39d76bda.

Roel Van de Paar added a comment - 2024-04-22 08:10 - edited

Thank you marko

Found a semi-reliable way to reproduce it. Requires replication.
Present in 10.6.18 33af5575a976c86163752cce77dddb640ffeef00 (Optimized) but was not able to reproduce in 11.5 yet.

Roel Van de Paar added a comment - 2024-04-22 08:10 - edited Thank you marko Found a semi-reliable way to reproduce it. Requires replication. Present in 10.6.18 33af5575a976c86163752cce77dddb640ffeef00 (Optimized) but was not able to reproduce in 11.5 yet.

Roel Van de Paar added a comment - 2024-04-23 01:06 - edited

The testcase was updated above. This testcase works near-perfectly (lightly sporadic likely due to timings) with a standard CLI-based m/s setup and CLI exec. I attempted to convert the testcase to MTR in plenty, but MTR does not reproduce the issue. Here is what I have thus far:

--source include/have_innodb.inc

--source include/have_binlog_format_row.inc  # or _mixed.inc

--source include/have_perfschema.inc

--source include/master-slave.inc

SET binlog_row_image='NOBLOB';

CREATE TABLE t (c INT PRIMARY KEY, d INT, i BLOB GENERATED ALWAYS AS (c), KEY k(i)) ENGINE=InnoDB;

CREATE TEMPORARY TABLE t2 (c INT) ENGINE=InnoDB;

INSERT INTO t (c) VALUES (1);

SELECT * FROM performance_schema.global_status;

UPDATE t SET d=0;

--sync_slave_with_master

SELECT SLEEP(3);

--source include/rpl_end.inc

SELECT SLEEP(2);

elenst If you have any MTR replay insights that may help, they are welcome!

Roel Van de Paar added a comment - 2024-04-23 01:06 - edited The testcase was updated above. This testcase works near-perfectly (lightly sporadic likely due to timings) with a standard CLI-based m/s setup and CLI exec. I attempted to convert the testcase to MTR in plenty, but MTR does not reproduce the issue. Here is what I have thus far: --source include/have_innodb.inc --source include/have_binlog_format_row.inc # or _mixed.inc --source include/have_perfschema.inc --source include/master-slave.inc SET binlog_row_image= 'NOBLOB' ; CREATE TABLE t (c INT PRIMARY KEY , d INT , i BLOB GENERATED ALWAYS AS (c), KEY k(i)) ENGINE=InnoDB; CREATE TEMPORARY TABLE t2 (c INT ) ENGINE=InnoDB; INSERT INTO t (c) VALUES (1); SELECT * FROM performance_schema.global_status; UPDATE t SET d=0; --sync_slave_with_master SELECT SLEEP(3); --source include/rpl_end.inc SELECT SLEEP(2); elenst If you have any MTR replay insights that may help, they are welcome!

Roel Van de Paar added a comment - 2024-04-23 02:34 - edited

An UB+ASAN build gives the same outcome (index corruption), but no additional UB/ASAN outcomes.

Roel Van de Paar added a comment - 2024-04-23 02:34 - edited An UB+ASAN build gives the same outcome (index corruption), but no additional UB/ASAN outcomes.

Roel Van de Paar added a comment - 2025-01-01 05:41 - edited

Reconfirmed on current 10.5, using CLI with standard m/s setup. Debug slave output:

CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Debug, Slave, P_S included in build)
2025-01-01 16:36:41 5 [Note] Slave I/O thread: connected to master 'repl_user@127.0.0.1:10374',replication starts at GTID position ''
2025-01-01 16:37:36 6 [ERROR] InnoDB: Record in index `k` of table `test`.`t` was not found on update: TUPLE (info_bits=0, 2 fields): {NULL,[4] (0x80000001)} at: COMPACT RECORD(info_bits=0, 1 fields): {[8]infimum (0x696E66696D756D00)}
mariadbd: /test/10.5_dbg/storage/innobase/row/row0upd.cc:2115: dberr_t row_upd_sec_index_entry(upd_node_t , que_thr_t ): Assertion `0' failed.

CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Debug)
Core was generated by `/test/MD010125-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd --no-defaults --ma'.
Program terminated with signal SIGABRT, Aborted.
Download failed: Invalid argument. Continuing without source file ./nptl/./nptl/pthread_kill.c.
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44

[Current thread is 1 (LWP 1077246)]
(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89
#3 0x0000146a0aa4526e in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26
#4 0x0000146a0aa288ff in __GI_abort () at ./stdlib/abort.c:79
#5 0x0000146a0aa2881b in __assert_fail_base (fmt=0x146a0abd01e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x55f2a1ea2750 "0", file=file@entry=0x55f2a1f52424 "/test/10.5_dbg/storage/innobase/row/row0upd.cc", line=line@entry=2115, function=function@entry=0x55f2a1f539ea "dberr_t row_upd_sec_index_entry(upd_node_t , que_thr_t )") at ./assert/assert.c:94
#6 0x0000146a0aa3b507 in __assert_fail (assertion=0x55f2a1ea2750 "0", file=0x55f2a1f52424 "/test/10.5_dbg/storage/innobase/row/row0upd.cc", line=2115, function=0x55f2a1f539ea "dberr_t row_upd_sec_index_entry(upd_node_t , que_thr_t )") at ./assert/assert.c:103
#7 0x000055f2a184c7da in row_upd_sec_index_entry (node=0x146994035be0, thr=0x146994038638) at /test/10.5_dbg/storage/innobase/row/row0upd.cc:2115
#8 0x000055f2a184645a in row_upd_sec_step (node=0x146994035be0, thr=0x146994038638) at /test/10.5_dbg/storage/innobase/row/row0upd.cc:2264
#9 0x000055f2a1843dbf in row_upd (node=0x146994035be0, thr=0x146994038638)at /test/10.5_dbg/storage/innobase/row/row0upd.cc:3051
#10 0x000055f2a1843961 in row_upd_step (thr=0x146994038638)at /test/10.5_dbg/storage/innobase/row/row0upd.cc:3166
#11 0x000055f2a17da2ce in row_update_for_mysql (prebuilt=0x146994035128)at /test/10.5_dbg/storage/innobase/row/row0mysql.cc:1777
#12 0x000055f2a15e7894 in ha_innobase::update_row (this=0x146994032d70, old_row=0x14699402a920 "\377\001", new_row=0x14699402a908 "\374\001")at /test/10.5_dbg/storage/innobase/handler/ha_innodb.cc:8511
#13 0x000055f2a118e995 in handler::ha_update_row (this=0x146994032d70, old_data=0x14699402a920 "\377\001", new_data=0x14699402a908 "\374\001")at /test/10.5_dbg/sql/handler.cc:7411
#14 0x000055f2a131bdc6 in Update_rows_log_event::do_exec_row (this=0x55f2a44a7608, rgi=0x146994000cf0)at /test/10.5_dbg/sql/log_event_server.cc:8499
#15 0x000055f2a1313150 in Rows_log_event::do_apply_event (this=0x55f2a44a7608, rgi=0x146994000cf0) at /test/10.5_dbg/sql/log_event_server.cc:5761
#16 0x000055f2a0cd924a in Log_event::apply_event (this=0x55f2a44a7608, rgi=0x146994000cf0) at /test/10.5_dbg/sql/log_event.h:1495
#17 0x000055f2a0cc8b8d in apply_event_and_update_pos_apply (ev=0x55f2a44a7608, thd=0x146994001aa8, rgi=0x146994000cf0, reason=0)at /test/10.5_dbg/sql/slave.cc:3909
#18 0x000055f2a0cc8992 in apply_event_and_update_pos (ev=0x55f2a44a7608, thd=0x146994001aa8, rgi=0x146994000cf0) at /test/10.5_dbg/sql/slave.cc:4081
#19 0x000055f2a0cd2196 in exec_relay_log_event (thd=0x146994001aa8, rli=0x55f2a452cb48, serial_rgi=0x146994000cf0)at /test/10.5_dbg/sql/slave.cc:4449
#20 0x000055f2a0cc0d02 in handle_slave_sql (arg=0x55f2a452ae80)at /test/10.5_dbg/sql/slave.cc:5671
#21 0x000055f2a1508fcf in pfs_spawn_thread (arg=0x1469a0026b68)at /test/10.5_dbg/storage/perfschema/pfs.cc:2201
#22 0x0000146a0aa9ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
#23 0x0000146a0ab29c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Note that this stack is also seen in MDEV-34344 and various other closed bugs before that.

Roel Van de Paar added a comment - 2025-01-01 05:41 - edited Reconfirmed on current 10.5, using CLI with standard m/s setup. Debug slave output: CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Debug, Slave, P_S included in build) 2025-01-01 16:36:41 5 [Note] Slave I/O thread: connected to master 'repl_user@127.0.0.1:10374',replication starts at GTID position '' 2025-01-01 16:37:36 6 [ERROR] InnoDB: Record in index `k` of table `test`.`t` was not found on update: TUPLE (info_bits=0, 2 fields): {NULL,[4] (0x80000001)} at: COMPACT RECORD(info_bits=0, 1 fields): {[8]infimum (0x696E66696D756D00)} mariadbd: /test/10.5_dbg/storage/innobase/row/row0upd.cc:2115: dberr_t row_upd_sec_index_entry(upd_node_t *, que_thr_t *): Assertion `0' failed. CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Debug) Core was generated by `/test/MD010125-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd --no-defaults --ma'. Program terminated with signal SIGABRT, Aborted. Download failed: Invalid argument. Continuing without source file ./nptl/./nptl/pthread_kill.c. #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44 [Current thread is 1 (LWP 1077246)] (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89 #3 0x0000146a0aa4526e in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26 #4 0x0000146a0aa288ff in __GI_abort () at ./stdlib/abort.c:79 #5 0x0000146a0aa2881b in __assert_fail_base (fmt=0x146a0abd01e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x55f2a1ea2750 "0", file=file@entry=0x55f2a1f52424 "/test/10.5_dbg/storage/innobase/row/row0upd.cc", line=line@entry=2115, function=function@entry=0x55f2a1f539ea "dberr_t row_upd_sec_index_entry(upd_node_t *, que_thr_t *)") at ./assert/assert.c:94 #6 0x0000146a0aa3b507 in __assert_fail (assertion=0x55f2a1ea2750 "0", file=0x55f2a1f52424 "/test/10.5_dbg/storage/innobase/row/row0upd.cc", line=2115, function=0x55f2a1f539ea "dberr_t row_upd_sec_index_entry(upd_node_t *, que_thr_t *)") at ./assert/assert.c:103 #7 0x000055f2a184c7da in row_upd_sec_index_entry (node=0x146994035be0, thr=0x146994038638) at /test/10.5_dbg/storage/innobase/row/row0upd.cc:2115 #8 0x000055f2a184645a in row_upd_sec_step (node=0x146994035be0, thr=0x146994038638) at /test/10.5_dbg/storage/innobase/row/row0upd.cc:2264 #9 0x000055f2a1843dbf in row_upd (node=0x146994035be0, thr=0x146994038638)at /test/10.5_dbg/storage/innobase/row/row0upd.cc:3051 #10 0x000055f2a1843961 in row_upd_step (thr=0x146994038638)at /test/10.5_dbg/storage/innobase/row/row0upd.cc:3166 #11 0x000055f2a17da2ce in row_update_for_mysql (prebuilt=0x146994035128)at /test/10.5_dbg/storage/innobase/row/row0mysql.cc:1777 #12 0x000055f2a15e7894 in ha_innobase::update_row (this=0x146994032d70, old_row=0x14699402a920 "\377\001", new_row=0x14699402a908 "\374\001")at /test/10.5_dbg/storage/innobase/handler/ha_innodb.cc:8511 #13 0x000055f2a118e995 in handler::ha_update_row (this=0x146994032d70, old_data=0x14699402a920 "\377\001", new_data=0x14699402a908 "\374\001")at /test/10.5_dbg/sql/handler.cc:7411 #14 0x000055f2a131bdc6 in Update_rows_log_event::do_exec_row (this=0x55f2a44a7608, rgi=0x146994000cf0)at /test/10.5_dbg/sql/log_event_server.cc:8499 #15 0x000055f2a1313150 in Rows_log_event::do_apply_event (this=0x55f2a44a7608, rgi=0x146994000cf0) at /test/10.5_dbg/sql/log_event_server.cc:5761 #16 0x000055f2a0cd924a in Log_event::apply_event (this=0x55f2a44a7608, rgi=0x146994000cf0) at /test/10.5_dbg/sql/log_event.h:1495 #17 0x000055f2a0cc8b8d in apply_event_and_update_pos_apply (ev=0x55f2a44a7608, thd=0x146994001aa8, rgi=0x146994000cf0, reason=0)at /test/10.5_dbg/sql/slave.cc:3909 #18 0x000055f2a0cc8992 in apply_event_and_update_pos (ev=0x55f2a44a7608, thd=0x146994001aa8, rgi=0x146994000cf0) at /test/10.5_dbg/sql/slave.cc:4081 #19 0x000055f2a0cd2196 in exec_relay_log_event (thd=0x146994001aa8, rli=0x55f2a452cb48, serial_rgi=0x146994000cf0)at /test/10.5_dbg/sql/slave.cc:4449 #20 0x000055f2a0cc0d02 in handle_slave_sql (arg=0x55f2a452ae80)at /test/10.5_dbg/sql/slave.cc:5671 #21 0x000055f2a1508fcf in pfs_spawn_thread (arg=0x1469a0026b68)at /test/10.5_dbg/storage/perfschema/pfs.cc:2201 #22 0x0000146a0aa9ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447 #23 0x0000146a0ab29c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 Note that this stack is also seen in MDEV-34344 and various other closed bugs before that.

Roel Van de Paar added a comment - 2025-01-01 05:50 - edited

On 10.5 optimized we now see:

CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Optimized, Slave, P_S included in build)
2025-01-01 16:48:21 5 [Note] Slave I/O thread: connected to master 'repl_user@127.0.0.1:11863',replication starts at GTID position ''
2025-01-01 16:48:29 6 [ERROR] InnoDB: Record in index `k` of table `test`.`t` was not found on update: TUPLE (info_bits=0, 2 fields): {NULL,[4] (0x80000001)} at: COMPACT RECORD(info_bits=0, 1 fields): {[8]infimum (0x696E66696D756D00)}
2025-01-01 16:48:29 0x15304451e6c0 InnoDB: Assertion failure in file /test/10.5_opt/storage/innobase/row/row0ins.cc line 219
InnoDB: Failing assertion: !cursor->index->is_committed()
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mariadbd startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
InnoDB: about forcing recovery.

CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Optimized)
Core was generated by `/test/MD010125-mariadb-10.5.28-linux-x86_64-opt/bin/mariadbd --no-defaults --ma'.
Program terminated with signal SIGABRT, Aborted.
Download failed: Invalid argument. Continuing without source file ./nptl/./nptl/pthread_kill.c.
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44

[Current thread is 1 (LWP 1116093)]
(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89
#3 0x000015304724526e in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26
#4 0x00001530472288ff in __GI_abort () at ./stdlib/abort.c:79
#5 0x000055b95b39cc5d in ut_dbg_assertion_failed (expr=0x55b95b7724f8 "!cursor->index->is_committed()", file=0x55b95b771fc4 "/test/10.5_opt/storage/innobase/row/row0ins.cc", line=219) at /test/10.5_opt/storage/innobase/ut/ut0dbg.cc:60
#6 0x000055b95b32a344 in row_ins_sec_index_entry_by_modify (flags=0, mode=2, cursor=0x15304451b820, offsets=0x15304451b8e0, offsets_heap=0x152f7802d5c8, heap=0x152f7802da58, entry=0x152f7802d110, mtr=0x15304451c000, thr=<optimized out>)at /test/10.5_opt/storage/innobase/row/row0ins.cc:3146
#7 row_ins_sec_index_entry_low (flags=flags@entry=0, mode=mode@entry=2, index=index@entry=0x152f78025630, offsets_heap=0x152f7802d5c8, heap=heap@entry=0x152f7802da58, entry=entry@entry=0x152f7802d110, trx_id=0, thr=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0ins.cc:3142
#8 0x000055b95b32b304 in row_ins_sec_index_entry (index=index@entry=0x152f78025630, entry=entry@entry=0x152f7802d110, thr=thr@entry=0x152f7802b390, check_foreign=<optimized out>)at /test/10.5_opt/storage/innobase/row/row0ins.cc:3354
#9 0x000055b95b3607bf in row_upd_sec_index_entry (node=0x152f78028668, thr=0x152f7802b390) at /test/10.5_opt/storage/innobase/row/row0upd.cc:2237
#10 row_upd_sec_step (node=0x152f78028668, thr=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0upd.cc:2264
#11 row_upd (node=0x152f78028668, thr=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0upd.cc:3051
#12 row_upd_step (thr=thr@entry=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0upd.cc:3166
#13 0x000055b95b33b15b in row_update_for_mysql (prebuilt=0x152f78027bf0)at /test/10.5_opt/storage/innobase/row/row0mysql.cc:1777
#14 0x000055b95b270f65 in ha_innobase::update_row (this=0x152f78026c70, old_row=<optimized out>, new_row=0x152f78020148 "\374\001")at /test/10.5_opt/storage/innobase/handler/ha_innodb.cc:8511
#15 0x000055b95af97e4e in handler::ha_update_row (this=0x152f78026c70, old_data=0x152f78020160 "\377\001", new_data=0x152f78020148 "\374\001")at /test/10.5_opt/sql/handler.cc:7411
#16 0x000055b95b0ac441 in Update_rows_log_event::do_exec_row (this=0x55b95e23d9c8, rgi=0x152f78000c50)at /test/10.5_opt/sql/log_event_server.cc:8499
#17 0x000055b95b0a6ca2 in Rows_log_event::do_apply_event (this=0x55b95e23d9c8, rgi=0x152f78000c50) at /test/10.5_opt/sql/log_event_server.cc:5761
#18 0x000055b95acd6343 in Log_event::apply_event (this=this@entry=0x55b95e23d9c8, rgi=rgi@entry=0x152f78000c50)at /test/10.5_opt/sql/log_event.h:1495
#19 0x000055b95acd3360 in apply_event_and_update_pos_apply (ev=ev@entry=0x55b95e23d9c8, thd=thd@entry=0x152f78001568, rgi=rgi@entry=0x152f78000c50, reason=1193929500, reason@entry=0)at /test/10.5_opt/sql/slave.cc:3909
#20 0x000055b95accd20e in apply_event_and_update_pos (ev=0x55b95e23d9c8, thd=0x152f78001568, rgi=0x152f78000c50) at /test/10.5_opt/sql/slave.cc:4081
#21 exec_relay_log_event (thd=0x152f78001568, rli=0x55b95e292d08, serial_rgi=0x152f78000c50) at /test/10.5_opt/sql/slave.cc:4449
#22 handle_slave_sql (arg=arg@entry=0x55b95e291340)at /test/10.5_opt/sql/slave.cc:5671
#23 0x000055b95b1dd87f in pfs_spawn_thread (arg=0x152f8401e0f8)at /test/10.5_opt/storage/perfschema/pfs.cc:2201
#24 0x000015304729ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
#25 0x0000153047329c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Note that this stack is also seen in MDEV-34344 and various other closed bugs before that.

Roel Van de Paar added a comment - 2025-01-01 05:50 - edited On 10.5 optimized we now see: CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Optimized, Slave, P_S included in build) 2025-01-01 16:48:21 5 [Note] Slave I/O thread: connected to master 'repl_user@127.0.0.1:11863',replication starts at GTID position '' 2025-01-01 16:48:29 6 [ERROR] InnoDB: Record in index `k` of table `test`.`t` was not found on update: TUPLE (info_bits=0, 2 fields): {NULL,[4] (0x80000001)} at: COMPACT RECORD(info_bits=0, 1 fields): {[8]infimum (0x696E66696D756D00)} 2025-01-01 16:48:29 0x15304451e6c0 InnoDB: Assertion failure in file /test/10.5_opt/storage/innobase/row/row0ins.cc line 219 InnoDB: Failing assertion: !cursor->index->is_committed() InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mariadbd startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/ InnoDB: about forcing recovery. CS 10.5.28 24e5d564005904bc589fc869ea2f913eec8e6c3a (Optimized) Core was generated by `/test/MD010125-mariadb-10.5.28-linux-x86_64-opt/bin/mariadbd --no-defaults --ma'. Program terminated with signal SIGABRT, Aborted. Download failed: Invalid argument. Continuing without source file ./nptl/./nptl/pthread_kill.c. #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44 [Current thread is 1 (LWP 1116093)] (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=<optimized out>)at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6)at ./nptl/pthread_kill.c:89 #3 0x000015304724526e in __GI_raise (sig=sig@entry=6)at ../sysdeps/posix/raise.c:26 #4 0x00001530472288ff in __GI_abort () at ./stdlib/abort.c:79 #5 0x000055b95b39cc5d in ut_dbg_assertion_failed (expr=0x55b95b7724f8 "!cursor->index->is_committed()", file=0x55b95b771fc4 "/test/10.5_opt/storage/innobase/row/row0ins.cc", line=219) at /test/10.5_opt/storage/innobase/ut/ut0dbg.cc:60 #6 0x000055b95b32a344 in row_ins_sec_index_entry_by_modify (flags=0, mode=2, cursor=0x15304451b820, offsets=0x15304451b8e0, offsets_heap=0x152f7802d5c8, heap=0x152f7802da58, entry=0x152f7802d110, mtr=0x15304451c000, thr=<optimized out>)at /test/10.5_opt/storage/innobase/row/row0ins.cc:3146 #7 row_ins_sec_index_entry_low (flags=flags@entry=0, mode=mode@entry=2, index=index@entry=0x152f78025630, offsets_heap=0x152f7802d5c8, heap=heap@entry=0x152f7802da58, entry=entry@entry=0x152f7802d110, trx_id=0, thr=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0ins.cc:3142 #8 0x000055b95b32b304 in row_ins_sec_index_entry (index=index@entry=0x152f78025630, entry=entry@entry=0x152f7802d110, thr=thr@entry=0x152f7802b390, check_foreign=<optimized out>)at /test/10.5_opt/storage/innobase/row/row0ins.cc:3354 #9 0x000055b95b3607bf in row_upd_sec_index_entry (node=0x152f78028668, thr=0x152f7802b390) at /test/10.5_opt/storage/innobase/row/row0upd.cc:2237 #10 row_upd_sec_step (node=0x152f78028668, thr=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0upd.cc:2264 #11 row_upd (node=0x152f78028668, thr=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0upd.cc:3051 #12 row_upd_step (thr=thr@entry=0x152f7802b390)at /test/10.5_opt/storage/innobase/row/row0upd.cc:3166 #13 0x000055b95b33b15b in row_update_for_mysql (prebuilt=0x152f78027bf0)at /test/10.5_opt/storage/innobase/row/row0mysql.cc:1777 #14 0x000055b95b270f65 in ha_innobase::update_row (this=0x152f78026c70, old_row=<optimized out>, new_row=0x152f78020148 "\374\001")at /test/10.5_opt/storage/innobase/handler/ha_innodb.cc:8511 #15 0x000055b95af97e4e in handler::ha_update_row (this=0x152f78026c70, old_data=0x152f78020160 "\377\001", new_data=0x152f78020148 "\374\001")at /test/10.5_opt/sql/handler.cc:7411 #16 0x000055b95b0ac441 in Update_rows_log_event::do_exec_row (this=0x55b95e23d9c8, rgi=0x152f78000c50)at /test/10.5_opt/sql/log_event_server.cc:8499 #17 0x000055b95b0a6ca2 in Rows_log_event::do_apply_event (this=0x55b95e23d9c8, rgi=0x152f78000c50) at /test/10.5_opt/sql/log_event_server.cc:5761 #18 0x000055b95acd6343 in Log_event::apply_event (this=this@entry=0x55b95e23d9c8, rgi=rgi@entry=0x152f78000c50)at /test/10.5_opt/sql/log_event.h:1495 #19 0x000055b95acd3360 in apply_event_and_update_pos_apply (ev=ev@entry=0x55b95e23d9c8, thd=thd@entry=0x152f78001568, rgi=rgi@entry=0x152f78000c50, reason=1193929500, reason@entry=0)at /test/10.5_opt/sql/slave.cc:3909 #20 0x000055b95accd20e in apply_event_and_update_pos (ev=0x55b95e23d9c8, thd=0x152f78001568, rgi=0x152f78000c50) at /test/10.5_opt/sql/slave.cc:4081 #21 exec_relay_log_event (thd=0x152f78001568, rli=0x55b95e292d08, serial_rgi=0x152f78000c50) at /test/10.5_opt/sql/slave.cc:4449 #22 handle_slave_sql (arg=arg@entry=0x55b95e291340)at /test/10.5_opt/sql/slave.cc:5671 #23 0x000055b95b1dd87f in pfs_spawn_thread (arg=0x152f8401e0f8)at /test/10.5_opt/storage/perfschema/pfs.cc:2201 #24 0x000015304729ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447 #25 0x0000153047329c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 Note that this stack is also seen in MDEV-34344 and various other closed bugs before that.

Roel Van de Paar added a comment - 2025-01-01 06:12

11.8 Does not look affected.

Roel Van de Paar added a comment - 2025-01-01 06:12 11.8 Does not look affected.

People

Assignee:: Nikita Malyavin

Reporter:: Roel Van de Paar

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 2024-04-20 07:39

Updated:: 4 days ago 08:59

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server