Details
Description
We received the first error of table corruption at
2024-06-08 1:31:13 1018409755 [ERROR] mariadbd: Index for table 'table1' is corrupt; try to repair it
The this happened when a query tried to access the table:
2024-06-08 14:16:34 0x7f8444b69640 InnoDB: Assertion failure in file ./storage/innobase/page/page0zip.cc line 4211
|
InnoDB: Failing assertion: slot_rec
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mariadbd startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
InnoDB: about forcing recovery.
|
240608 14:16:34 [ERROR] mysqld got signal 6 ;
|
Sorry, we probably made a mistake, and this is a bug.
|
|
|
Your assistance in bug reporting will enable us to fix this for the next release.
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.6.17-MariaDB-1:10.6.17+maria~ubu2204-log source revision: 15c75ad083a55e198ae78324f22970694b72f22b
|
key_buffer_size=16777216
|
read_buffer_size=131072
|
max_used_connections=1272
|
max_threads=10002
|
thread_count=400
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22044434 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7eed8c12b4e8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f8444b68c38 thread_stack 0x40000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x32)[0x55d7bc2dabe2]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x478)[0x55d7bbd862c8]
|
libc_sigaction.c:0(__restore_rt)[0x7f8455cd0520]
|
nptl/pthread_kill.c:44(__pthread_kill_implementation)[0x7f8455d249fc]
|
posix/raise.c:27(__GI_raise)[0x7f8455cd0476]
|
stdlib/abort.c:81(__GI_abort)[0x7f8455cb67f3]
|
/usr/sbin/mariadbd(+0x67c0d6)[0x55d7bb9e50d6]
|
/usr/sbin/mariadbd(+0xdb2c9e)[0x55d7bc11bc9e]
|
/usr/sbin/mariadbd(+0xda2635)[0x55d7bc10b635]
|
/usr/sbin/mariadbd(+0xe5a83c)[0x55d7bc1c383c]
|
/usr/sbin/mariadbd(+0xdd3cfd)[0x55d7bc13ccfd]
|
/usr/sbin/mariadbd(+0xdd693d)[0x55d7bc13f93d]
|
/usr/sbin/mariadbd(+0xdd6df2)[0x55d7bc13fdf2]
|
/usr/sbin/mariadbd(+0xde6c19)[0x55d7bc14fc19]
|
/usr/sbin/mariadbd(+0xd398ca)[0x55d7bc0a28ca]
|
/usr/sbin/mariadbd(_ZN7handler12ha_write_rowEPKh+0x1f0)[0x55d7bbd95830]
|
/usr/sbin/mariadbd(_Z12write_recordP3THDP5TABLEP12st_copy_infoP13select_result+0x1dd)[0x55d7bbafd9ed]
|
/usr/sbin/mariadbd(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesbP13select_result+0xd3a)[0x55d7bbb072ea]
|
/usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x14c9)[0x55d7bbb3bb09]
|
/usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x55d7bbb40487]
|
/usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14cd)[0x55d7bbb42c7d]
|
/usr/sbin/mariadbd(_Z10do_commandP3THDb+0x138)[0x55d7bbb44b98]
|
/usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x55d7bbc5a5af]
|
/usr/sbin/mariadbd(handle_one_connection+0x5d)[0x55d7bbc5a8fd]
|
/usr/sbin/mariadbd(+0xc7c8d6)[0x55d7bbfe58d6]
|
nptl/pthread_create.c:442(start_thread)[0x7f8455d22ac3]
|
x86_64/clone3.S:83(__clone3)[0x7f8455db4850]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7eed8f80c160): INSERT INTO `table1`(`fk1_id`, `fk2_id`, `fk3_id`, `start_date`, `end_date`) VALUES ('11', '61', '1486327', '2024-06-08 14:16:34', '2024-06-15 23:59:59')
|
|
|
Connection ID (thread ID): 1023439276
|
Status: NOT_KILLED
|
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off
|
,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,p
|
artial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_ca
|
che_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derive
|
d=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narr
|
owing=off
|
|
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 4126213 4126213 processes
|
Max open files 32768 32768 files
|
Max locked memory 524288 524288 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 4126213 4126213 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E
|
|
|
|
|
Kernel version: Linux version 5.15.0-97-generic (buildd@lcy02-amd64-033) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #107-Ubuntu SMP
|
Wed Feb 7 13:26:48 UTC 2024
|
We have the file that apport created. It is 13gb long and it seems to contain the core dump. This is from a production system and will contain sensitive data, so we can't just upload it. If there are specific requests / commands, we can run them on the core file.
CREATE TABLE `table1` ( |
`table1_id` int(10) unsigned NOT NULL AUTO_INCREMENT, |
`fk1_id` int(10) unsigned DEFAULT NULL, |
`used` tinyint(1) NOT NULL DEFAULT 0, |
`fk2_id` int(10) unsigned DEFAULT NULL, |
`fk3_id` int(10) unsigned DEFAULT NULL, |
`start_date` datetime NOT NULL, |
`end_date` datetime NOT NULL, |
PRIMARY KEY (`table1_id`), |
UNIQUE KEY `table1_uk` (`fk1_id`,`fk3_id`), |
KEY `table1_fk2` (`fk3_id`), |
KEY `table1_fk3` (`fk2_id`), |
CONSTRAINT `table1_ibfk_1` FOREIGN KEY (`fk1_id`) REFERENCES `table_for_fk1` (`fk1_id`), |
CONSTRAINT `table1_ibfk_2` FOREIGN KEY (`fk3_id`) REFERENCES `table_for_fk3` (`fk3_id`), |
CONSTRAINT `table1_ibfk_3` FOREIGN KEY (`fk2_id`) REFERENCES `table_for_fk2` (`fk2_id`) |
) ENGINE=InnoDB AUTO_INCREMENT=8505097 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=COMPRESSED |
Data was successfully inserted into the affected table between the corruption log entry at 1:31:13 and the assertion.
CHECK TABLE produced this log output:
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: In pages [page id: space=227292, page number=44] and [page id: space=227292, page number=45] of index `table1_fk2` of table `clientDb`.`table1`
|
InnoDB: records in wrong order on adjacent pages
|
InnoDB: record PHYSICAL RECORD: n_fields 2; compact format; info bits 0
|
0: len 4; hex 0016adf7; asc ;;
|
1: len 4; hex 0081680a; asc h ;;
|
|
|
InnoDB: record PHYSICAL RECORD: n_fields 2; compact format; info bits 0
|
0: len 4; hex 00152869; asc (i;;
|
1: len 4; hex 0080dfd3; asc ;;
|
|
|
InnoDB: node pointer to the right page is wrong
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: In page 44 of index `table1_fk2` of table `clientDb`.`table1`
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: In page 45 of index `table1_fk2` of table `clientDb`.`table1`
|
InnoDB: node pointer to the page is wrong
|
InnoDB: node ptr PHYSICAL RECORD: n_fields 3; compact format; info bits 0
|
0: len 4; hex 00151fef; asc ;;
|
1: len 4; hex 006bb459; asc k Y;;
|
2: len 4; hex 0000002c; asc ,;;
|
|
|
InnoDB: node ptr child page n:o 45
|
InnoDB: record on page PHYSICAL RECORD: n_fields 3; compact format; info bits 0
|
0: len 4; hex 0017001d; asc ;;
|
1: len 4; hex 00813e5b; asc >[;;
|
2: len 4; hex 0000002d; asc -;;
|
|
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: index records in a wrong order in `table1_fk2` of table `clientDb`.`table1`: TUPLE (info_bits=0, 2 fields): {[4] (0x0016ADF7),[4] h (0x0081680A)}, COMPACT RECORD(info_bits=0, 2 fields): {[4] (i(0x00152869),[4] (0x0080DFD3)}
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: Flagged corruption of `table1_fk2` in table `clientDb`.`table1` in CHECK TABLE-check index
|
We already restored the table on the server. This is about the crash itself.
We do not have ROLLBACK on the tables according to the binary log.
Attachments
Issue Links
- is caused by
-
MDEV-21098 Crash in rec_get_offsets_func() due to invalid rec_get_status()
-
- Closed
-
- is duplicated by
-
-
- Closed
-
Activity
#0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140206065227328) at ./nptl/pthread_kill.c:44
|
tid = <optimized out>
|
ret = 0
|
pd = 0x7f8444b69640
|
old_mask = {__val = {0, 0, 0, 94385054474889, 0, 0, 0, 0, 139577803266756, 8192, 0, 4294967296, 139577803266752, 161004825424111104, 140119101289064, 161004825424111104}}
|
ret = <optimized out>
|
pd = <optimized out>
|
old_mask = <optimized out>
|
ret = <optimized out>
|
tid = <optimized out>
|
ret = <optimized out>
|
resultvar = <optimized out>
|
resultvar = <optimized out>
|
__arg3 = <optimized out>
|
__arg2 = <optimized out>
|
__arg1 = <optimized out>
|
_a3 = <optimized out>
|
_a2 = <optimized out>
|
_a1 = <optimized out>
|
__futex = <optimized out>
|
resultvar = <optimized out>
|
__arg3 = <optimized out>
|
__arg2 = <optimized out>
|
__arg1 = <optimized out>
|
_a3 = <optimized out>
|
_a2 = <optimized out>
|
_a1 = <optimized out>
|
__futex = <optimized out>
|
__private = <optimized out>
|
__oldval = <optimized out>
|
result = <optimized out>
|
#1 __pthread_kill_internal (signo=6, threadid=140206065227328) at ./nptl/pthread_kill.c:78
|
No locals.
|
#2 __GI___pthread_kill (threadid=140206065227328, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
|
No locals.
|
#3 0x00007f8455cd0476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
|
ret = <optimized out>
|
#4 0x00007f8455cb6885 in __GI_abort () at ./stdlib/abort.c:100
|
act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {18446744073709551615, 0 <repeats 15 times>}}, sa_flags = 0, sa_restorer = 0x0}
|
sigs = {__val = {32, 140206065211276, 140206353847584, 161004825424111104, 6, 161004825424111104, 140206353848416, 140206065211472, 94385066050736, 140206352771284, 4211, 398, 94385066106632, 398, 140206065211248, 140206352207597}}
|
#5 0x000055d7bb9e50d6 in ut_dbg_assertion_failed (expr=expr@entry=0x55d7bc5584b0 "slot_rec", file=file@entry=0x55d7bc557f70 "./storage/innobase/page/page0zip.cc", line=line@entry=4211) at ./storage/innobase/ut/ut0dbg.cc:60
|
No locals.
|
#6 0x000055d7bc11bc9e in page_zip_dir_insert (cursor=cursor@entry=0x7f8444b65dd0, free_rec=free_rec@entry=0, rec=rec@entry=0x7f72200cfcc2 "", mtr=mtr@entry=0x7f8444b66620) at ./storage/innobase/page/page0zip.cc:4211
|
end = <optimized out>
|
start = <optimized out>
|
page_zip = 0x7f7005423278
|
n_dense = <optimized out>
|
slot_rec = 0x0
|
slot_free = <optimized out>
|
offs = <optimized out>
|
#7 0x000055d7bc10b635 in page_cur_insert_rec_zip (cursor=cursor@entry=0x7f8444b65dd0, rec=rec@entry=0x7eed8c0cf566 "", offsets=0x7f8444b65f40, mtr=mtr@entry=0x7f8444b66620) at ./storage/innobase/page/page0cur.cc:2138
|
page_zip = <optimized out>
|
page = <optimized out>
|
index = 0x7eef260e01c0
|
rec_size = <optimized out>
|
reorg_before_insert = <optimized out>
|
page_free_f = 44
|
page_free = <optimized out>
|
free_rec = 0
|
heap_no = 1104
|
insert_buf = <optimized out>
|
next_rec = 0x7f72200cdea0 ""
|
insert_rec = <optimized out>
|
n_recs = 0x7f72200cc036 ""
|
last_insert = <optimized out>
|
last_insert_rec = <optimized out>
|
n_owned = <optimized out>
|
#8 0x000055d7bc1c383c in page_cur_tuple_insert (mtr=0x7f8444b66620, n_ext=<optimized out>, heap=0x7f8444b65d58, offsets=0x7f8444b65d60, tuple=0x7eed8c24e810, cursor=0x7f8444b65dd0) at ./storage/innobase/include/page0cur.inl:195
|
size = <optimized out>
|
rec = 0x7eed8c0cf566 ""
|
size = <optimized out>
|
rec = <optimized out>
|
#9 btr_cur_optimistic_insert (flags=<optimized out>, cursor=0x7f8444b65dd0, offsets=offsets@entry=0x7f8444b65d60, heap=heap@entry=0x7f8444b65d58, entry=entry@entry=0x7eed8c24e810, rec=rec@entry=0x7f8444b65e60, big_rec=0x7f8444b661a0, n_ext=<optimized out>, thr=0x7eed8c2b0150, mtr=0x7f8444b66620) at ./storage/innobase/btr/btr0cur.cc:2514
|
page_cursor_rec = 0x7f72200cde68 ""
|
big_rec_vec = <optimized out>
|
index = 0x7eef260e01c0
|
page_cursor = 0x7f8444b65dd0
|
block = <optimized out>
|
page = 0x7f72200cc000 ""
|
dummy = 0x2000 <error: Cannot access memory at address 0x2000>
|
leaf = true
|
reorg = <optimized out>
|
inherit = false
|
rec_size = 14
|
err = <optimized out>
|
max_size = 272
|
n_recs = <optimized out>
|
#10 0x000055d7bc13ccfd in row_ins_sec_index_entry_low (flags=<optimized out>, mode=<optimized out>, index=0x7eef260e01c0, offsets_heap=<optimized out>, heap=<optimized out>, entry=0x7eed8c24e810, trx_id=<optimized out>, thr=<optimized out>) at ./storage/innobase/row/row0ins.cc:3134
|
insert_rec = 0x7f097a7740d8 ""
|
big_rec = 0x0
|
cursor = {page_cur = {index = 0x7eef260e01c0, rec = 0x7f72200cde68 "", offsets = 0x0, block = 0x7f7005423240}, purge_node = 0x0, thr = 0x7eed8c2b0150, flag = BTR_CUR_BINARY, tree_height = 2, up_match = 0, up_bytes = 0, low_match = 0, low_bytes = 0, n_fields = 0, n_bytes = 0, fold = 0, path_arr = 0x0, rtr_info = 0x0}
|
search_mode = <optimized out>
|
err = <optimized out>
|
n_unique = <optimized out>
|
mtr = {m_last = 0x7f7005423240, m_last_offset = 56, m_log_mode = 0, m_modifications = 1, m_made_dirty = 1, m_inside_ibuf = 0, m_trim_pages = 0, m_memo = {<small_vector_base> = {BeginX = 0x7f8444b66640, Size = 1, Capacity = 16}, small = {{object = 0x7f7005423240, type = MTR_MEMO_PAGE_X_MODIFY}, {object = 0x7f7005423240, type = MTR_MEMO_PAGE_X_FIX}, {object = 0x7f7005423240, type = MTR_MEMO_PAGE_X_FIX}, {object = 0x7f040148e580, type = MTR_MEMO_PAGE_X_FIX}, {object = 0x0, type = 0} <repeats 12 times>}}, m_log = {m_heap = 0x0, m_list = {<ilist<mtr_buf_t::block_t, void>> = {sentinel_ = {next = 0x7f8444b66768, prev = 0x7f8444b66768}}, size_ = 1}, m_size = 35, m_first_block = {<ilist_node<void>> = {next = 0x7f8444b66748, prev = 0x7f8444b66748}, m_data = "8\303\067\\,=\357\357\343\071\303\067\\,(<ʄQ\262\vO<\303\067\\,1\302\000\005\000\000\004O՚ȃ\356\002\263\000\000\330\060\006\303\067\\&\217\352\f\000\000\000\000\v\000\026\255\367\000\201g\315\000p'&\263\236\070Z0\005\303\067\\p\226\324\000\000\067\357\357\343\360\000\a\000\023\001\020\060\026\303\067\\p\221\243\201(\000\000\201g\315\000\000\000\v\200\000\000\000=\000\026\255\367\231\263\220\344\"\231\263\237~\373", '\000' <repeats 290 times>..., m_used = 35}}, m_user_space = 0x7eef895060d8, m_commit_lsn = 0, m_freed_space = 0x0, m_freed_pages = 0x0}
|
offsets_ = {300, 2, 32774, 4, 8, 12, 32644, 0, 26224, 17590, 32644, 0, 2, 0, 0, 0, 24560, 17590, 32644, 0, 61657, 48143, 21975, 0, 2, 0, 0, 0, 61568, 49344, 32612, 0, 47452, 57695, 7775, 0, 9728, 8839, 265, 572, 129, 0, 0, 0, 65088, 65535, 65535, 65535, 11, 0, 0, 0, 4584, 35844, 32493, 0, 4584, 35844, 32493, 0, 45328, 50214, 32485, 0, 26160, 17590, 32644, 0, 13395, 21971, 32644, 0, 23226, 35105, 32516, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4584, 35844, 32493, 0, 27984, 17590, 32644, 0, 44370, 48147, 21975, 0 <repeats 17 times>, 62016, 9741, 32495, 0, 29312, 18213, 32644, 0, 0, 0, 0, 0, 0, 0, 0, 0, 300, 17590, 32644, 0, 0, 0, 0, 0, 336, 35883, 32493, 0, 24720, 17590, 32644, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4584, 35844, 32493, 0, 24944, 17590, 32644, 0, 0, 0, 0, 0, 23272, 35105, 32516, 0, 62016, 9741, 32495, 0, 29770, 48146, 21975, 0, 0, 0, 0, 0, 58752, 328, 32516, 0, 0, 0, 0, 0, 336, 35883, 32493, 0, 3, 0, 0, 0, 2, 0 <repeats 39 times>...}
|
offsets = 0x7f8444b65f40
|
rtr_info = {path = 0x7eed8c2b0150, parent_path = 0x7eef260e1050, matches = 0x0, rtr_path_mutex = {m_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x7eef2610c530, __next = 0x7f105e48a7e8}}, __size = '\000' <repeats 24 times>, "\060\305\020&\357~\000\000\350\247H^\020\177\000", __align = 0}, m_psi = 0x0}, mbr = {xmin = 6.9022710314663287e-310, xmax = 0, ymin = 0, ymax = 1.4821969375237396e-323}, thr = 0x3, heap = 0x1, cursor = 0x0, index = 0x0, need_prdt_lock = false, need_page_lock = false, allocated = false, mbr_adj = false, fd_del = false, search_tuple = 0x0, search_mode = PAGE_CUR_UNSUPP}
|
#11 0x000055d7bc13f93d in row_ins_sec_index_entry (index=0x7eef260e01c0, entry=0x7eed8c24e810, thr=0x7eed8c2b0150, check_foreign=<optimized out>) at ./storage/innobase/row/row0ins.cc:3331
|
err = <optimized out>
|
offsets_heap = 0x7eed8c0cf4f8
|
heap = 0x7eed8c0f8e38
|
trx_id = <optimized out>
|
flags = 0
|
#12 0x000055d7bc13fdf2 in row_ins_index_entry (thr=0x7eed8c2b0150, entry=<optimized out>, index=<optimized out>) at ./storage/innobase/row/row0ins.cc:3379
|
No locals.
|
#13 row_ins_index_entry_step (thr=0x7eed8c2b0150, node=0x7eed8c2afea0) at ./storage/innobase/row/row0ins.cc:3545
|
err = DB_SUCCESS
|
err = <optimized out>
|
#14 row_ins (thr=0x7eed8c2b0150, node=0x7eed8c2afea0) at ./storage/innobase/row/row0ins.cc:3670
|
err = <optimized out>
|
index = <optimized out>
|
#15 row_ins_step (thr=thr@entry=0x7eed8c2b0150) at ./storage/innobase/row/row0ins.cc:3799
|
error_handling = <optimized out>
|
node = 0x7eed8c2afea0
|
parent = <optimized out>
|
sel_node = 0x0
|
trx = 0x7f8447257280
|
err = <optimized out>
|
#16 0x000055d7bc14fc19 in row_insert_for_mysql (mysql_rec=<optimized out>, prebuilt=0x7eed8c2af8c0, ins_mode=ROW_INS_NORMAL) at ./storage/innobase/row/row0mysql.cc:1315
|
run_again = <optimized out>
|
savept = {least_undo_no = 0}
|
thr = 0x7eed8c2b0150
|
err = 32644
|
was_lock_wait = <optimized out>
|
trx = 0x7f8447257280
|
node = 0x7eed8c2afea0
|
table = <optimized out>
|
blob_heap = <optimized out>
|
#17 0x000055d7bc0a28ca in ha_innobase::write_row (this=0x7eed8c25d000, record=0x7eec540831a0 "\361\315g\201") at ./storage/innobase/handler/ha_innodb.cc:7902
|
func_exit = <optimized out>
|
error = <optimized out>
|
wsrep_auto_inc_inserted = false
|
error_result = 0
|
auto_inc_used = true
|
set_stats_temporary = <optimized out>
|
trx = 0x7f8447257280
|
vers_set_fields = <optimized out>
|
#18 0x000055d7bbd95830 in handler::ha_write_row (this=0x7eed8c25d000, buf=0x7eec540831a0 "\361\315g\201") at ./sql/handler.cc:7659
|
this_tracker = 0x0
|
error = <optimized out>
|
#19 0x000055d7bbafd9ed in write_record (thd=thd@entry=0x7eed8c12b4e8, table=table@entry=0x7eed8c13ad88, info=info@entry=0x7f8444b67360, sink=sink@entry=0x0) at ./sql/sql_insert.cc:2187
|
error = <optimized out>
|
trg_error = <optimized out>
|
key = <optimized out>
|
save_read_set = <optimized out>
|
save_write_set = <optimized out>
|
prev_insert_id = 0
|
insert_id_for_cur_row = 0
|
prev_insert_id_for_cur_row = 0
|
#20 0x000055d7bbb072ea in mysql_insert (thd=thd@entry=0x7eed8c12b4e8, table_list=<optimized out>, fields=@0x7eed8c130610: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7eed8f80ccc8, last = 0x7eed8f80d228, elements = 5}, <No data fields>}, values_list=@0x7eed8c130658: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x7eed8f80da88, last = 0x7eed8f80da88, elements = 1}, <No data fields>}, update_fields=@0x7eed8c130640: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x55d7bcbafcc0 <end_of_list>, last = 0x7eed8c130640, elements = 0}, <No data fields>}, update_values=@0x7eed8c130628: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x55d7bcbafcc0 <end_of_list>, last = 0x7eed8c130628, elements = 0}, <No data fields>}, duplic=DUP_ERROR, ignore=false, result=0x0) at ./sql/sql_insert.cc:1133
|
retval = true
|
error = 0
|
res = <optimized out>
|
transactional_table = <optimized out>
|
joins_freed = false
|
changed = <optimized out>
|
was_insert_delayed = false
|
using_bulk_insert = false
|
value_count = <optimized out>
|
counter = <optimized out>
|
iteration = <optimized out>
|
id = <optimized out>
|
info = {records = 1, deleted = 0, updated = 0, copied = 0, accepted_rows = 0, error_count = 0, touched = 0, handle_duplicates = DUP_ERROR, escape_char = 0, last_errno = 0, ignore = false, update_fields = 0x7eed8c130640, update_values = 0x7eed8c130628, view = 0x0, table_list = 0x7eed8f80c428}
|
table = 0x7eed8c13ad88
|
its = {<base_list_iterator> = {list = 0x7eed8c130658, el = 0x7eed8f80da88, prev = 0x0, current = 0x0}, <No data fields>}
|
values = <optimized out>
|
context = <optimized out>
|
ctx_state = <optimized out>
|
returning = 0x0
|
readbuff = 0x0
|
query = 0x7eed8f80c160 "INSERT INTO `table1` (`fk1_id`, `fk2_id`, `fk3_id`, `start_date`, `end_date`) VALUES ('11', '61', '1486327', '2024-06-08 14:16:34', '2024-06-15 23:59:59')" (sensitive table name and fields redacted)
|
log_on = <optimized out>
|
lock_type = TL_WRITE_CONCURRENT_INSERT
|
unused_conds = 0x0
|
#21 0x000055d7bbb3bb09 in mysql_execute_command (thd=0x7eed8c12b4e8, is_called_from_prepared_stmt=<optimized out>) at ./sql/sql_parse.cc:4601
|
sel_result = 0x0
|
save_protocol = 0x0
|
multi_delete_error = <optimized out>
|
wsrep_error_label = <optimized out>
|
error = <optimized out>
|
res = <optimized out>
|
up_result = 0
|
lex = 0x7eed8c12f628
|
select_lex = <optimized out>
|
first_table = 0x7eed8f80c428
|
all_tables = 0x7eed8f80c428
|
unit = 0x7eed8c12f6f0
|
have_table_map_for_update = false
|
rpl_filter = <optimized out>
|
ots = {ctx = 0x7eed8c12f2f8, traceable = false}
|
orig_binlog_format = BINLOG_FORMAT_STMT
|
orig_current_stmt_binlog_format = BINLOG_FORMAT_STMT
|
#22 0x000055d7bbb40487 in mysql_parse (thd=0x7eed8c12b4e8, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at ./sql/sql_parse.cc:8117
|
found_semicolon = <optimized out>
|
error = <optimized out>
|
lex = 0x7eed8c12f628
|
err = <optimized out>
|
#23 0x000055d7bbb42c7d in dispatch_command (command=COM_QUERY, thd=0x7eed8c12b4e8, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at ./sql/sql_parse.cc:1995
|
beginning_of_next_stmt = <optimized out>
|
length = <optimized out>
|
packet_end = 0x7eed8f80c226 ""
|
parser_state = {m_lip = {lookahead_token = -1, lookahead_yylval = 0x0, m_thd = 0x7eed8c12b4e8, m_ptr = 0x7eed8f80c227 "\032", m_tok_start = 0x7eed8f80c227 "\032", m_tok_end = 0x7eed8f80c227 "\032", m_end_of_query = 0x7eed8f80c226 "", m_tok_start_prev = 0x7eed8f80c226 "", m_buf = 0x7eed8f80c160 "INSERT INTO `table1` (`fk1_id`, `fk2_id`, `fk3_id`, `start_date`, `end_date`) VALUES ('11', '61', '1486327', '2024-06-08 14:16:34', '2024-06-15 23:59:59')", m_buf_length = 198, m_echo = true, m_echo_saved = false, m_cpp_buf = 0x7eed8f80c298 "INSERT INTO `table1` (`fk1_id`, `fk2_id`, `fk3_id`, `start_date`, `end_date`) VALUES ('11', '61', '1486327', '2024-06-08 14:16:34', '2024-06-15 23:59:59')", m_cpp_ptr = 0x7eed8f80c35e "", m_cpp_tok_start = 0x7eed8f80c35e "", m_cpp_tok_start_prev = 0x7eed8f80c35e "", m_cpp_tok_end = 0x7eed8f80c35e "", m_body_utf8 = 0x0, m_body_utf8_ptr = 0x7f8444b685e0 "", m_cpp_utf8_processed_ptr = 0x0, next_state = MY_LEX_END, found_semicolon = 0x0, ignore_space = false, stmt_prepare_mode = false, multi_statements = true, yylineno = 1, m_digest = 0x0, in_comment = NO_COMMENT, in_comment_saved = (unknown: 0x7f84), m_cpp_text_start = 0x7eed8f80c349 "2024-06-15 23:59:59')", m_cpp_text_end = 0x7eed8f80c35c "')", m_underscore_cs = 0x0}, m_yacc = {yacc_yyss = 0x0, yacc_yyvs = 0x0, m_set_signal_info = {m_item = {0x0 <repeats 12 times>}}, m_lock_type = TL_READ_DEFAULT, m_mdl_type = MDL_SHARED_READ}, m_digest_psi = 0x0}
|
net = 0x7eed8c12b7a0
|
error = false
|
do_end_of_statement = true
|
drop_more_results = false
|
__FUNCTION__ = "dispatch_command"
|
res = <optimized out>
|
#24 0x000055d7bbb44b98 in do_command (thd=0x7eed8c12b4e8, blocking=blocking@entry=true) at ./sql/sql_parse.cc:1409
|
return_value = <optimized out>
|
packet = 0x7eed8c213ec8 "\003INSERT INTO `table1` (`fk1_id`, `fk2_id`, `fk3_id`, `start_date`, `end_date`) VALUES ('11', '61', '1486327', '2024-06-08 14:16:34', '2024-06-15 23:59:59')"
|
packet_length = <optimized out>
|
net = 0x7eed8c12b7a0
|
command = COM_QUERY
|
__FUNCTION__ = "do_command"
|
#25 0x000055d7bbc5a5af in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at ./sql/sql_connect.cc:1415
|
create_user = true
|
thr_create_utime = <optimized out>
|
thd = 0x7eed8c12b4e8
|
#26 0x000055d7bbc5a8fd in handle_one_connection (arg=arg@entry=0x55d7c02f1b18) at ./sql/sql_connect.cc:1317
|
connect = 0x55d7c02f1b18
|
#27 0x000055d7bbfe58d6 in pfs_spawn_thread (arg=0x55d7bf7cfa38) at ./storage/perfschema/pfs.cc:2201
|
typed_arg = 0x55d7bf7cfa38
|
user_arg = 0x55d7c02f1b18
|
user_start_routine = 0x55d7bbc5a8a0 <handle_one_connection(void*)>
|
pfs = <optimized out>
|
klass = <optimized out>
|
#28 0x00007f8455d22ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
|
ret = <optimized out>
|
pd = <optimized out>
|
out = <optimized out>
|
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140720360879104, 273785778019059493, 140206065227328, 11, 140206352246736, 140720360879456, -233017455094446299, -233055702713415899}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
|
not_first_call = <optimized out>
|
#29 0x00007f8455db4850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
|
No locals.
|
Can't figure out how to quote the plain text nicely.
I've got the all thread full backtrace, but there is a lot of sensitive information in it and its 37405 lines long so redacting it all would take a long time. Is there specific info I can grab out of it that can help?
Thank you. I see that the corruption is noticed during an INSERT operation. Can you execute CHECK TABLE on this table? Would it crash?
I think that in order to fix this, I’d need an rr replay execution trace using the excellent tool from https://rr-project.org. That worked for MDEV-31574 some time ago. Actually, for that bug the trick was to compile the server with cmake -DWITH_INNODB_EXTRA_DEBUG=ON to enable the #ifdef UNIV_ZIP_DEBUG checks so that the corruption could be found closer to the time when it was introduced.
I've included the check table output in the initial post.
If I can find a sequence that trigger the bug, I'll gladly do a rr replay of it, but that may take a while to happen.
ramesh reproduced this on 10.6 with rr record and rr replay. I will not have time to analyze this before I return from vacation in about 1 month.
In page_copy_rec_list_start(), we do have the following wrong check:
/* Before copying, "ret" was the predecessor
|
of the predefined supremum record. If it was
|
the predefined infimum record, then it would
|
still be the infimum, and we would have
|
ret_pos == 0. */
|
if (UNIV_UNLIKELY(!ret_pos
|
|| ret_pos == ULINT_UNDEFINED)) {
|
*err = DB_CORRUPTION;
|
return nullptr;
|
}
|
|
InnoDB splits the root page and moves the record from old page to new page. InnoDB finds the split record and copies the record from first record to
split record in new block. In that function, ret stores the initial position from where we start to insert in the new block.
Since it is new block, ret value is 0. page_zip_compress can fail and maybe we do have wrong assert saying ret_pos shouldn't be zero.
I have to debug more why page_zip_compress() fails even though we have less records than original page.
Thank you for the report. The built-in stack trace report that you posted does not include any InnoDB function names. A resolved stack trace of the crashing thread, or better yet, all threads that existed at the time of the crash, would help a little.
There is a known corruption bug
MDEV-32174that affects transaction rollbacks on ROW_FORMAT=COMPRESSED tables. Unfortunately, it has not been reproduced in a way that it could be diagnosed. You should be aware that transactions can be rolled back for other reasons than an explicit ROLLBACK statement. These reasons would include deadlocks between transactions, lock wait timeouts, duplicate key errors and foreign key violations.