Details
Description
We received the first error of table corruption at
2024-06-08 1:31:13 1018409755 [ERROR] mariadbd: Index for table 'table1' is corrupt; try to repair it
The this happened when a query tried to access the table:
2024-06-08 14:16:34 0x7f8444b69640 InnoDB: Assertion failure in file ./storage/innobase/page/page0zip.cc line 4211
|
InnoDB: Failing assertion: slot_rec
|
InnoDB: We intentionally generate a memory trap.
|
InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
|
InnoDB: If you get repeated assertion failures or crashes, even
|
InnoDB: immediately after the mariadbd startup, there may be
|
InnoDB: corruption in the InnoDB tablespace. Please refer to
|
InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
|
InnoDB: about forcing recovery.
|
240608 14:16:34 [ERROR] mysqld got signal 6 ;
|
Sorry, we probably made a mistake, and this is a bug.
|
|
|
Your assistance in bug reporting will enable us to fix this for the next release.
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.6.17-MariaDB-1:10.6.17+maria~ubu2204-log source revision: 15c75ad083a55e198ae78324f22970694b72f22b
|
key_buffer_size=16777216
|
read_buffer_size=131072
|
max_used_connections=1272
|
max_threads=10002
|
thread_count=400
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 22044434 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7eed8c12b4e8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f8444b68c38 thread_stack 0x40000
|
/usr/sbin/mariadbd(my_print_stacktrace+0x32)[0x55d7bc2dabe2]
|
/usr/sbin/mariadbd(handle_fatal_signal+0x478)[0x55d7bbd862c8]
|
libc_sigaction.c:0(__restore_rt)[0x7f8455cd0520]
|
nptl/pthread_kill.c:44(__pthread_kill_implementation)[0x7f8455d249fc]
|
posix/raise.c:27(__GI_raise)[0x7f8455cd0476]
|
stdlib/abort.c:81(__GI_abort)[0x7f8455cb67f3]
|
/usr/sbin/mariadbd(+0x67c0d6)[0x55d7bb9e50d6]
|
/usr/sbin/mariadbd(+0xdb2c9e)[0x55d7bc11bc9e]
|
/usr/sbin/mariadbd(+0xda2635)[0x55d7bc10b635]
|
/usr/sbin/mariadbd(+0xe5a83c)[0x55d7bc1c383c]
|
/usr/sbin/mariadbd(+0xdd3cfd)[0x55d7bc13ccfd]
|
/usr/sbin/mariadbd(+0xdd693d)[0x55d7bc13f93d]
|
/usr/sbin/mariadbd(+0xdd6df2)[0x55d7bc13fdf2]
|
/usr/sbin/mariadbd(+0xde6c19)[0x55d7bc14fc19]
|
/usr/sbin/mariadbd(+0xd398ca)[0x55d7bc0a28ca]
|
/usr/sbin/mariadbd(_ZN7handler12ha_write_rowEPKh+0x1f0)[0x55d7bbd95830]
|
/usr/sbin/mariadbd(_Z12write_recordP3THDP5TABLEP12st_copy_infoP13select_result+0x1dd)[0x55d7bbafd9ed]
|
/usr/sbin/mariadbd(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesbP13select_result+0xd3a)[0x55d7bbb072ea]
|
/usr/sbin/mariadbd(_Z21mysql_execute_commandP3THDb+0x14c9)[0x55d7bbb3bb09]
|
/usr/sbin/mariadbd(_Z11mysql_parseP3THDPcjP12Parser_state+0x1e7)[0x55d7bbb40487]
|
/usr/sbin/mariadbd(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x14cd)[0x55d7bbb42c7d]
|
/usr/sbin/mariadbd(_Z10do_commandP3THDb+0x138)[0x55d7bbb44b98]
|
/usr/sbin/mariadbd(_Z24do_handle_one_connectionP7CONNECTb+0x3bf)[0x55d7bbc5a5af]
|
/usr/sbin/mariadbd(handle_one_connection+0x5d)[0x55d7bbc5a8fd]
|
/usr/sbin/mariadbd(+0xc7c8d6)[0x55d7bbfe58d6]
|
nptl/pthread_create.c:442(start_thread)[0x7f8455d22ac3]
|
x86_64/clone3.S:83(__clone3)[0x7f8455db4850]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x7eed8f80c160): INSERT INTO `table1`(`fk1_id`, `fk2_id`, `fk3_id`, `start_date`, `end_date`) VALUES ('11', '61', '1486327', '2024-06-08 14:16:34', '2024-06-15 23:59:59')
|
|
|
Connection ID (thread ID): 1023439276
|
Status: NOT_KILLED
|
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off
|
,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,p
|
artial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_ca
|
che_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derive
|
d=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off,cset_narr
|
owing=off
|
|
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mariadbd/ contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 4126213 4126213 processes
|
Max open files 32768 32768 files
|
Max locked memory 524288 524288 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 4126213 4126213 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: |/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E
|
|
|
|
|
Kernel version: Linux version 5.15.0-97-generic (buildd@lcy02-amd64-033) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #107-Ubuntu SMP
|
Wed Feb 7 13:26:48 UTC 2024
|
We have the file that apport created. It is 13gb long and it seems to contain the core dump. This is from a production system and will contain sensitive data, so we can't just upload it. If there are specific requests / commands, we can run them on the core file.
CREATE TABLE `table1` ( |
`table1_id` int(10) unsigned NOT NULL AUTO_INCREMENT, |
`fk1_id` int(10) unsigned DEFAULT NULL, |
`used` tinyint(1) NOT NULL DEFAULT 0, |
`fk2_id` int(10) unsigned DEFAULT NULL, |
`fk3_id` int(10) unsigned DEFAULT NULL, |
`start_date` datetime NOT NULL, |
`end_date` datetime NOT NULL, |
PRIMARY KEY (`table1_id`), |
UNIQUE KEY `table1_uk` (`fk1_id`,`fk3_id`), |
KEY `table1_fk2` (`fk3_id`), |
KEY `table1_fk3` (`fk2_id`), |
CONSTRAINT `table1_ibfk_1` FOREIGN KEY (`fk1_id`) REFERENCES `table_for_fk1` (`fk1_id`), |
CONSTRAINT `table1_ibfk_2` FOREIGN KEY (`fk3_id`) REFERENCES `table_for_fk3` (`fk3_id`), |
CONSTRAINT `table1_ibfk_3` FOREIGN KEY (`fk2_id`) REFERENCES `table_for_fk2` (`fk2_id`) |
) ENGINE=InnoDB AUTO_INCREMENT=8505097 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=COMPRESSED |
Data was successfully inserted into the affected table between the corruption log entry at 1:31:13 and the assertion.
CHECK TABLE produced this log output:
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: In pages [page id: space=227292, page number=44] and [page id: space=227292, page number=45] of index `table1_fk2` of table `clientDb`.`table1`
|
InnoDB: records in wrong order on adjacent pages
|
InnoDB: record PHYSICAL RECORD: n_fields 2; compact format; info bits 0
|
0: len 4; hex 0016adf7; asc ;;
|
1: len 4; hex 0081680a; asc h ;;
|
|
|
InnoDB: record PHYSICAL RECORD: n_fields 2; compact format; info bits 0
|
0: len 4; hex 00152869; asc (i;;
|
1: len 4; hex 0080dfd3; asc ;;
|
|
|
InnoDB: node pointer to the right page is wrong
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: In page 44 of index `table1_fk2` of table `clientDb`.`table1`
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: In page 45 of index `table1_fk2` of table `clientDb`.`table1`
|
InnoDB: node pointer to the page is wrong
|
InnoDB: node ptr PHYSICAL RECORD: n_fields 3; compact format; info bits 0
|
0: len 4; hex 00151fef; asc ;;
|
1: len 4; hex 006bb459; asc k Y;;
|
2: len 4; hex 0000002c; asc ,;;
|
|
|
InnoDB: node ptr child page n:o 45
|
InnoDB: record on page PHYSICAL RECORD: n_fields 3; compact format; info bits 0
|
0: len 4; hex 0017001d; asc ;;
|
1: len 4; hex 00813e5b; asc >[;;
|
2: len 4; hex 0000002d; asc -;;
|
|
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: index records in a wrong order in `table1_fk2` of table `clientDb`.`table1`: TUPLE (info_bits=0, 2 fields): {[4] (0x0016ADF7),[4] h (0x0081680A)}, COMPACT RECORD(info_bits=0, 2 fields): {[4] (i(0x00152869),[4] (0x0080DFD3)}
|
2024-06-08 18:31:53 3022967 [ERROR] InnoDB: Flagged corruption of `table1_fk2` in table `clientDb`.`table1` in CHECK TABLE-check index
|
We already restored the table on the server. This is about the crash itself.
We do not have ROLLBACK on the tables according to the binary log.
Attachments
Issue Links
- is caused by
-
MDEV-21098 Crash in rec_get_offsets_func() due to invalid rec_get_status()
-
- Closed
-
- is duplicated by
-
-
- Closed
-