Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Duplicate
-
10.5.15
-
CentOS 7
Description
This happened yesterday:
mariadbd: 2022-03-21 13:15:30 13094 [ERROR] [FATAL] InnoDB: Rec offset 99, cur1 offset 8485, cur2 offset 15780 |
mariadbd: 220321 13:15:30 [ERROR] mysqld got signal 6 ; |
mariadbd: This could be because you hit a bug. It is also possible that this binary |
mariadbd: or one of the libraries it was linked against is corrupt, improperly built,
|
mariadbd: or misconfigured. This error can also be caused by malfunctioning hardware.
|
mariadbd: To report this bug, see https://mariadb.com/kb/en/reporting-bugs |
mariadbd: We will try our best to scrape up some info that will hopefully help |
mariadbd: diagnose the problem, but since we have already crashed,
|
mariadbd: something is definitely wrong and this may fail. |
mariadbd: Server version: 10.5.15-MariaDB-log |
mariadbd: key_buffer_size=134217728 |
mariadbd: read_buffer_size=131072 |
mariadbd: max_used_connections=4 |
mariadbd: max_threads=153 |
mariadbd: thread_count=3 |
mariadbd: It is possible that mysqld could use up to
|
mariadbd: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467872 K bytes of memory |
mariadbd: Hope that's ok; if not, decrease some variables in the equation. |
mariadbd: Thread pointer: 0x7f42980029a8 |
mariadbd: Attempting backtrace. You can use the following information to find out
|
mariadbd: where mysqld died. If you see no messages after this, something went |
mariadbd: terribly wrong...
|
mariadbd: stack_bottom = 0x7f446861a3e0 thread_stack 0x49000 |
mariadbd: ??:0(my_print_stacktrace)[0x55a751fffe3e] |
mariadbd: ??:0(handle_fatal_signal)[0x55a7519f19c7] |
mariadbd: sigaction.c:0(__restore_rt)[0x7f448224d630] |
mariadbd: :0(__GI_raise)[0x7f4481698387] |
mariadbd: :0(__GI_abort)[0x7f4481699a78] |
mariadbd: ??:0(std::pair<std::_Rb_tree_iterator<unsigned int>, bool> std::_Rb_tree<unsigned int, unsigned int, std::_Identity<unsigned int>, std::less<unsigned int>, std::allocator<unsigned int> >::_M_insert_unique<unsigned int const&>(unsigned int const&))[0x55a751e568b0] |
mariadbd: ??:0(std::unique_lock<std::mutex>::unlock())[0x55a751d71579] |
mariadbd: ??:0(std::pair<std::_Rb_tree_iterator<unsigned int>, bool> std::_Rb_tree<unsigned int, unsigned int, std::_Identity<unsigned int>, std::less<unsigned int>, std::allocator<unsigned int> >::_M_insert_unique<unsigned int const&>(unsigned int const&))[0x55a751e5ce86] |
mariadbd: ??:0(std::pair<std::_Rb_tree_iterator<unsigned int>, bool> std::_Rb_tree<unsigned int, unsigned int, std::_Identity<unsigned int>, std::less<unsigned int>, std::allocator<unsigned int> >::_M_insert_unique<unsigned int const&>(unsigned int const&))[0x55a751e5dab7] |
mariadbd: ??:0(std::pair<std::_Rb_tree_iterator<unsigned int>, bool> std::_Rb_tree<unsigned int, unsigned int, std::_Identity<unsigned int>, std::less<unsigned int>, std::allocator<unsigned int> >::_M_insert_unique<unsigned int const&>(unsigned int const&))[0x55a751e7c63c] |
mariadbd: ??:0(void std::__introsort_loop<unsigned char**, long>(unsigned char**, unsigned char**, long))[0x55a751db5c46] |
mariadbd: ??:0(void std::__introsort_loop<unsigned char**, long>(unsigned char**, unsigned char**, long))[0x55a751db6932] |
mariadbd: ??:0(void std::__introsort_loop<unsigned char**, long>(unsigned char**, unsigned char**, long))[0x55a751db6e2a] |
mariadbd: ??:0(void std::__introsort_loop<unsigned char**, long>(unsigned char**, unsigned char**, long))[0x55a751dc9a14] |
mariadbd: ??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x55a751cf6d94] |
mariadbd: ??:0(handler::ha_write_row(unsigned char const*))[0x55a7519feb4f] |
mariadbd: ??:0(Rows_log_event::write_row(rpl_group_info*, bool))[0x55a751b0c2f5] |
mariadbd: ??:0(Write_rows_log_event::do_exec_row(rpl_group_info*))[0x55a751b0c75d] |
mariadbd: ??:0(Rows_log_event::do_apply_event(rpl_group_info*))[0x55a751b01b84] |
mariadbd: ??:0(non-virtual thunk to Item_string_sys::~Item_string_sys())[0x55a75172f65f] |
mariadbd: ??:0(handle_slave_sql)[0x55a75173984c] |
mariadbd: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55a751c471b7] |
mariadbd: pthread_create.c:0(start_thread)[0x7f4482245ea5] |
mariadbd: ??:0(__clone)[0x7f4481760b0d] |
mariadbd: Trying to get some variables.
|
mariadbd: Some pointers may be invalid and cause the dump to abort.
|
mariadbd: Query (0x7f42980a540b): INSERT INTO ... |
mariadbd: Status: NOT_KILLED
|
mariadbd: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=of |
f,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
|
mariadbd: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains |
mariadbd: information that should help you find out what is causing the crash.
|
mariadbd: Writing a core file...
|
mariadbd: Working directory at /var/lib/mysql
|
mariadbd: Resource Limits:
|
mariadbd: Limit Soft Limit Hard Limit Units
|
mariadbd: Max cpu time unlimited unlimited seconds
|
mariadbd: Max file size unlimited unlimited bytes
|
mariadbd: Max data size unlimited unlimited bytes
|
mariadbd: Max stack size 8388608 unlimited bytes |
mariadbd: Max core file size 0 unlimited bytes |
mariadbd: Max resident set unlimited unlimited bytes
|
mariadbd: Max processes 31150 31150 processes |
mariadbd: Max open files 32768 32768 files |
mariadbd: Max locked memory 65536 65536 bytes |
mariadbd: Max address space unlimited unlimited bytes
|
mariadbd: Max file locks unlimited unlimited locks
|
mariadbd: Max pending signals 31150 31150 signals |
mariadbd: Max msgqueue size 819200 819200 bytes |
mariadbd: Max nice priority 0 0 |
mariadbd: Max realtime priority 0 0 |
mariadbd: Max realtime timeout unlimited unlimited us
|
mariadbd: Core pattern: |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h
|
abrt-hook-ccpp: Process 9433 (mariadbd) of user 987 killed by SIGABRT - dumping core |
abrtd: Size of '/var/spool/abrt' >= 5000 MB (MaxCrashReportsSize), deleting old directory 'xorg-2019-01-19-08:12:24-6238-1' |
systemd: mariadb.service: main process exited, code=killed, status=6/ABRT |
systemd: Unit mariadb.service entered failed state.
|
systemd: mariadb.service failed.
|
abrtd: Size of '/var/spool/abrt' >= 5000 MB (MaxCrashReportsSize), deleting old directory 'xorg-2019-02-23-08:17:59-6276-1' |
abrtd: Size of '/var/spool/abrt' >= 5000 MB (MaxCrashReportsSize), deleting old directory 'oops-2017-09-13-18:37:14-4564-0' |
abrtd: Size of '/var/spool/abrt' >= 5000 MB (MaxCrashReportsSize), deleting old directory 'oops-2017-09-13-18:36:34-4511-0' |
abrtd: Size of '/var/spool/abrt' >= 5000 MB (MaxCrashReportsSize), deleting old directory 'Python-2017-09-13-17:53:58-1093' |
abrt-server: Package 'MariaDB-server' isn't signed with proper key |
abrt-server: 'post-create' on '/var/spool/abrt/ccpp-2022-03-21-13:15:30-9433' exited with 1 |
abrt-server: Deleting problem directory '/var/spool/abrt/ccpp-2022-03-21-13:15:30-9433' |
systemd: mariadb.service holdoff time over, scheduling restart.
|
systemd: Cannot add dependency job for unit firewalld.service, ignoring: Unit is masked. |
systemd: Stopped MariaDB 10.5.15 database server. |
I was able to get server back running by starting it with innodb_force_recovery = 1, dropping databases and finally recreating them from another slave.
Attachments
Issue Links
- duplicates
-
MDEV-13542 Crashing on a corrupted page is unhelpful
- Closed
-
MDEV-22739 !cursor->index->is_committed() in row0ins.cc after update to 10.4.13 from 10.3.21
- Closed
- relates to
-
MDEV-26977 mariadb 10.5.12 reboot loop in AWS | [ERROR] InnoDB: Insert buffer insert fails | Corruption of an index tree
- Closed
-
MDEV-27734 Set innodb_change_buffering=none by default
- Closed