Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Incomplete
-
10.4.13
-
None
-
CentOS 7.8 x86_64
Description
Today on of our galera members crash's on the start up. The second start up trigged by systemd was successfully.
Here the logs:
systemd:
2020-06-30T06:39:03+0200 sl-it-t-galera2 systemd[1]: Started MariaDB 10.4.13 database server.
|
2020-06-30T06:39:06+0200 sl-it-t-galera2 systemd[1]: mariadb.service: main process exited, code=killed, status=6/ABRT
|
2020-06-30T06:39:06+0200 sl-it-t-galera2 systemd[1]: Unit mariadb.service entered failed state.
|
2020-06-30T06:39:06+0200 sl-it-t-galera2 systemd[1]: mariadb.service failed.
|
2020-06-30T06:39:11+0200 sl-it-t-galera2 systemd[1]: mariadb.service holdoff time over, scheduling restart.
|
2020-06-30T06:39:11+0200 sl-it-t-galera2 systemd[1]: Stopped MariaDB 10.4.13 database server.
|
2020-06-30T06:39:11+0200 sl-it-t-galera2 systemd[1]: Starting MariaDB 10.4.13 database server...
|
MariaDB:
2020-06-30 6:39:03 2 [Warning] WSREP: server: sl-it-t-galera2 unallowed state transition: joined -> joiner
|
|
mysqld: /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.4.13/wsrep-lib/src/server_state.cpp:1356: void wsrep::server_state::state(wsrep::unique_lock<wsrep::mutex>&, wsrep::server_state::state): Assertion `0' failed.
|
200630 6:39:03 [ERROR] mysqld got signal 6 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
Server version: 10.4.13-MariaDB
|
key_buffer_size=134217728
|
read_buffer_size=131072
|
max_used_connections=0
|
max_threads=153
|
thread_count=8
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467752 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
Thread pointer: 0x7f87d80009a8
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7f87e9ff9d20 thread_stack 0x49000
|
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55b73429a6ce]
|
/usr/sbin/mysqld(handle_fatal_signal+0x30f)[0x55b733d30f7f]
|
sigaction.c:0(__restore_rt)[0x7f87fea6f630]
|
:0(__GI_raise)[0x7f87fcd40387]
|
:0(__GI_abort)[0x7f87fcd41a78]
|
:0(__assert_fail_base)[0x7f87fcd391a6]
|
:0(__GI___assert_fail)[0x7f87fcd39252]
|
/usr/sbin/mysqld(_ZN5wsrep12server_state5stateERNS_11unique_lockINS_5mutexEEENS0_5stateE+0x642)[0x55b734328642]
|
/usr/sbin/mysqld(_ZN5wsrep12server_state15prepare_for_sstEv+0x3d)[0x55b73432881d]
|
/usr/sbin/mysqld(+0xeb651b)[0x55b73433751b]
|
/usr/lib64/galera-4/libgalera_smm.so(+0x1a858b)[0x7f87f951558b]
|
/usr/lib64/galera-4/libgalera_smm.so(+0x1a9081)[0x7f87f9516081]
|
/usr/lib64/galera-4/libgalera_smm.so(+0x1a93de)[0x7f87f95163de]
|
/usr/lib64/galera-4/libgalera_smm.so(+0x180a40)[0x7f87f94eda40]
|
src/replicator_smm.cpp:2644(galera::ReplicatorSMM::process_st_required(void*, int, wsrep_view_info const*))[0x7f87f94edbee]
|
src/replicator_smm.cpp:2834(operator())[0x7f87f9514afb]
|
src/gcs_action_source.cpp:155(galera::GcsActionSource::dispatch(void*, gcs_action const&, bool&))[0x7f87f9527518]
|
/usr/sbin/mysqld(_ZN5wsrep18wsrep_provider_v2611run_applierEPNS_21high_priority_serviceE+0xe)[0x55b73433692e]
|
/usr/sbin/mysqld(+0x825e42)[0x55b733ca6e42]
|
/usr/sbin/mysqld(_Z15start_wsrep_THDPv+0x35d)[0x55b733c9b72d]
|
pthread_create.c:0(start_thread)[0x7f87fea67ea5]
|
/lib64/libc.so.6(clone+0x6d)[0x7f87fce088dd]
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x0):
|
Connection ID (thread ID): 2
|
Status: NOT_KILLED
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on
|
|
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
|
information that should help you find out what is causing the crash.
|
Writing a core file...
|
Working directory at /var/lib/mysql
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 31119 31119 processes
|
Max open files 33000 33000 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 31119 31119 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
2020-06-30 6:39:14 0 [Note] WSREP: Loading provider /usr/lib64/galera-4/libgalera_smm.so initial position: b7cd80dc-3e9a-11ea-8787-073665549b8d:6722
|
wsrep loader: [INFO] wsrep_load(): loading provider library '/usr/lib64/galera-4/libgalera_smm.so'
|
wsrep loader: [INFO] wsrep_load(): Galera 26.4.4(r4599) by Codership Oy <info@codership.com> loaded successfully.
|
2020-06-30 6:39:14 0 [Note] WSREP: CRC-32C: using hardware acceleration.
|
2020-06-30 6:39:14 0 [Note] WSREP: disabling SSL compression
|
2020-06-30 6:39:14 0 [Note] WSREP: SSL cipher list set to 'ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384'
|
2020-06-30 6:39:14 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 0
|
2020-06-30 6:39:14 0 [Note] WSREP: GCache DEBUG: opened preamble:
|
Version: 2
|
UUID: b7cd80dc-3e9a-11ea-8787-073665549b8d
|
Seqno: -1 - -1
|
Offset: -1
|
Synced: 0
|
I have attached the abrt catch (which contains the core dump) of the os as abrt.tar.xz
Attachments
Activity
Transition | Time In Source Status | Execution Times |
---|
|
541d 1h 33m | 1 |
|
31d 14h 16m | 1 |