[MDEV-9901] Mariadb crashes Created: 2016-04-12  Updated: 2017-12-13  Resolved: 2017-12-13

Status: Closed
Project: MariaDB Server
Component/s: Galera, Replication
Affects Version/s: 10.1.13
Fix Version/s: 10.1.30

Type: Bug Priority: Major
Reporter: André Bauer Assignee: Andrei Elkin
Resolution: Duplicate Votes: 0
Labels: galera
Environment:

Ubuntu 14.04


Issue Links:
Duplicate
duplicates MDEV-9510 Segmentation fault in binlog thread c... Closed

 Description   

From time to time one of the nodes of our 3 node Galera Cluster crashes.
After puppet restarts the crashed node, everything is working again until next crash.
The problem exists since Mariadb 10.0.x.

This is what the log says:

Apr 12 06:53:57 mysql1 kernel: [11218669.570088] systemd-udevd[21154]: starting version 204
Apr 12 06:54:19 mysql1 kernel: [11218690.738200] init: systemd-logind main process (6541) killed by TERM signal
Apr 12 06:54:30 mysql1 mysqld: 160412  6:54:30 [ERROR] mysqld got signal 11 ;
Apr 12 06:54:30 mysql1 mysqld: This could be because you hit a bug. It is also possible that this binary
Apr 12 06:54:30 mysql1 mysqld: or one of the libraries it was linked against is corrupt, improperly built,
Apr 12 06:54:30 mysql1 mysqld: or misconfigured. This error can also be caused by malfunctioning hardware.
Apr 12 06:54:30 mysql1 mysqld: 
Apr 12 06:54:30 mysql1 mysqld: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
Apr 12 06:54:30 mysql1 mysqld: 
Apr 12 06:54:30 mysql1 mysqld: We will try our best to scrape up some info that will hopefully help
Apr 12 06:54:30 mysql1 mysqld: diagnose the problem, but since we have already crashed, 
Apr 12 06:54:30 mysql1 mysqld: something is definitely wrong and this may fail.
Apr 12 06:54:30 mysql1 mysqld: 
Apr 12 06:54:30 mysql1 mysqld: Server version: 10.1.13-MariaDB-1~trusty
Apr 12 06:54:30 mysql1 mysqld: key_buffer_size=134217728
Apr 12 06:54:30 mysql1 mysqld: read_buffer_size=2097152
Apr 12 06:54:30 mysql1 mysqld: max_used_connections=10
Apr 12 06:54:30 mysql1 mysqld: max_threads=52
Apr 12 06:54:30 mysql1 mysqld: thread_count=27
Apr 12 06:54:30 mysql1 mysqld: It is possible that mysqld could use up to 
Apr 12 06:54:30 mysql1 mysqld: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1942548 K  bytes of memory
Apr 12 06:54:30 mysql1 mysqld: Hope that's ok; if not, decrease some variables in the equation.
Apr 12 06:54:30 mysql1 mysqld: 
Apr 12 06:54:30 mysql1 mysqld: Thread pointer: 0x0x7ff0345a0008
Apr 12 06:54:30 mysql1 mysqld: Attempting backtrace. You can use the following information to find out
Apr 12 06:54:30 mysql1 mysqld: where mysqld died. If you see no messages after this, something went
Apr 12 06:54:30 mysql1 mysqld: terribly wrong...
Apr 12 06:54:30 mysql1 mysqld: stack_bottom = 0x7ff38d0851f0 thread_stack 0x48400
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x7ff395f3fb6e]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(handle_fatal_signal+0x38d)[0x7ff395a6c48d]
Apr 12 06:54:30 mysql1 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7ff393fbf340]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG21do_checkpoint_requestEm+0x9d)[0x7ff395b2dafd]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG20checkpoint_and_purgeEm+0x11)[0x7ff395b2db91]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG16rotate_and_purgeEb+0xc2)[0x7ff395b300b2]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_Z20reload_acl_and_cacheP3THDyP10TABLE_LISTPi+0x12c)[0x7ff3959d504c]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1334)[0x7ff3958eb994]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x26d)[0x7ff3958f3f4d]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(+0x438759)[0x7ff3958f4759]
Apr 12 06:54:30 mysql1 mysqld: /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1e2b)[0x7ff3958f6c5b]
Apr 12 06:54:31 mysql1 mysqld: /usr/sbin/mysqld(_Z10do_commandP3THD+0x169)[0x7ff3958f7a49]
Apr 12 06:54:31 mysql1 mysqld: /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x7ff3959baf5a]
Apr 12 06:54:31 mysql1 mysqld: /usr/sbin/mysqld(handle_one_connection+0x40)[0x7ff3959bb130]
Apr 12 06:54:31 mysql1 mysqld: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x7ff393fb7182]
Apr 12 06:54:31 mysql1 mysqld: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7ff3934d647d]
Apr 12 06:54:31 mysql1 mysqld: 
Apr 12 06:54:31 mysql1 mysqld: Trying to get some variables.
Apr 12 06:54:31 mysql1 mysqld: Some pointers may be invalid and cause the dump to abort.
Apr 12 06:54:31 mysql1 mysqld: Query (0x7ff02f0da020): is an invalid pointer
Apr 12 06:54:31 mysql1 mysqld: Connection ID (thread ID): 918
Apr 12 06:54:31 mysql1 mysqld: Status: NOT_KILLED
Apr 12 06:54:31 mysql1 mysqld: 
Apr 12 06:54:31 mysql1 mysqld: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
Apr 12 06:54:31 mysql1 mysqld: 
Apr 12 06:54:31 mysql1 mysqld: The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
Apr 12 06:54:31 mysql1 mysqld: information that should help you find out what is causing the crash.
Apr 12 06:54:31 mysql1 mysqld_safe: Number of processes running now: 0
Apr 12 06:54:31 mysql1 mysqld_safe: WSREP: not restarting wsrep node automatically
Apr 12 06:54:31 mysql1 mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended



 Comments   
Comment by Elena Stepanova [ 2016-04-15 ]

nirbhay_c, does it look anyhow familiar?

Comment by Andrei Elkin [ 2017-11-07 ]

According to the stack MDEV-9510 is the parent.

Generated at Thu Feb 08 07:38:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.