[MDEV-9556] mysqld got signal 11 Created: 2016-02-14  Updated: 2016-03-15  Resolved: 2016-03-15

Status: Closed
Project: MariaDB Server
Component/s: OTHER
Affects Version/s: 5.5.47
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: olabi Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback
Environment:

centos 6.6 64bit , kernel 2.6.32-431.20.3.el6.x86_64 , 1 cpu 6 core , ram 16g , 1.1T external storage MSA 2040 FC



 Description   

Dears,
after 53 days up time our database going down without any reason

mysqld --print-defaults

mysqld would have been started with the following arguments:
--datadir=/var/lib/mysql --tmpdir=/var/lib/tmp --socket=/var/lib/mysql/mysql.sock --pid_file=/var/lib/mysql/mysql.pid --user=mysql --port=3306 --max_connections=1500 --default_storage_engine=innodb --innodb_buffer_pool_size=4G --innodb_log_file_size=64M --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --innodb_thread_concurrency=6 --innodb_autoinc_lock_mode=2 --key_buffer_size=512M --log_error=/var/log/mysqld.err --log_slow_queries=/var/log/mysql/mysql-slow.log --long_query_time=5 --server-id=1 --binlog_format=ROW --relay-log=mysql-relay-bin --log-slave-updates=1 --sync_binlog=1 --read_only=0 --skip_slave_start --slave-skip-errors=1062,1054,1146,1452,1580,1032,1050,1235,1236,1053,1051 --log-bin=/var/lib/mysql/mysql-bin --log-bin-index=/var/lib/mysql/mysql-bin.index --expire-logs-days=5 --tmp_table_size=32M --max_heap_table_size=32M --query_cache_type=0 --query_cache_size=0 --max_connections=1500 --thread_cache_size=8 --open_files_limit=65535 --thread_concurrency=12 --table_open_cache=15000 --read_buffer_size=4M --read_rnd_buffer_size=16M --interactive_timeout=1200 --wait_timeout=1200 --plugin-load=ha_tokudb.so 

Database down with the following error :

160211 17:50:12 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 5.5.47-MariaDB-log
key_buffer_size=536870912
read_buffer_size=4194304
max_used_connections=889
max_threads=1502
thread_count=309
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 9779359 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0x7f2d5adba000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f2d97f23d60 thread_stack 0x48000
/usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xa75aeb]
/usr/sbin/mysqld(handle_fatal_signal+0x398)[0x6d15f8]
/lib64/libpthread.so.0[0x3b0ec0f710]
/usr/sbin/mysqld[0x5be4fd]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x166)[0x5be946]
/usr/sbin/mysqld[0x5be6b2]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x166)[0x5be946]
/usr/sbin/mysqld[0x5be6b2]
/usr/sbin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x1c9)[0x5be9a9]
/usr/sbin/mysqld[0x5d5a3d]
/usr/sbin/mysqld(_ZN4JOIN4execEv+0xa0b)[0x5eaffb]
/usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x125)[0x5e6385]
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x2c3)[0x5ecfc3]
/usr/sbin/mysqld[0x5979a6]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x34bc)[0x5a0ffc]
/usr/sbin/mysqld[0x5a38d4]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1bf8)[0x5a5da8]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x21b)[0x656b2b]
/usr/sbin/mysqld(handle_one_connection+0x4c)[0x656bac]
/lib64/libpthread.so.0[0x3b0ec079d1]
/lib64/libc.so.6(clone+0x6d)[0x3b0e8e88fd]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f2d60beb018): is an invalid pointer
Connection ID (thread ID): 26984798
Status: NOT_KILLED



 Comments   
Comment by Elena Stepanova [ 2016-02-15 ]

Unfortunately, there is nothing to work with here. The readable part of the stack trace is very unspecific, server options show that you don't have general log enabled, and since it happened after 53 days, I assume it's not reproducible easily.

I don't expect you know what kind of query the connection 26984798 could be executing, do you?

If you can enable the general log in case it crashes again, it would be big help. For now we can suspect it's related to nearly any known bug, e.g. MDEV-9304.

Comment by Elena Stepanova [ 2016-03-15 ]

Please comment to re-open if you have more information on the issue.

Generated at Thu Feb 08 07:35:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.