[MDEV-8333] mysqld got signal 11 Created: 2015-06-18  Updated: 2015-06-29  Resolved: 2015-06-29

Status: Closed
Project: MariaDB Server
Component/s: OTHER
Affects Version/s: 10.1.5
Fix Version/s: 10.1.6

Type: Bug Priority: Critical
Reporter: Cyberagent technik Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 0
Labels: None
Environment:

Debian Jessie
Linux version 3.16.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24)
CPU: 12 x Intel(R) Xeon(R) CPU E5-1650 v2 @ 3.50GHz


Issue Links:
Relates
relates to MDEV-8339 Server crash during table cache eviction Closed
relates to MDEV-8370 mysqld got signal 11 with aggressive ... Closed

 Description   

MariaDB is running as as slave, currently with very little activity.
At the time of the crash there was a script running doing a "check table" on all tables in all databases. There was no other client connected at this time.
The server failure cannot be repeted. Running th script a second and third time was fine.
Server crashed with message:

150618  1:50:07 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
 
Server version: 10.1.5-MariaDB-log
key_buffer_size=67108864
read_buffer_size=131072
max_used_connections=5
max_threads=502
thread_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1168091 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x0x7fef8fefb008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fef90491e58 thread_stack 0x48000
mysys/stacktrace.c:247(my_print_stacktrace)[0x7fef931dcbf9]
sql/signal_handler.cc:153(handle_fatal_signal)[0x7fef92def98d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0)[0x7fef924498d0]
sql/sql_plist.h:181(I_P_List_iterator)[0x7fef92da6360]
mysys/lf_hash.c:125(lfind)[0x7fef931e3a5c]
include/my_atomic.h:281(my_atomic_storeptr)[0x7fef931e4459]
sql/table_cache.cc:1084(tdc_iterate(THD*, char (*)(void*, void*), void*, bool))[0x7fef92da7fb3]
sql/table_cache.cc:267(tc_add_table(THD*, TABLE*))[0x7fef92da833a]
sql/sql_base.cc:2572(open_table(THD*, TABLE_LIST*, Open_table_context*))[0x7fef92c42f0d]
sql/sql_base.cc:4019(open_and_process_table)[0x7fef92c46557]
sql/sql_base.cc:5184(open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*))[0x7fef92c46c1d]
sql/sql_admin.cc:446(mysql_admin_table)[0x7fef92d576d6]
sql/sql_admin.cc:1210(Sql_cmd_check_table::execute(THD*))[0x7fef92d59046]
sql/sql_parse.cc:5573(mysql_execute_command(THD*))[0x7fef92c839fc]
sql/sql_parse.cc:7173(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x7fef92c8a334]
sql/sql_parse.cc:1470(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x7fef92c8cb80]
sql/sql_parse.cc:1095(do_command(THD*))[0x7fef92c8d29b]
sql/sql_connect.cc:1347(do_handle_one_connection(THD*))[0x7fef92d4c80c]
sql/sql_connect.cc:1260(handle_one_connection)[0x7fef92d4c9e7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4)[0x7fef924420a4]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fef9085c04d]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7fef5eb49020): is an invalid pointer
Connection ID (thread ID): 17307
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_conditio
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
150618 01:50:07 mysqld_safe Number of processes running now: 0
150618 01:50:07 mysqld_safe mysqld restarted
2015-06-18  1:50:07 139723066570624 [Note] /usr/local/mysql/bin/mysqld (mysqld 10.1.5-MariaDB-log) starting as process 16564 ...



 Comments   
Comment by Elena Stepanova [ 2015-06-19 ]

Thanks for the report.

Here is a variation of the concurrent test from MDEV-8339, but adjust for CHECK TABLE:

RQG grammar

query_init:
	CREATE TABLE IF NOT EXISTS t1 (`pk` INT AUTO_INCREMENT PRIMARY KEY, `c` VARCHAR(32), KEY(`c`)) ENGINE=MyISAM; CREATE TABLE IF NOT EXISTS t2 LIKE t1; CREATE TABLE IF NOT EXISTS t3 LIKE t1; CREATE TABLE IF NOT EXISTS t4 LIKE t1;
 
query:
	INSERT INTO { 't'.(int(rand(500))+5) } (`c`) VALUES ( _english ), ( _english ), ( _english ), ( _english ), ( _english ) |
	DELETE FROM { 't'.(int(rand(500))+5) } WHERE `c` = _english ORDER BY `pk` LIMIT _digit | 
	CHECK TABLE tbl;
 
tbl:
	t1 | t2 | t3 | t4;

The command line is the same.

Stack trace from debug build of 10.1 commit 93fc04ff1dc613a9ad75ccc57988a6b57b94b6cb

# 2015-06-19T22:43:05 [18731] #3  <signal handler called>
# 2015-06-19T22:43:05 [18731] #4  0x00007fc5b55e4af9 in TDC_element::free_tables_back (this=0x7fc58fcf1810) at 10.1/sql/table_cache.h:110
# 2015-06-19T22:43:05 [18731] #5  0x00007fc5b55e259f in tc_add_table_callback (element=0x7fc58fcf1810, arg=0x7fc5b4dfc550) at 10.1/sql/table_cache.cc:237
# 2015-06-19T22:43:05 [18731] #6  0x00007fc5b5c74d2d in lfind (head=0x7fc58f8203b0, cs=0x0, hashnr=0, key=0x7fc5b4dfc550 "test", keylen=0, cursor=0x7fc5b4dfc3d0, pins=0x7fc5b215f180, callback=0x7fc5b55e2565 <tc_add_table_callback(TDC_element*, tc_add_table_arg*)>) at 10.1/mysys/lf_hash.c:125
# 2015-06-19T22:43:05 [18731] #7  0x00007fc5b5c759a4 in lf_hash_iterate (hash=0x7fc5b671b040, pins=0x7fc5b215f180, action=0x7fc5b55e2565 <tc_add_table_callback(TDC_element*, tc_add_table_arg*)>, argument=0x7fc5b4dfc550) at 10.1/mysys/lf_hash.c:517
# 2015-06-19T22:43:05 [18731] #8  0x00007fc5b55e4649 in tdc_iterate (thd=0x7fc58f70c070, action=0x7fc5b55e2565 <tc_add_table_callback(TDC_element*, tc_add_table_arg*)>, argument=0x7fc5b4dfc550, no_dups=false) at 10.1/sql/table_cache.cc:1082
# 2015-06-19T22:43:05 [18731] #9  0x00007fc5b55e278c in tc_add_table (thd=0x7fc58f70c070, table=0x7fc58f8e8e70) at 10.1/sql/table_cache.cc:265
# 2015-06-19T22:43:05 [18731] #10 0x00007fc5b53af91d in open_table (thd=0x7fc58f70c070, table_list=0x7fc58f82c150, ot_ctx=0x7fc5b4dfcb80) at 10.1/sql/sql_base.cc:2572
# 2015-06-19T22:43:05 [18731] #11 0x00007fc5b53b2070 in open_and_process_table (thd=0x7fc58f70c070, lex=0x7fc58f70f968, tables=0x7fc58f82c150, counter=0x7fc5b4dfcc24, flags=0, prelocking_strategy=0x7fc5b4dfcc90, has_prelocking_list=false, ot_ctx=0x7fc5b4dfcb80) at 10.1/sql/sql_base.cc:4016
# 2015-06-19T22:43:05 [18731] #12 0x00007fc5b53b3165 in open_tables (thd=0x7fc58f70c070, options=..., start=0x7fc5b4dfcc08, counter=0x7fc5b4dfcc24, flags=0, prelocking_strategy=0x7fc5b4dfcc90) at 10.1/sql/sql_base.cc:4526
# 2015-06-19T22:43:05 [18731] #13 0x00007fc5b53b433e in open_and_lock_tables (thd=0x7fc58f70c070, options=..., tables=0x7fc58f82c150, derived=true, flags=0, prelocking_strategy=0x7fc5b4dfcc90) at 10.1/sql/sql_base.cc:5184
# 2015-06-19T22:43:05 [18731] #14 0x00007fc5b53a7e3b in open_and_lock_tables (thd=0x7fc58f70c070, tables=0x7fc58f82c150, derived=true, flags=0) at 10.1/sql/sql_base.h:530
# 2015-06-19T22:43:05 [18731] #15 0x00007fc5b55665ae in mysql_admin_table(THD *, TABLE_LIST *, HA_CHECK_OPT *, const char *, thr_lock_type, bool, bool, uint, int (*)(THD *, TABLE_LIST *, HA_CHECK_OPT *), struct {...}, int (*)(THD *, TABLE_LIST *, HA_CHECK_OPT *)) (thd=0x7fc58f70c070, tables=0x7fc58f82c150, check_opt=0x7fc58f710848, operator_name=0x7fc5b5d8abf4 "check", lock_type=TL_READ_NO_INSERT, open_for_modify=false, repair_table_use_frm=false, extra_open_options=32, prepare_func=0, operator_func=(int (handler::*)(handler * const, THD *, HA_CHECK_OPT *)) 0x7fc5b566153c <handler::ha_check(THD*, st_ha_check_opt*)>, view_operator_func=0x7fc5b550aa6d <view_check(THD*, TABLE_LIST*, st_ha_check_opt*)>) at 10.1/sql/sql_admin.cc:447
# 2015-06-19T22:43:05 [18731] #16 0x00007fc5b55691bf in Sql_cmd_check_table::execute (this=0x7fc58f82c738, thd=0x7fc58f70c070) at 10.1/sql/sql_admin.cc:1208
# 2015-06-19T22:43:05 [18731] #17 0x00007fc5b5422ac3 in mysql_execute_command (thd=0x7fc58f70c070) at 10.1/sql/sql_parse.cc:5573
# 2015-06-19T22:43:05 [18731] #18 0x00007fc5b5426dff in mysql_parse (thd=0x7fc58f70c070, rawbuf=0x7fc58f82c088 "CHECK TABLE t2", length=14, parser_state=0x7fc5b4dfe1f0) at 10.1/sql/sql_parse.cc:7173
# 2015-06-19T22:43:05 [18731] #19 0x00007fc5b5415d78 in dispatch_command (command=COM_QUERY, thd=0x7fc58f70c070, packet=0x7fc58f7ed071 "", packet_length=14) at 10.1/sql/sql_parse.cc:1470
# 2015-06-19T22:43:05 [18731] #20 0x00007fc5b5414a98 in do_command (thd=0x7fc58f70c070) at 10.1/sql/sql_parse.cc:1093
# 2015-06-19T22:43:05 [18731] #21 0x00007fc5b555640c in do_handle_one_connection (thd_arg=0x7fc58f70c070) at 10.1/sql/sql_connect.cc:1347
# 2015-06-19T22:43:05 [18731] #22 0x00007fc5b5556151 in handle_one_connection (arg=0x7fc58f70c070) at 10.1/sql/sql_connect.cc:1258
# 2015-06-19T22:43:05 [18731] #23 0x00007fc5b4acab50 in start_thread (arg=<optimized out>) at pthread_create.c:304
# 2015-06-19T22:43:05 [18731] #24 0x00007fc5b2b5f95d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

Comment by Sergey Vojtovich [ 2015-06-29 ]

Fixed in scope of MDEV-8339.

Generated at Thu Feb 08 07:26:22 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.