[MDEV-14960] [ERROR] mysqld got signal 11 with join_buffer and join_cache Created: 2018-01-16  Updated: 2020-08-25  Resolved: 2018-01-19

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 5.5, 10.0, 10.1, 10.2, 10.3
Fix Version/s: 5.5.59

Type: Bug Priority: Major
Reporter: Nilnandan Joshi Assignee: Igor Babaev
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-4626 Setting join_buffer_size causes join ... Open

 Description    

180116 12:23:24 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.1.19-MariaDB-enterprise
 
key_buffer_size=67108864
 
read_buffer_size=131072
 
max_used_connections=14
 
max_threads=302
 
thread_count=5
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 728870 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x0x7f7c8bb99008
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f889612acb0 thread_stack 0x40000
 
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x55de13fd5eee]
 
/usr/sbin/mysqld(handle_fatal_signal+0x2d5)[0x55de13af90f5]
 
/lib64/libpthread.so.0(+0xf5e0)[0x7f889bd7a5e0]
 
/usr/sbin/mysqld(_ZN4JOIN19shrink_join_buffersEP13st_join_tableyy+0x3d)[0x55de139a6d7d]
 
/usr/sbin/mysqld(_ZN10JOIN_CACHE12alloc_bufferEv+0x1c2)[0x55de13a6e692]
 
/usr/sbin/mysqld(_ZN10JOIN_CACHE4initEb+0x60)[0x55de13a6eeb0]
 
/usr/sbin/mysqld(_ZN17JOIN_CACHE_HASHED4initEb+0x44)[0x55de13a710a4]
 
/usr/sbin/mysqld(_Z33check_join_cache_usage_for_tablesP4JOINyj+0x745)[0x55de139baf35]
 
/usr/sbin/mysqld(_ZN4JOIN14optimize_innerEv+0x1509)[0x55de139cbae9]
 
/usr/sbin/mysqld(_ZN4JOIN8optimizeEv+0x2f)[0x55de139cd60f]
 
/usr/sbin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x8f)[0x55de139cd74f]
 
/usr/sbin/mysqld(_Z13handle_selectP3THDP3LEXP13select_resultm+0x245)[0x55de139ce2b5]
 
/usr/sbin/mysqld(+0x429291)[0x55de1396d291]
 
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5f8f)[0x55de139795bf]
 
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x352)[0x55de1397cf52]
 
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x25db)[0x55de1398042b]
 
/usr/sbin/mysqld(_Z10do_commandP3THD+0x169)[0x55de13980ca9]
 
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x18a)[0x55de13a4780a]
 
/usr/sbin/mysqld(handle_one_connection+0x40)[0x55de13a479e0]
 
/lib64/libpthread.so.0(+0x7e25)[0x7f889bd72e25]
 
/lib64/libc.so.6(clone+0x6d)[0x7f889a19134d]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f7c7f1e4020): select * from grc_db.incdnt_cause  where incdnt_num in (select incdnt_num from (SELECT   grc_mig.f_ELW_INCDNT_NUM(summ.error_id) incdnt_num,  grc_mig.f_ELW_CAUSE_NUM(ROOTCAUSE_CATEGORY2),  SUBSTRING(ROOTCAUSE_DESCIPTION,1,1000),  substring(description,1,1000),  'Y'  FROM   grc_mig.elw_error_event_summary summ   left join grc_mig.elw_error_event_impact impt on summ.ERROR_ID = impt.ERROR_ID  LEFT JOIN (SELECT ERROR_ID, GROUP_CONCAT(DESCRIPTION SEPARATOR ';') AS DESCRIPTION   FROM GRC_MIG.ELW_ACTIONITEM_DETAILS  WHERE TYPE = 2 GROUP BY ERROR_ID) as details on  summ.ERROR_ID = details.ERROR_ID  WHERE (error_type = 4 and LOSSIN_SGD > 50000)   or (error_type <> 4 and LOSSIN_SGD > 10000)   or ror_id in (0,9,1)  and ERROR_INITIATOR not in ('weizhongmun','%farahfoo%','carolinesenglh','ryantanweihiong','kokyuencheang','ryanangyiheng')  and summ.error_id not in (233285607,  286668596,  286668597,  289170081,  350952945,  362266862,  -1701,  -1698,  246477401,  450342668) and summ.ERROR_STATUS <> 'Draft'and rootcause_category2 <> 0 and   CASE WHEN ROOTCAUSE_CATEGORY1 <> ROOTCAUSE_CATEGORY2 and ROOTCAUSE_CATEGORY2 <> 24 THEN 'R2P'else 'R2S' end = 'R2P'  and summ.error_id <> 246477401) a)
 
Connection ID (thread ID): 7998
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=on,mrr_cost_based=on,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off

More details are attached with Ticket like output of EXPLAIN ..FORMAT=JSON and Table structures.



 Comments   
Comment by Sergei Petrunia [ 2018-01-16 ]

Note that the EXPLAIN has type='hash_ALL', key=key0. Could this mean there's some weird interplay between hash join and derived_with_keys like in
MDEV-14241?

Comment by Alice Sherepa [ 2018-01-17 ]

Reproducible with innodb and myisam, 10.1,10.2,10.3 (have not checked other versions)

testcase:

--source include/have_sequence.inc
 
 
 
SET SESSION join_cache_level = 6;
 
SET SESSION join_buffer_size=4096;
 
SET SESSION join_buffer_space_limit=4096;
 
SET  optimizer_switch = 'join_cache_hashed=on,optimize_join_buffer_size=on';
 
 
 
CREATE TABLE t1 (i1 int);
 
CREATE TABLE t2 (e1 int);
 
CREATE TABLE t4 (e1 int);
 
CREATE TABLE t5 (e1 int);
 
 
 
INSERT INTO t1 SELECT seq FROM seq_1_to_100;
 
INSERT INTO t2 SELECT * FROM t1;
 
INSERT INTO t4 SELECT * FROM t1;
 
INSERT INTO t5 SELECT * FROM t1;
 
 
 
 SELECT * FROM t1  WHERE i1 IN
 
  (SELECT i1 FROM
 
    (SELECT  (t4.e1) i1  FROM   t4
 
      LEFT JOIN t5  ON t4.e1 = t5.e1
 
      LEFT JOIN  (SELECT e1  FROM t2 ) AS d ON  t4.e1 = d.e1) a) ;
 
 
 
DROP TABLE t1,t4,t5,t2;

stacktrace 10.1 commit 4794e5b091c384

Thread 1 (Thread 0x7fd2e5fedb00 (LWP 17331)):
 
#0  __pthread_kill (threadid=<optimized out>, signo=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
 
#1  0x000055832dbc3377 in my_write_core (sig=11) at /home/alice/git/10.1/mysys/stacktrace.c:477
 
#2  0x000055832d55e565 in handle_fatal_signal (sig=11) at /home/alice/git/10.1/sql/signal_handler.cc:296
 
#3  <signal handler called>
 
#4  0x000055832d362501 in JOIN::shrink_join_buffers (this=0x7fd2cdd26e20, jt=0x7fd2cdd30438, curr_space=6403, needed_space=4096) at /home/alice/git/10.1/sql/sql_select.cc:2333
 
#5  0x000055832d494622 in JOIN_CACHE::alloc_buffer (this=0x7fd2cdd32318) at /home/alice/git/10.1/sql/sql_join_cache.cc:934
 
#6  0x000055832d494a5b in JOIN_CACHE::init (this=0x7fd2cdd32318, for_explain=false) at /home/alice/git/10.1/sql/sql_join_cache.cc:1077
 
#7  0x000055832d497a30 in JOIN_CACHE_HASHED::init (this=0x7fd2cdd32318, for_explain=false) at /home/alice/git/10.1/sql/sql_join_cache.cc:2685
 
#8  0x000055832d4992e0 in JOIN_CACHE_BNLH::init (this=0x7fd2cdd32318, for_explain=false) at /home/alice/git/10.1/sql/sql_join_cache.cc:3817
 
#9  0x000055832d3799b2 in check_join_cache_usage (tab=0x7fd2cdd30438, options=0, no_jbuf_after=4, table_index=3, prev_tab=0x7fd2cdd300f0) at /home/alice/git/10.1/sql/sql_select.cc:11039
 
#10 0x000055832d379dfe in check_join_cache_usage_for_tables (join=0x7fd2cdd26e20, options=0, no_jbuf_after=4) at /home/alice/git/10.1/sql/sql_select.cc:11163
 
#11 0x000055832d37a5a1 in make_join_readinfo (join=0x7fd2cdd26e20, options=0, no_jbuf_after=4) at /home/alice/git/10.1/sql/sql_select.cc:11335
 
#12 0x000055832d3606cd in JOIN::optimize_inner (this=0x7fd2cdd26e20) at /home/alice/git/10.1/sql/sql_select.cc:1804
 
#13 0x000055832d35dc28 in JOIN::optimize (this=0x7fd2cdd26e20) at /home/alice/git/10.1/sql/sql_select.cc:1045
 
#14 0x000055832d3662ab in mysql_select (thd=0x7fd2da7c7070, rref_pointer_array=0x7fd2da7cb518, tables=0x7fd2cdcfa380, wild_num=1, fields=..., conds=0x7fd2cdd26ae8, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7fd2cdd26e00, unit=0x7fd2da7cab70, select_lex=0x7fd2da7cb270) at /home/alice/git/10.1/sql/sql_select.cc:3435
 
#15 0x000055832d35bd69 in handle_select (thd=0x7fd2da7c7070, lex=0x7fd2da7caaa8, result=0x7fd2cdd26e00, setup_tables_done_option=0) at /home/alice/git/10.1/sql/sql_select.cc:384
 
#16 0x000055832d32bc5e in execute_sqlcom_select (thd=0x7fd2da7c7070, all_tables=0x7fd2cdcfa380) at /home/alice/git/10.1/sql/sql_parse.cc:5926
 
#17 0x000055832d321e45 in mysql_execute_command (thd=0x7fd2da7c7070) at /home/alice/git/10.1/sql/sql_parse.cc:2976
 
#18 0x000055832d32f37e in mysql_parse (thd=0x7fd2da7c7070, rawbuf=0x7fd2cdcfa088 "SELECT * FROM t1  WHERE i1 IN\n(SELECT i1 FROM\n(SELECT  (t4.e1) i1  FROM   t4 \nLEFT JOIN t5  ON t4.e1 = t5.e1\nLEFT JOIN  (SELECT e1  FROM t2 ) AS d ON  t4.e1 = d.e1) a)", length=167, parser_state=0x7fd2e5fec5e0) at /home/alice/git/10.1/sql/sql_parse.cc:7347
 
#19 0x000055832d31df0e in dispatch_command (command=COM_QUERY, thd=0x7fd2da7c7070, packet=0x7fd2dc6c9071 "SELECT * FROM t1  WHERE i1 IN\n(SELECT i1 FROM\n(SELECT  (t4.e1) i1  FROM   t4 \nLEFT JOIN t5  ON t4.e1 = t5.e1\nLEFT JOIN  (SELECT e1  FROM t2 ) AS d ON  t4.e1 = d.e1) a) ", packet_length=168) at /home/alice/git/10.1/sql/sql_parse.cc:1477
 
#20 0x000055832d31cc8c in do_command (thd=0x7fd2da7c7070) at /home/alice/git/10.1/sql/sql_parse.cc:1106
 
#21 0x000055832d455d49 in do_handle_one_connection (thd_arg=0x7fd2da7c7070) at /home/alice/git/10.1/sql/sql_connect.cc:1330
 
#22 0x000055832d455a98 in handle_one_connection (arg=0x7fd2da7c7070) at /home/alice/git/10.1/sql/sql_connect.cc:1242
 
#23 0x000055832d74f8fe in pfs_spawn_thread (arg=0x7fd2dc6bb8f0) at /home/alice/git/10.1/storage/perfschema/pfs.cc:1861
 
#24 0x00007fd2e4b426ba in start_thread (arg=0x7fd2e5fedb00) at pthread_create.c:333
 
#25 0x00007fd2e41ed3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Comment by Igor Babaev [ 2018-01-18 ]

I reproduced this bug for 5.5 as well

Comment by Igor Babaev [ 2018-01-19 ]

A fix for this bug was pushed into the 5.5 tree.

Generated at Thu Feb 08 08:17:38 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.