Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
5.5.28a, 5.3.8, 5.3.9
-
None
-
None
-
Linux 2.6.32-279.1.1.el6.x86_64 (Red Hat EL6)
Description
Hello and thank you for mariadb,
I have a crash I can reproduce against 5.3.8 and 5.3.9 (only versions tried so far), it is currently a select with some sub select and joins against 5 MyISAM tables, about 1.2G in size, but the mysqldump compresses to 116M. Using the binary tar file images, running on Red Hat EL6. I have not spent time reducing it into a smaller reproducer.
Once I have created this issue I will send the private compressed mysqldump by anonymous FTP.
From the error log:
121006 7:58:26 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Server version: 5.3.9-MariaDB
key_buffer_size=134213632
read_buffer_size=131072
max_used_connections=1
max_threads=152
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 463795 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0x367dee0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fd27c6ece68 thread_stack 0x48000
./bin/mysqld(my_print_stacktrace+0x2e) [0xa3015e]
./bin/mysqld(handle_fatal_signal+0x3f9) [0x7662c9]
/lib64/libpthread.so.0() [0x3f5dc0f4a0]
./bin/mysqld(PROFILING::~PROFILING()+0x49) [0x6df649]
./bin/mysqld(create_internal_tmp_table_from_heap(THD*, st_table*, st_maria_columndef*, st_maria_columndef**, int, bool)+0x25) [0x6c0275]
./bin/mysqld(select_union::send_data(List<Item>&)+0x133) [0x7af483]
./bin/mysqld() [0x6ab22b]
./bin/mysqld() [0x6c13dc]
./bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xe0) [0x6c1610]
./bin/mysqld() [0x6c1b13]
./bin/mysqld(JOIN::exec()+0x859) [0x6d2e69]
Fatal signal 6 while backtracing
121006 07:58:26 mysqld_safe Number of processes running now: 0
121006 07:58:26 mysqld_safe mysqld restarted
This is the query:
SELECT t1.deal_id AS `deal_id` FROM table_e AS t1 JOIN table_d AS t10 ON (t1.deal_id = t10.deal_id) AND (t10.co_role_id IN ( SELECT co_role_id FROM table_a WHERE target ) = '1') JOIN table_c AS t11 ON ((t10.deal_id = t11.deal_id) AND (t10.feid = t11.feid)) AND (t10.co_role_id IN ( SELECT co_role_id FROM table_a WHERE target ) = '1') WHERE ((t1.deal_id) IN ( SELECT * FROM ( (SELECT DISTINCT t1.deal_id FROM table_e AS t1 JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id) AND (t2.co_role_id IN ( SELECT co_role_id FROM table_a WHERE target_all ) = '1') JOIN table_c AS t3 ON ((t2.deal_id = t3.deal_id) AND (t2.feid = t3.feid)) AND (t2.co_role_id IN ( SELECT co_role_id FROM table_a WHERE target ) = '1') JOIN table_f AS t4 ON t3.exchange = t4.exchange WHERE t4.exchange IN ('CT','ND','NY','AM')) UNION (SELECT DISTINCT t1.deal_id FROM table_e AS t1 JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id) AND (t2.co_role_id IN ( SELECT co_role_id FROM table_a WHERE target_all ) = '1') JOIN table_c AS t3 ON ((t2.deal_id = t3.deal_id) AND (t2.feid = t3.feid)) AND (t2.co_role_id IN ( SELECT co_role_id FROM table_a WHERE target ) = '1') JOIN table_b AS t5 ON t3.country = t5.country WHERE t5.country = 'US') ) tU )) AND ((t1.deal_id) IN ( SELECT * FROM ( (SELECT DISTINCT t1.deal_id FROM table_e AS t1 JOIN table_d AS t6 ON (t1.deal_id = t6.deal_id) AND (t6.co_role_id IN ( SELECT co_role_id FROM table_a WHERE acquirer_all ) = '1') JOIN table_c AS t7 ON ((t6.deal_id = t7.deal_id) AND (t6.feid = t7.feid)) AND (t6.co_role_id IN ( SELECT co_role_id FROM table_a WHERE acquirer_all ) = '1') JOIN table_f AS t8 ON t7.exchange = t8.exchange WHERE t8.exchange IN ('CT','ND','NY','AM')) UNION (SELECT DISTINCT t1.deal_id FROM table_e AS t1 JOIN table_d AS t6 ON (t1.deal_id = t6.deal_id) AND (t6.co_role_id IN ( SELECT co_role_id FROM table_a WHERE acquirer_all ) = '1') JOIN table_c AS t7 ON ((t6.deal_id = t7.deal_id) AND (t6.feid = t7.feid)) AND (t6.co_role_id IN ( SELECT co_role_id FROM table_a WHERE acquirer_all ) = '1') ) ) tU )) LIMIT 200 ;
-
-
- glibc detected *** ./bin/mysqld: corrupted double-linked list: 0x00007fd22811cb70 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3f5d8750c6]
/lib64/libc.so.6[0x3f5d878b34]
/lib64/libc.so.6(__libc_malloc+0x6d)[0x3f5d87948d]
/usr/lib64/libstdc++.so.6[0x3f620c0f5b]
/usr/lib64/libstdc++.so.6[0x3f620c2aff]
/usr/lib64/libstdc++.so.6(__cxa_demangle+0x6e)[0x3f620c2cce]
./bin/mysqld(my_print_stacktrace+0xe6)[0xa30216]
./bin/mysqld(handle_fatal_signal+0x3f9)[0x7662c9]
/lib64/libpthread.so.0[0x3f5dc0f4a0]
./bin/mysqld(_ZN9PROFILINGD2Ev+0x49)[0x6df649]
./bin/mysqld(_Z35create_internal_tmp_table_from_heapP3THDP8st_tableP18st_maria_columndefPS4_ib+0x25)[0x6c0275]
./bin/mysqld(_ZN12select_union9send_dataER4ListI4ItemE+0x133)[0x7af483]
./bin/mysqld[0x6ab22b]
./bin/mysqld[0x6c13dc]
./bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0xe0)[0x6c1610]
./bin/mysqld[0x6c1b13]
./bin/mysqld(_ZN4JOIN4execEv+0x859)[0x6d2e69]
./bin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x199)[0x6d4f69]
./bin/mysqld(_ZN18st_select_lex_unit4execEv+0xf59)[0x7b0709]
./bin/mysqld(_Z18mysql_derived_fillP3THDP6st_lexP10TABLE_LIST+0x198)[0x7b20b8]
./bin/mysqld(_Z27mysql_handle_single_derivedP6st_lexP10TABLE_LISTj+0xb8)[0x7b31d8]
./bin/mysqld(_ZN13st_join_table12preread_initEv+0xa1)[0x6ae131]
./bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x496)[0x6c19c6]
./bin/mysqld(_Z26join_tab_execution_startupP13st_join_table+0xd8)[0x748928]
./bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x251)[0x6c1781]
./bin/mysqld[0x6c1b13]
./bin/mysqld(_ZN4JOIN4execEv+0x859)[0x6d2e69]
./bin/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x199)[0x6d4f69]
./bin/mysqld(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x16f)[0x6d593f]
./bin/mysqld[0x64a7be]
./bin/mysqld(_Z21mysql_execute_commandP3THD+0x3a43)[0x6503a3]
./bin/mysqld(_Z11mysql_parseP3THDPcjPPKc+0x299)[0x653189]
./bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xa9b)[0x65407b]
./bin/mysqld(_Z10do_commandP3THD+0x101)[0x654bd1]
./bin/mysqld(handle_one_connection+0xfd)[0x645fbd]
/lib64/libpthread.so.0[0x3f5dc077f1]
/lib64/libc.so.6(clone+0x6d)[0x3f5d8e570d]
- glibc detected *** ./bin/mysqld: corrupted double-linked list: 0x00007fd22811cb70 ***
-
Attachments
Activity
Need to try to create a simpler test case in order to include it into the MTR suite.
Patch from Igor:
=== modified file 'sql/sql_select.cc'
— sql/sql_select.cc 2012-10-10 05:21:22 +0000
+++ sql/sql_select.cc 2012-10-20 05:49:13 +0000
@@ -8981,7 +8981,7 @@ void JOIN::drop_unused_derived_keys()
JOIN_TAB *tab;
for (tab= first_linear_tab(this, WITHOUT_CONST_TABLES);
tab;
- tab= next_linear_tab(this, tab, WITHOUT_BUSH_ROOTS))
+ tab= next_linear_tab(this, tab, WITH_BUSH_ROOTS))
{
TABLE *table=tab->table;
Simplified test case:
CREATE TABLE t1 (
pk int(10) unsigned NOT NULL AUTO_INCREMENT,
a char(2) DEFAULT NULL,
PRIMARY KEY (pk),
KEY a (a)
) ENGINE=MyISAM;
INSERT INTO t1 (a)
VALUES (NULL),(NULL),(NULL),('AB'),(NULL),('CD'),(NULL),(NULL);
INSERT INTO t1 SELECT NULL, a1.a FROM t1 a1, t1 a2, t1 a3, t1 a4, t1 a5;
CREATE TABLE t2 (
pk int(10) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY
) ENGINE=MyISAM;
INSERT INTO t2 VALUES (NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL);
INSERT INTO t2 SELECT NULL FROM t2 a1, t2 a2, t2 a3, t2 a4, t2 a5;
CREATE TABLE t3 (
pk int(10) unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY
) ENGINE=MyISAM;
INSERT INTO t3 VALUES (NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL),(NULL);
INSERT INTO t3 SELECT NULL FROM t3 a1, t3 a2, t3 a3, t3 a4, t3 a5;
CREATE TABLE t4 (
a char(2) NOT NULL DEFAULT '',
PRIMARY KEY (a)
) ENGINE=MyISAM;
INSERT INTO t4 VALUES ('CD');
set @@tmp_table_size=8192;
SELECT * FROM t3 AS tX JOIN t2 AS tY ON (tX.pk = tY.pk)
WHERE
tX.pk IN
(SELECT *
FROM (SELECT DISTINCT tA.pk
FROM t3 AS tA
JOIN t2 AS tB ON (tA.pk = tB.pk)
JOIN t1 AS tC ON (tB.pk = tC.pk)
JOIN t4 AS tD ON tC.a = tD.a) tU)
limit 10;
I managed to reduce the original test case through the following commands:
CREATE TABLE table_c_s (
deal_id int(10) unsigned NOT NULL,
feid int(10) unsigned NOT NULL,
exchange char(2) DEFAULT NULL,
PRIMARY KEY (deal_id,feid),
KEY feid (feid,deal_id),
KEY exchange (exchange),
KEY deal_feid (deal_id,feid),
KEY feid_date (feid)
) ENGINE=MyISAM;
insert into table_c_s select deal_id, feid, exchange from table_c where co_role_id = 2;
delete from table_d where co_role_id <> 2;
delete from table_f where exchange <> 'ND';
The simplified query that reproduces the bug is:
set @@tmp_table_size=16384;
SELECT t1.deal_id
FROM table_e AS t1 JOIN table_d AS t10 ON (t1.deal_id = t10.deal_id)
WHERE
t1.deal_id IN
(SELECT *
FROM (SELECT DISTINCT t1.deal_id
FROM table_e AS t1
JOIN table_d AS t2 ON (t1.deal_id = t2.deal_id)
JOIN table_c_s AS t3 ON (t2.deal_id = t3.deal_id) AND (t2.feid = t3.feid)
JOIN table_f AS t4 ON t3.exchange = t4.exchange) tU)
limit 10;
– EXPLAIN in 5.5
-----
-----------------------------------------------------------------------------------------------------------------------------------------------+-----
-----------------------------------------------------------------------------------------------------------------------------------------------+-----
-----------------------------------------------------------------------------------------------------------------------------------------------+– EXPLAIN in 5.3
---
-----------------------------------------------------------------------------------------------------------------------------------------------+---
-----------------------------------------------------------------------------------------------------------------------------------------------+---
-----------------------------------------------------------------------------------------------------------------------------------------------+There is a bug both in the current 5.3 and 5.5 versions. Based on the query plan, and the stack trace below, my hypothesis is that the problem is in the code that creates the temporary table for the derived table. More specifically the problem could be related to the index created on the fly for this temporary table.
== In 5.3 (both optimized and debug build) the query results in a crash with the following stack trace:
#0 0x00007f8f31944319 in __memset_sse2 () from /lib64/libc.so.6
#1 0x0000000000a021ce in sort_insert_key (sort_param=0x7f8f3339c2f0, key_block=0x7f8ec410d358, key=0x7f8ec4153358 "", prev_block=18446744073709551615) at ma_check.c:5755
#2 0x0000000000a016a7 in sort_key_write (sort_param=0x7f8f3339c2f0, a=0x7f8ec4153358 "") at ma_check.c:5526
#3 0x0000000000a80c30 in write_index (info=0x7f8f3339c2f0, sort_keys=0x7f8ec414d650, count=1953) at ma_sort.c:772
#4 0x0000000000a7ec77 in _ma_create_index_by_sort (info=0x7f8f3339c2f0, no_messages=1 '\001', sortbuff_size=134217728) at ma_sort.c:201
#5 0x00000000009fc7ad in maria_repair_by_sort (param=0x7f8ec40ec530, info=0x7f8ec40e7568, name=0x7f8f3339cbb0 "/tmp/#sql_6aca_0", rep_quick=1 '\001') at ma_check.c:3877
#6 0x00000000009eb4c0 in ha_maria::repair (this=0x7f8ec40b8af0, thd=0x2d69cf8, param=0x7f8ec40ec530, do_optimize=false) at ha_maria.cc:1666
#7 0x00000000009ec104 in ha_maria::enable_indexes (this=0x7f8ec40b8af0, mode=2) at ha_maria.cc:1989
#8 0x00000000009ec802 in ha_maria::end_bulk_insert (this=0x7f8ec40b8af0) at ha_maria.cc:2178
#9 0x00000000007a8583 in handler::ha_end_bulk_insert (this=0x7f8ec40b8af0) at handler.h:1784
#10 0x0000000000794b7e in create_internal_tmp_table_from_heap2 (thd=0x2d69cf8, table=0x7f8ec4059370, start_recinfo=0x7f8ec405a140, recinfo=0x7f8ec404fc38, error=135, ignore_last_dupp_key_error=true, hton=0x1b5ec58, proc_info=0xde8e83 "converting HEAP to Aria") at sql_select.cc:15130
#11 0x00000000007946e1 in create_internal_tmp_table_from_heap (thd=0x2d69cf8, table=0x7f8ec4059370, start_recinfo=0x7f8ec405a140, recinfo=0x7f8ec404fc38, error=135, ignore_last_dupp_key_error=true) at sql_select.cc:14875
#12 0x00000000008f17e2 in select_union::send_data (this=0x7f8ec404fbb0, values=...) at sql_union.cc:87
#13 0x0000000000798a1a in end_send (join=0x7f8ec404fca8, join_tab=0x7f8ec4069d48, end_of_records=false) at sql_select.cc:16757
#14 0x0000000000796922 in evaluate_join_record (join=0x7f8ec404fca8, join_tab=0x7f8ec4069a28, error=0) at sql_select.cc:15915
#15 0x00000000007962ed in sub_select (join=0x7f8ec404fca8, join_tab=0x7f8ec4069a28, end_of_records=false) at sql_select.cc:15757
#16 0x000000000079592e in do_select (join=0x7f8ec404fca8, fields=0x7f8ec404fff8, table=0x0, procedure=0x0) at sql_select.cc:15378
#17 0x0000000000776ba5 in JOIN::exec (this=0x7f8ec404fca8) at sql_select.cc:2761
#18 0x000000000077742f in mysql_select (thd=0x2d69cf8, rref_pointer_array=0x7f8ec404ade8, tables=0x7f8ec404b4e8, wild_num=0, fields=..., conds=0x7f8ec405ad08, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2416200193, result=0x7f8ec404fbb0, unit=0x7f8ec404aed0, select_lex=0x7f8ec404ab90) at sql_select.cc:2982
#19 0x00000000008f5f9d in mysql_derived_fill (thd=0x2d69cf8, lex=0x2d6c1e8, derived=0x7f8ec404ea70) at sql_derived.cc:889
#20 0x00000000008f4bb8 in mysql_handle_single_derived (lex=0x2d6c1e8, derived=0x7f8ec404ea70, phases=96) at sql_derived.cc:176
#21 0x0000000000789017 in st_join_table::preread_init (this=0x7f8ec4065e70) at sql_select.cc:10171
#22 0x0000000000795f1c in sub_select (join=0x7f8ec404f098, join_tab=0x7f8ec4065e70, end_of_records=false) at sql_select.cc:15672
#23 0x000000000085c0db in join_tab_execution_startup (tab=0x7f8ec40654f0) at opt_subselect.cc:4945
#24 0x0000000000796087 in sub_select (join=0x7f8ec404f098, join_tab=0x7f8ec40654f0, end_of_records=false) at sql_select.cc:15705
#25 0x000000000079592e in do_select (join=0x7f8ec404f098, fields=0x2d6c8a0, table=0x0, procedure=0x0) at sql_select.cc:15378
#26 0x0000000000776ba5 in JOIN::exec (this=0x7f8ec404f098) at sql_select.cc:2761
== In 5.5 (both optimized and debug build) the query is interrupted with the following error:
ERROR 1030 (HY000): Got error 140 from storage engine
This error means: "MySQL error code 140: Wrong create options".