[MDEV-22498] SIGSEGV in Bitmap<64u>::merge on SELECT Created: 2020-05-08  Updated: 2022-07-03  Resolved: 2020-05-14

Status: Closed
Project: MariaDB Server
Component/s: Data Manipulation - Subquery
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.5.4, 10.1.46, 10.2.33, 10.3.24, 10.4.14

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Varun Gupta (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-14862 Server crashes in Bitmap<64u>::merge ... Closed
relates to MDEV-29017 SIGSEGV in Bitmap<64u>::merge on SELECT Confirmed
relates to MDEV-28509 Server crash via Item_func_ne::add_ke... In Review

 Description    

USE test;
 
SET SESSION sql_select_limit=0;
 
CREATE TABLE t(b INT);
 
CREATE TEMPORARY TABLE t(a INT);
 
DROP TABLE IF EXISTS t;
 
CREATE TABLE t2(a TEXT);
 
SELECT * FROM t2 HAVING a IN (SELECT a FROM t);

Leads to:

10.5.3 64488a6f2dd6aa43462292b757e783cfba11a8c6

 
Core was generated by `/test/MD050520-mariadb-10.5.3-linux-x86_64-opt/bin/mysqld --no-defaults --core-'.
 
Program terminated with signal SIGSEGV, Segmentation fault.
 
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=11)
 
    at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
 
[Current thread is 1 (Thread 0x14642c359700 (LWP 49916))]
 
(gdb) bt
 
#0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
 
#1  0x000055a6d343e757 in my_write_core (sig=sig@entry=11) at /test/10.5_opt/mysys/stacktrace.c:518
 
#2  0x000055a6d2e0081a in handle_fatal_signal (sig=11) at /test/10.5_opt/sql/signal_handler.cc:329
 
#3  <signal handler called>
 
#4  Bitmap<64u>::merge (map2=..., this=<optimized out>) at /test/10.5_opt/sql/sql_bitmap.h:211
 
#5  add_key_field (join=join@entry=0x14640984a820, key_fields=key_fields@entry=0x14642c356f58, and_level=and_level@entry=0, cond=cond@entry=0x14640984bf08, field=field@entry=0x14640985c868, eq_func=eq_func@entry=true, value=0x14640984bfa0, usable_tables=18446744073709551615, sargables=0x14642c357028, row_col_no=0, num_values=1) at /test/10.5_opt/sql/sql_select.cc:5921
 
#6  0x000055a6d2c3516c in add_key_equal_fields (join=join@entry=0x14640984a820, key_fields=key_fields@entry=0x14642c356f58, and_level=0, cond=cond@entry=0x14640984bf08, field_item=0x14640984a5a0, eq_func=eq_func@entry=true, val=0x14640984bfa0, usable_tables=18446744073709551615, sargables=0x14642c357028, row_col_no=0, num_values=1) at /test/10.5_opt/sql/sql_select.cc:6038
 
#7  0x000055a6d2c3c3a7 in Item_bool_func2::add_key_fields_optimize_op (this=0x14640984bf08, join=0x14640984a820, key_fields=0x14642c356f58, and_level=0x14642c356f54, usable_tables=18446744073709551615, sargables=0x14642c357028, equal_func=true) at /test/10.5_opt/sql/sql_select.cc:6321
 
#8  0x000055a6d2e477cf in Item_func_eq::add_key_fields (this=<optimized out>, join=<optimized out>, key_fields=<optimized out>, and_level=<optimized out>, usable_tables=<optimized out>, sargables=<optimized out>) at /test/10.5_opt/sql/item_cmpfunc.h:727
 
#9  0x000055a6d2c3c9c3 in update_ref_and_keys (thd=0x146409812018, keyuse=keyuse@entry=0x14642c357030, join_tab=0x14640984b498, tables=1, cond=0x14640984bf08, normal_tables=18446744073709551615, sargables=0x14642c357028, select_lex=<optimized out>, select_lex=<optimized out>) at /test/10.5_opt/sql/sql_select.cc:6794
 
#10 0x000055a6d2c61222 in JOIN::reoptimize (this=this@entry=0x14640984a820, added_where=<optimized out>, join_tables=join_tables@entry=1, save_to=save_to@entry=0x0) at /test/10.5_opt/sql/sql_select.cc:27926
 
#11 0x000055a6d2d3b302 in JOIN::choose_subquery_plan (this=this@entry=0x14640984a820, join_tables=1) at /test/10.5_opt/sql/opt_subselect.cc:6616
 
#12 0x000055a6d2c674b1 in make_join_statistics (keyuse_array=0x14640984ab10, tables_list=..., join=0x14640984a820) at /test/10.5_opt/sql/sql_select.cc:5589
 
#13 JOIN::optimize_inner (this=this@entry=0x14640984a820) at /test/10.5_opt/sql/sql_select.cc:2260
 
#14 0x000055a6d2c67acb in JOIN::optimize (this=this@entry=0x14640984a820) at /test/10.5_opt/sql/sql_select.cc:1606
 
#15 0x000055a6d2bed525 in st_select_lex::optimize_unflattened_subqueries (this=0x1464098470f0, const_only=const_only@entry=false) at /test/10.5_opt/sql/sql_lex.cc:4779
 
#16 0x000055a6d2d396b3 in JOIN::optimize_unflattened_subqueries (this=this@entry=0x14640984a018) at /test/10.5_opt/sql/opt_subselect.cc:5547
 
#17 0x000055a6d2c61f2b in JOIN::optimize_stage2 (this=this@entry=0x14640984a018) at /test/10.5_opt/sql/sql_select.cc:3042
 
#18 0x000055a6d2c6597d in JOIN::optimize_inner (this=this@entry=0x14640984a018) at /test/10.5_opt/sql/sql_select.cc:2286
 
#19 0x000055a6d2c67acb in JOIN::optimize (this=this@entry=0x14640984a018) at /test/10.5_opt/sql/sql_select.cc:1606
 
#20 0x000055a6d2c67bd1 in mysql_select (thd=thd@entry=0x146409812018, tables=0x1464098476e8, fields=..., conds=0x0, og_num=<optimized out>, order=0x0, group=0x0, having=0x1464098493f8, proc_param=0x0, select_options=2147748608, result=0x146409849ff0, unit=0x146409815e68, select_lex=0x1464098470f0) at /test/10.5_opt/sql/sql_select.cc:4655
 
#21 0x000055a6d2c685e1 in handle_select (thd=thd@entry=0x146409812018, lex=lex@entry=0x146409815da0, result=result@entry=0x146409849ff0, setup_tables_done_option=setup_tables_done_option@entry=0) at /test/10.5_opt/sql/sql_select.cc:417
 
#22 0x000055a6d2c0f1c1 in execute_sqlcom_select (thd=thd@entry=0x146409812018, all_tables=0x1464098476e8) at /test/10.5_opt/sql/sql_parse.cc:6172
 
#23 0x000055a6d2c0b0e2 in mysql_execute_command (thd=thd@entry=0x146409812018) at /test/10.5_opt/sql/sql_parse.cc:3901
 
#24 0x000055a6d2c1227c in mysql_parse (thd=0x146409812018, rawbuf=<optimized out>, length=46, parser_state=0x14642c3584d0, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /test/10.5_opt/sql/sql_parse.cc:7957
 
#25 0x000055a6d2c078a5 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x146409812018, packet=packet@entry=0x14640983a019 "SELECT * FROM t2 HAVING a IN (SELECT a FROM t)", packet_length=packet_length@entry=46, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.5_opt/sql/sql_parse.cc:1839
 
#26 0x000055a6d2c05b36 in do_command (thd=0x146409812018) at /test/10.5_opt/sql/sql_parse.cc:1358
 
#27 0x000055a6d2cfa2ee in do_handle_one_connection (connect=<optimized out>, connect@entry=0x14642a0329b8, put_in_cache=put_in_cache@entry=true) at /test/10.5_opt/sql/sql_connect.cc:1422
 
#28 0x000055a6d2cfa494 in handle_one_connection (arg=arg@entry=0x14642a0329b8) at /test/10.5_opt/sql/sql_connect.cc:1319
 
#29 0x000055a6d30665ea in pfs_spawn_thread (arg=0x14642a04b018) at /test/10.5_opt/storage/perfschema/pfs.cc:2201
 
#30 0x000014642b7806db in start_thread (arg=0x14642c359700) at pthread_create.c:463
 
#31 0x000014642ab7e88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Bug confirmed present in:
MariaDB: 10.1.45 (dbg), 10.1.45 (opt), 10.2.32 (dbg), 10.2.32 (opt), 10.3.23 (dbg), 10.3.23 (opt), 10.4.13 (dbg), 10.4.13 (opt), 10.5.2 (dbg), 10.5.2 (opt), 10.5.3 (dbg), 10.5.3 (opt)

Bug confirmed not present in:
MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.47 (dbg), 5.6.47 (opt), 5.7.29 (dbg), 5.7.29 (opt), 8.0.19 (dbg), 8.0.19 (opt)

 Comments   
Comment by Roel Van de Paar [ 2020-05-08 ]

Whereas at first glance the testcase looks like an edge case given the sql_select_limit=0, there are some interesting things happening here around t1.

Comment by Roel Van de Paar [ 2020-05-08 ]

Found that changing the column name from 'b' to 'a' in table t makes the testcase work without fail;

10.5.3>USE test;
 
Database changed
 
10.5.3>SET SESSION sql_select_limit=0;
 
Query OK, 0 rows affected (0.000 sec)
 
10.5.3>CREATE TABLE t(a INT);     # changed 'b' to 'a'
 
Query OK, 0 rows affected (0.010 sec)
 
10.5.3>CREATE TEMPORARY TABLE t(a INT);
 
Query OK, 0 rows affected (0.001 sec)
 
10.5.3>DROP TABLE IF EXISTS t;
 
Query OK, 0 rows affected (0.000 sec)
 
10.5.3>CREATE TABLE t2(a TEXT);
 
Query OK, 0 rows affected (0.010 sec)
 
10.5.3>SELECT * FROM t2 HAVING a IN (SELECT a FROM t);
 
Empty set (0.001 sec)

Comment by Roel Van de Paar [ 2020-05-08 ]

Also see: https://jira.mariadb.org/browse/MDEV-22498?jql=text%20~%20%22%5C%22Bitmap%3C64u%3E%3A%3Amerge%5C%22%22%20ORDER%20BY%20status%20ASC for various cloded issues in same area.

Comment by Varun Gupta (Inactive) [ 2020-05-11 ]

The test case that is created fails only when the column in the select list of the IN subquery refers to the item in the select list of the parent select

Added the table names to the select to make it more clear.

SELECT sum(a), t2.a FROM t2 HAVING t2.a IN (SELECT t2.a FROM t1);

Comment by Varun Gupta (Inactive) [ 2020-05-11 ]

Here is a better test case:

--source include/have_innodb.inc
 
SET SESSION sql_select_limit=0;
 
CREATE TABLE t1(b INT, c INT);
 
INSERT INTO t1 VALUES (1,1),(2,2),(3,3);
 
CREATE TABLE t2(a INT, b INT);
 
INSERT INTO t2 VALUES (1,1),(2,2),(3,3);
 
SELECT * FROM t2 HAVING t2.a IN (SELECT t2.b FROM t1);
 
drop table t1,t2;

To trigger the crash we need 2 conditions

  • the subquery needs to have an outer reference
  • the parent select is a degenerate join (ensure by sql_select_limit=0)
Comment by Varun Gupta (Inactive) [ 2020-05-11 ]

So lets start to write down some observations

Lets take a look at the function
Item_in_subselect::create_single_in_to_exists_cond

Here we end up in this part of the code

  else
 
  {
 
    Item *item= (Item*)select_lex->item_list.head();
 
    if (item->type() != REF_ITEM ||
 
        ((Item_ref*)item)->ref_type() != Item_ref::VIEW_REF)
 
      item= item->real_item();

We are taking the branch as we don't have a having clause, group by clause or aggregate function in the subquery.

One thing that looked suspicious was that item_list does not remain updated after fix_fields and one should use JOIN::fields_list,
but then going through more code in the function JOIN::optimize_inner, I see

 /*
 
    Needed in case optimizer short-cuts,
 
    set properly in make_aggr_tables_info()
 
  */
 
  fields= &select_lex->item_list;

So these are the same list

Comment by Varun Gupta (Inactive) [ 2020-05-11 ]

Now coming back to the function where we create the equality when the IN->EXISTS transformation happen

    Item *item= (Item*)select_lex->item_list.head();


(gdb) p (Item*) select_lex->item_list.head()
 
$4 = (Item *) 0x62b000003428
 
(gdb) p dbug_print_item((Item*) select_lex->item_list.head())
 
$5 = 0x3965020 <dbug_item_print_buf> "t2.b"
 
(gdb) p item->type()
 
$6 = Item::REF_ITEM

and then we have the code, where for REF items (except Item_direct_view_ref) we use real_item()

    if (item->type() != REF_ITEM ||
 
        ((Item_ref*)item)->ref_type() != Item_ref::VIEW_REF)
 
      item= item->real_item();

I thing using real_item is incorrect here, we should just use the original item to create the equality because the ref item has the information like the item belongs to the parent select.
This would make sure that the item t2.b will not be local to the subquery and hence we would not try to create a keyuse for the equi-join condition

real_item does not make sense here because the ref item here are referring to the parent select.
Even for cases where we have ref item in the subquery like in HAVING, GROUP BY etc are handled separately in the same function (the if branch)

Comment by Varun Gupta (Inactive) [ 2020-05-11 ]

Pushed a patch to the branch 10.1-mdev22498

Comment by Oleksandr Byelkin [ 2020-05-13 ] 

    /*
 
      No need to use real_item for the item, as the ref items that are possible
 
      in the subquery either belong to views or to the parent select.
 
      For such case we need to refer to the reference and not to the original
 
      item.
 
   */

above is not exactly true, because one can use alias there, but I checked it works

SELECT sum(a), t2.a aaa, t2.b FROM t2 HAVING aaa IN (SELECT aaa FROM t1);

otherwise it is ok

Comment by Roel Van de Paar [ 2020-05-27 ]

Filter removed from runs

Generated at Thu Feb 08 09:15:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.