[MDEV-25346] Server crashes in Item_field::fix_outer_field upon subquery with unknown column Created: 2021-04-06  Updated: 2023-07-06  Resolved: 2023-07-06

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 10.2, 10.3, 10.4, 10.5
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Igor Babaev
Resolution: Cannot Reproduce Votes: 1
Labels: None

Issue Links:
Relates
relates to MDEV-26412 Server crash in Item_field::fix_outer... Closed
relates to MDEV-25362 Incorrect name resolution for subquer... Closed
relates to MDEV-31632 Server crashes in Item_field::fix_out... Closed

 Description   

Note: The test case doesn't involve JSON tables, but this is a regression to a general use case introduced by changes made in the scope of JSON_TABLE task in the development tree, hence the relation and the "Blocker" status.

CREATE TABLE t1 (a INT);
 
INSERT INTO t1 VALUES (1),(2); # Optional, fails either way
 
CREATE TABLE t2 (b INT);
 
INSERT INTO t2 VALUES (1),(2); # Optional, fails either way
 
 
 
SELECT * FROM ( SELECT * FROM t1 JOIN t2 ON (b IN(SELECT x FROM (SELECT 1 AS c) AS sq1))) AS sq2;
 
 
 
DROP TABLE t1, t2;

bb-10.6-mdev17399-hf 160bd1691b

 
#3  <signal handler called>
 
#4  0x00005638f79e5c13 in Item_field::fix_outer_field (this=0x62b00003a520, thd=0x62b000069288, from_field=0x7fd5901cf740, reference=0x62b00003a670) at /data/src/bb-10.6-mdev17399-hf/sql/item.cc:5519
 
#5  0x00005638f79e9788 in Item_field::fix_fields (this=0x62b00003a520, thd=0x62b000069288, reference=0x62b00003a670) at /data/src/bb-10.6-mdev17399-hf/sql/item.cc:5939
 
#6  0x00005638f6eadcb3 in Item::fix_fields_if_needed (this=0x62b00003a520, thd=0x62b000069288, ref=0x62b00003a670) at /data/src/bb-10.6-mdev17399-hf/sql/item.h:995
 
#7  0x00005638f6eadced in Item::fix_fields_if_needed_for_scalar (this=0x62b00003a520, thd=0x62b000069288, ref=0x62b00003a670) at /data/src/bb-10.6-mdev17399-hf/sql/item.h:999
 
#8  0x00005638f6fd5137 in setup_fields (thd=0x62b000069288, ref_pointer_array=..., fields=..., column_usage=MARK_COLUMNS_READ, sum_func_list=0x62900025eb70, pre_fix=0x62b00003a1e0, allow_sum_func=true) at /data/src/bb-10.6-mdev17399-hf/sql/sql_base.cc:7671
 
#9  0x00005638f7207322 in JOIN::prepare (this=0x62900025e848, tables_init=0x62b00003b488, conds_init=0x0, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x62b00003a070, unit_arg=0x62b00003bb98) at /data/src/bb-10.6-mdev17399-hf/sql/sql_select.cc:1256
 
#10 0x00005638f7bbedf6 in subselect_single_select_engine::prepare (this=0x62b00003c600, thd=0x62b000069288) at /data/src/bb-10.6-mdev17399-hf/sql/item_subselect.cc:3835
 
#11 0x00005638f7b982c5 in Item_subselect::fix_fields (this=0x62b00003c3a8, thd_param=0x62b000069288, ref=0x62b0000397c8) at /data/src/bb-10.6-mdev17399-hf/sql/item_subselect.cc:289
 
#12 0x00005638f7bbbbdd in Item_in_subselect::fix_fields (this=0x62b00003c3a8, thd_arg=0x62b000069288, ref=0x62b0000397c8) at /data/src/bb-10.6-mdev17399-hf/sql/item_subselect.cc:3497
 
#13 0x00005638f6eadcb3 in Item::fix_fields_if_needed (this=0x62b00003c3a8, thd=0x62b000069288, ref=0x62b0000397c8) at /data/src/bb-10.6-mdev17399-hf/sql/item.h:995
 
#14 0x00005638f6eadced in Item::fix_fields_if_needed_for_scalar (this=0x62b00003c3a8, thd=0x62b000069288, ref=0x62b0000397c8) at /data/src/bb-10.6-mdev17399-hf/sql/item.h:999
 
#15 0x00005638f6fe3a47 in Item::fix_fields_if_needed_for_bool (this=0x62b00003c3a8, thd=0x62b000069288, ref=0x62b0000397c8) at /data/src/bb-10.6-mdev17399-hf/sql/item.h:1003
 
#16 0x00005638f6fda309 in setup_on_expr (thd=0x62b000069288, table=0x62b000039768, is_update=false) at /data/src/bb-10.6-mdev17399-hf/sql/sql_base.cc:8307
 
#17 0x00005638f6fdaec2 in setup_conds (thd=0x62b000069288, tables=0x62b000039028, leaves=..., conds=0x62900025e448) at /data/src/bb-10.6-mdev17399-hf/sql/sql_base.cc:8424
 
#18 0x00005638f7200240 in setup_without_group (thd=0x62b000069288, ref_pointer_array=..., tables=0x62b000039028, leaves=..., fields=..., all_fields=..., conds=0x62900025e448, order=0x0, group=0x0, win_specs=..., win_funcs=..., hidden_group_fields=0x62900025e31f, reserved=0x62b000038cec) at /data/src/bb-10.6-mdev17399-hf/sql/sql_select.cc:698
 
#19 0x00005638f720766d in JOIN::prepare (this=0x62900025e038, tables_init=0x62b000039028, conds_init=0x0, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x62b0000389d8, unit_arg=0x62b00003cdf8) at /data/src/bb-10.6-mdev17399-hf/sql/sql_select.cc:1261
 
#20 0x00005638f7440d35 in st_select_lex_unit::prepare_join (this=0x62b00003cdf8, thd_arg=0x62b000069288, sl=0x62b0000389d8, tmp_result=0x62b00003df58, additional_options=0, is_union_select=false) at /data/src/bb-10.6-mdev17399-hf/sql/sql_union.cc:1088
 
#21 0x00005638f7444969 in st_select_lex_unit::prepare (this=0x62b00003cdf8, derived_arg=0x62b00003d660, sel_result=0x62b00003df58, additional_options=0) at /data/src/bb-10.6-mdev17399-hf/sql/sql_union.cc:1481
 
#22 0x00005638f70753dc in mysql_derived_prepare (thd=0x62b000069288, lex=0x62b00006d398, derived=0x62b00003d660) at /data/src/bb-10.6-mdev17399-hf/sql/sql_derived.cc:817
 
#23 0x00005638f707161e in mysql_handle_single_derived (lex=0x62b00006d398, derived=0x62b00003d660, phases=2) at /data/src/bb-10.6-mdev17399-hf/sql/sql_derived.cc:206
 
#24 0x00005638f74d0489 in TABLE_LIST::handle_derived (this=0x62b00003d660, lex=0x62b00006d398, phases=2) at /data/src/bb-10.6-mdev17399-hf/sql/table.cc:9205
 
#25 0x00005638f70ba388 in LEX::handle_list_of_derived (this=0x62b00006d398, table_list=0x62b00003d660, phases=2) at /data/src/bb-10.6-mdev17399-hf/sql/sql_lex.h:4431
 
#26 0x00005638f70e082d in st_select_lex::handle_derived (this=0x62b0000383d8, lex=0x62b00006d398, phases=2) at /data/src/bb-10.6-mdev17399-hf/sql/sql_lex.cc:4944
 
#27 0x00005638f720606a in JOIN::prepare (this=0x62900025dab8, tables_init=0x62b00003d660, conds_init=0x0, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x62b0000383d8, unit_arg=0x62b00006d460) at /data/src/bb-10.6-mdev17399-hf/sql/sql_select.cc:1160
 
#28 0x00005638f722d0df in mysql_select (thd=0x62b000069288, tables=0x62b00003d660, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x62b00003df28, unit=0x62b00006d460, select_lex=0x62b0000383d8) at /data/src/bb-10.6-mdev17399-hf/sql/sql_select.cc:4739
 
#29 0x00005638f71fe3e0 in handle_select (thd=0x62b000069288, lex=0x62b00006d398, result=0x62b00003df28, setup_tables_done_option=0) at /data/src/bb-10.6-mdev17399-hf/sql/sql_select.cc:419
 
#30 0x00005638f7167f26 in execute_sqlcom_select (thd=0x62b000069288, all_tables=0x62b00003d660) at /data/src/bb-10.6-mdev17399-hf/sql/sql_parse.cc:6231
 
#31 0x00005638f7157208 in mysql_execute_command (thd=0x62b000069288) at /data/src/bb-10.6-mdev17399-hf/sql/sql_parse.cc:3927
 
#32 0x00005638f71731e5 in mysql_parse (thd=0x62b000069288, rawbuf=0x62b0000382a8 "SELECT * FROM ( SELECT * FROM t1 JOIN t2 ON (b IN( SELECT x FROM (SELECT 1 AS c) AS sq1))) AS sq2", length=97, parser_state=0x7fd5901d3bb0) at /data/src/bb-10.6-mdev17399-hf/sql/sql_parse.cc:8006
 
#33 0x00005638f7149c0c in dispatch_command (command=COM_QUERY, thd=0x62b000069288, packet=0x629000258289 "SELECT * FROM ( SELECT * FROM t1 JOIN t2 ON (b IN( SELECT x FROM (SELECT 1 AS c) AS sq1))) AS sq2", packet_length=97, blocking=true) at /data/src/bb-10.6-mdev17399-hf/sql/sql_parse.cc:1888
 
#34 0x00005638f7146947 in do_command (thd=0x62b000069288, blocking=true) at /data/src/bb-10.6-mdev17399-hf/sql/sql_parse.cc:1399
 
#35 0x00005638f75895ea in do_handle_one_connection (connect=0x61100000b388, put_in_cache=true) at /data/src/bb-10.6-mdev17399-hf/sql/sql_connect.cc:1410
 
#36 0x00005638f7588f47 in handle_one_connection (arg=0x61100000b248) at /data/src/bb-10.6-mdev17399-hf/sql/sql_connect.cc:1312
 
#37 0x00005638f829cdbf in pfs_spawn_thread (arg=0x616000101b08) at /data/src/bb-10.6-mdev17399-hf/storage/perfschema/pfs.cc:2201
 
#38 0x00007fd5999e8609 in start_thread (arg=<optimized out>) at pthread_create.c:477
 
#39 0x00007fd5995bc293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Not reproducible on 10.6 main tree, it throws ER_BAD_FIELD_ERROR ("Unknown column 'x' in 'field list'") as expected.

 Comments   
Comment by Sergei Petrunia [ 2021-04-08 ]

We crash when trying to derefence this in Item_field::fix_outer_field:

    place= prev_subselect_item->parsing_place;

Look at the query:

SELECT * 
FROM 
( 
   SELECT * 
   FROM 
     t1 JOIN t2 ON (b IN (SELECT x 
                          FROM 
                            (SELECT 1 AS c) AS sq1
 
                         )
 
                   )
 
) AS sq2;

The subquery is in the ON expression.
We try to resolve "x" from "SELECT x".
It cannot be found locally
or int its parent query.
Note that the parent query has the subquery in the ON clause, so we're using
the ON expression's Name_resolution_context.

also note that the query that has "t1 JOIN t2 ON ..." is inside a derived
table.
References out of derived tables are not allowed, so one would expect the
ON expression's Name_resolution_context to have outer_context=NULL, but
this isn't the case.

And this is directly responsible for the crash. These lines:

    Item_subselect *prev_subselect_item=
 
      last_checked_context->select_lex->master_unit()->item;
 
    ...
 
    place= prev_subselect_item->parsing_place;

assume that we only resolve references out of scalar-context subqueries, not
out of derived tables.

Comment by Sergei Petrunia [ 2021-04-08 ]

Why does ON-expression's Name_resolution_context have a wrong value in outer_context , while WHERE's Name_resolution_context doesn't have this problem?

ON-expression's Name_resolution_context is created in push_new_name_resolution_context().
outer_context is set like so:

  on_context->outer_context = outer_sel ? &outer_sel->context : 0;

This is wrong but one can't to do any better as the SELECT structure is not constructed, yet. That is, 

curr_select->master=curr_select->master_unit() = NULL
 
curr_select->linkage=UNSPECIFIED_TYPE

and we can't tell what kind of subquery we're in.

As for WHERE's Name_resolution_context, it has:

lex->context_stack.head()->outer_context=NULL.

this is correct.
But then execution proceeds and reaches LEX::parsed_subselect,st_select_lex::register_unit: 

  for(SELECT_LEX *sel= unit->first_select();sel; sel= sel->next_select())
 
  {
 
    sel->context.outer_context= outer_context;
 
  }
 
}

Here "sel" is the derived subquery, outer_context is the context of the top-level SELECT, so it clearly does the wrong thing!

After that, execution reaches st_select_lex_node::set_linkage(). Now, it becomes possible to tell what kind of subquery it was.

But then, we reach this in mysql_derived_prepare():

    /* prevent name resolving out of derived table */
 
    for (SELECT_LEX *sl= first_select; sl; sl= sl->next_select())
 
    { 
      sl->context.outer_context= 0;

and this is how SELECT's Name_resolution_context gets the right value of outer_context field.

Comment by Sergei Petrunia [ 2021-04-08 ]

http://lists.askmonty.org/pipermail/commits/2021-April/014544.html

Comment by Oleksandr Byelkin [ 2021-04-12 ]

OK to push after making the Lambda separate function

Comment by Sergei Petrunia [ 2021-04-12 ]

Pushed into the JSON_TABLE tree.

Comment by Igor Babaev [ 2021-04-14 ]

After adding a fix for the bug MDEV-25362 to 10.2 we can observe the crash of /MDEV-25346 in 10.2+ as well

Comment by Sergei Petrunia [ 2021-04-14 ]

igor can we separate this effort from the JSON_TABLE work? Either file another MDEV that covers fixing the issue in the earlier versions, or clear this MDEV from any mention of JSON_TABLE ?

Comment by Alice Sherepa [ 2021-08-27 ]

currently error 1054: Unknown column 'x' in 'field list' on 10.2-10.6 (10.2 228630f61ac10240c3671), but MDEV-26412 seems to be the same bug and with that testcase server crashes on 10.4-10.6

Comment by Alice Sherepa [ 2023-07-06 ]

not reproducible on 10.4-10.11, 11.0, all branches return Unknown column 'x' in 'field list' , as expected

Generated at Thu Feb 08 09:37:00 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.