[MDEV-14862] Server crashes in Bitmap<64u>::merge / add_key_field Created: 2018-01-03  Updated: 2020-05-11  Resolved: 2018-02-01

Status: Closed
Project: MariaDB Server
Component/s: Optimizer, Views
Affects Version/s: 10.0, 10.1, 10.2
Fix Version/s: 10.0.35, 10.1.31, 10.2.13, 10.3.5

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 0
Labels: upstream-fixed

Issue Links:
Relates
relates to MDEV-22498 SIGSEGV in Bitmap<64u>::merge on SELECT Closed
Sprint: 10.0.34

 Description   

--source include/have_innodb.inc
 
CREATE TABLE t1 (a INT) ENGINE=InnoDB;
CREATE VIEW v1 AS SELECT * FROM t1;
CREATE TABLE t2 (b INT) ENGINE=InnoDB;
DELETE FROM v1 WHERE a IN ( SELECT a FROM t2 );

10.0 51e4650ed0644e

#3  <signal handler called>
#4  0x00000000005f6b48 in Bitmap<64u>::merge (this=0x180, map2=...) at /data/src/10.0/sql/sql_bitmap.h:158
#5  0x0000000000689db2 in add_key_field (join=0x7f39733ae198, key_fields=0x7f3989085d68, and_level=0, cond=0x7f39733af450, field=0x7f39730289a8, eq_func=true, value=0x7f39733af4e8, num_values=1, usable_tables=18446744073709551615, sargables=0x7f3989085e40) at /data/src/10.0/sql/sql_select.cc:4410
#6  0x000000000068a25f in add_key_equal_fields (join=0x7f39733ae198, key_fields=0x7f3989085d68, and_level=0, cond=0x7f39733af450, field_item=0x7f39731a5e50, eq_func=true, val=0x7f39733af4e8, num_values=1, usable_tables=18446744073709551615, sargables=0x7f3989085e40) at /data/src/10.0/sql/sql_select.cc:4559
#7  0x000000000068ae0e in add_key_fields (join=0x7f39733ae198, key_fields=0x7f3989085d68, and_level=0x7f3989085d74, cond=0x7f39733af450, usable_tables=18446744073709551615, sargables=0x7f3989085e40) at /data/src/10.0/sql/sql_select.cc:4791
#8  0x000000000068c26a in update_ref_and_keys (thd=0x7f397b757070, keyuse=0x7f3989085e20, join_tab=0x7f39733aeaf8, tables=1, cond=0x7f39733af450, normal_tables=18446744073709551615, select_lex=0x7f39731a4888, sargables=0x7f3989085e40) at /data/src/10.0/sql/sql_select.cc:5262
#9  0x00000000006bb0f4 in JOIN::reoptimize (this=0x7f39733ae198, added_where=0x7f39733af450, join_tables=1, save_to=0x0) at /data/src/10.0/sql/sql_select.cc:24897
#10 0x00000000007b4c5a in JOIN::choose_subquery_plan (this=0x7f39733ae198, join_tables=1) at /data/src/10.0/sql/opt_subselect.cc:5834
#11 0x0000000000689243 in make_join_statistics (join=0x7f39733ae198, tables_list=..., conds=0x0, keyuse_array=0x7f39733ae4c0) at /data/src/10.0/sql/sql_select.cc:4074
#12 0x000000000067fad1 in JOIN::optimize_inner (this=0x7f39733ae198) at /data/src/10.0/sql/sql_select.cc:1345
#13 0x000000000067e9ca in JOIN::optimize (this=0x7f39733ae198) at /data/src/10.0/sql/sql_select.cc:1029
#14 0x000000000063e68e in st_select_lex::optimize_unflattened_subqueries (this=0x7f397b75b0f8, const_only=false) at /data/src/10.0/sql/sql_lex.cc:3553
#15 0x0000000000990bbc in mysql_delete (thd=0x7f397b757070, table_list=0x7f39731a4190, conds=0x7f39733ae840, order_list=0x7f397b75b360, limit=18446744073709551615, options=0, result=0x7f39731a5c58) at /data/src/10.0/sql/sql_delete.cc:292
#16 0x000000000064c1b9 in mysql_execute_command (thd=0x7f397b757070) at /data/src/10.0/sql/sql_parse.cc:3602
#17 0x0000000000653f16 in mysql_parse (thd=0x7f397b757070, rawbuf=0x7f39731a4088 "DELETE FROM v1 WHERE a IN ( SELECT a FROM t2 )", length=46, parser_state=0x7f3989087640) at /data/src/10.0/sql/sql_parse.cc:6569
#18 0x0000000000646a55 in dispatch_command (command=COM_QUERY, thd=0x7f397b757070, packet=0x7f39811dc071 "DELETE FROM v1 WHERE a IN ( SELECT a FROM t2 )", packet_length=46) at /data/src/10.0/sql/sql_parse.cc:1296
#19 0x0000000000645d55 in do_command (thd=0x7f397b757070) at /data/src/10.0/sql/sql_parse.cc:999
#20 0x0000000000765db4 in do_handle_one_connection (thd_arg=0x7f397b757070) at /data/src/10.0/sql/sql_connect.cc:1377
#21 0x0000000000765b26 in handle_one_connection (arg=0x7f397b757070) at /data/src/10.0/sql/sql_connect.cc:1292
#22 0x0000000000a00c4e in pfs_spawn_thread (arg=0x7f397b707670) at /data/src/10.0/storage/perfschema/pfs.cc:1861
#23 0x00007f3988cce064 in start_thread (arg=0x7f3989088700) at pthread_create.c:309
#24 0x00007f39870f462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Reproducible with MariaDB 10.x; not reproducible with MariaDB 5.5, MySQL 5.6.

Note: I think it can be the same problem as was filed quite long ago for MySQL as https://bugs.mysql.com/bug.php?id=73556 (private report, but its header was "fatal_signal in add_key_equal_fields() with UPDATE VIEW using outer subquery", and apparently its internal clone was #19434916. If it's indeed the same, there is a fix for it in MySQL 5.6 tree:

commit 415faa122b9c683661dafac82fff414fa6864151 f83d9e4217827b4d0b078bc93dff7d8f173adbef
Author: Sreeharsha Ramanavarapu <sreeharsha.ramanavarapu@oracle.com>
Date:   Thu Oct 1 07:45:27 2015 +0530
 
    Bug #19434916: FATAL_SIGNAL IN ADD_KEY_EQUAL_FIELDS() WITH
                   UPDATE VIEW USING OUTER SUBQUERY
    
    Issue:
    -----
    While resolving a column which refers to a table/view in an
    outer query, it's respecitve item object is marked with the
    outer query's select_lex object. But when the column refers
    to a view or if the column is part of a subquery in the
    HAVING clause, an Item_ref object is created. While the
    reference to the outer query is stored by the Item_ref
    object, the same is not stored in it's real_item.
    
    This creates a problem with the IN-TO-EXISTS optmization.
    When there is an index over the column in the inner query,
    it will be considered since the column's real_item object
    will be mistaken for a local field. This will lead to a
    crash.
    
    SOLUTION:
    ---------
    Under the current design, the only way to fix this issue is
    to check the reginfo.join_tab for a NULL value. If yes, the
    query should not be worrying about the key use.
    
    The testcase and comments added as part of the fix for
    Bug#17766653 have been backported.

However, it's just my guess, I didn't check whether MySQL 5.6 fails with the provided test case before the fix, and whether it's the patch above that fixes it.



 Comments   
Comment by Oleksandr Byelkin [ 2018-01-23 ]

somehow it stuck in the code which create possible indexes... why it went there is not clear (I do not see possible temporary tables here). And it try to process field a of table t1.

Comment by Oleksandr Byelkin [ 2018-01-23 ]

Problem is probably in "merge for insert"...

Comment by Oleksandr Byelkin [ 2018-01-23 ]

The problem appeared during optimization the subquery, so the question is why it try to find key of a.t1 i.e. outer field.

Comment by Oleksandr Byelkin [ 2018-01-23 ]

So the MySQL bug and this one is the same and it looks like analysis in the bug is right. Fix of the bug solves the problem.

Comment by Oleksandr Byelkin [ 2018-01-23 ]

revision-id: 6f78e6f98e470c874f691e130a5d74968af06063 (mariadb-10.0.33-50-g6f78e6f98e4)
parent(s): a4663af05c1d1bd3abb537205df070ed2158e702
author: Oleksandr Byelkin
committer: Oleksandr Byelkin
timestamp: 2018-01-23 21:38:29 +0100
message:

MDEV-14862: Server crashes in Bitmap<64u>::merge / add_key_field

It is backport of MySQL fix:

Bug #19434916: FATAL_SIGNAL IN ADD_KEY_EQUAL_FIELDS() WITH
UPDATE VIEW USING OUTER SUBQUERY

Issue:


While resolving a column which refers to a table/view in an
outer query, it's respecitve item object is marked with the
outer query's select_lex object. But when the column refers
to a view or if the column is part of a subquery in the
HAVING clause, an Item_ref object is created. While the
reference to the outer query is stored by the Item_ref
object, the same is not stored in it's real_item.

This creates a problem with the IN-TO-EXISTS optmization.
When there is an index over the column in the inner query,
it will be considered since the column's real_item object
will be mistaken for a local field. This will lead to a
crash.

SOLUTION:
---------
Under the current design, the only way to fix this issue is
to check the reginfo.join_tab for a NULL value. If yes, the
query should not be worrying about the key use.

The testcase and comments added as part of the fix for
Bug#17766653 have been backported.

Comment by Oleksandr Byelkin [ 2018-01-23 ]

github: bb-10.0-MDEV-14862

Comment by Igor Babaev [ 2018-01-28 ]

Sent my analysis to Sanja:

In the function Item_in_subselect::create_single_in_to_exists_cond()

at the line
Item item= (Item) select_lex->item_list.head()->real_item();

we have
(gdb) p dbug_print_select(select_lex)
$82 = 0x1c489c0 <dbug_item_print_buf> "select t1.a from test.t2"

gdb) p (Item*) select_lex->item_list.head()->used_tables()
$83 = (Item *) 0x4000000000000000

(gdb) p (Item*) select_lex->item_list.head()>real_item()>used_tables()
$84 = (Item *) 0x1

This is the cause of the problem: we have to build the expressions with
select_lex->item_list.head(), not with select_lex->item_list.head()->real_item()
if select_lex->item_list.head() is just Item_direct_ref (a wrapper for a view/derived).

Comment by Oleksandr Byelkin [ 2018-01-29 ]

revision-id: d337368470eb935ed69ecc441626e021b70aba0d (mariadb-10.0.33-69-gd337368470e)
parent(s): d01dbe66a8bf9cb6031f95159c49100f9299a768
author: Oleksandr Byelkin
committer: Oleksandr Byelkin
timestamp: 2018-01-29 09:44:17 +0100
message:

MDEV-14862: Server crashes in Bitmap<64u>::merge / add_key_field

Do not unwrapp view references to keep table info.
(Row-IN subselect do not unwrap items so it does not need fix)

Generated at Thu Feb 08 08:16:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.