NOT EXISTS to IN (part of exists2in transformation for 10.0) (MDEV-38)

[MDEV-3881] Endless loop for query with EXISTS predicate using outer reference to view Created: 2012-11-25  Updated: 2023-09-08  Resolved: 2023-09-06

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: None
Fix Version/s: 10.0.1, N/A

Type: Technical task Priority: Major
Reporter: Elena Stepanova Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-22534 Trivial correlation detection/removal... In Review

 Description    

#0  0x00000000005c3752 in Item_ref::real_item (this=0x0) at 10.0-exists2in/sql/item.h:2920
 
2920      Item *real_item()
 
#0  0x00000000005c3752 in Item_ref::real_item (this=0x0) at 10.0-exists2in/sql/item.h:2920
 
#1  0x00000000005c3793 in Item_ref::real_item (this=0x3908828) at 10.0-exists2in/sql/item.h:2922
 
...
 
#8846 0x00000000005c3793 in Item_ref::real_item (this=0x3908828) at 10.0-exists2in/sql/item.h:2922
 
#8847 0x00000000005c3793 in Item_ref::real_item (this=0x3908828) at 10.0-exists2in/sql/item.h:2922
 
#8848 0x00000000007fb4a1 in Item_direct_view_ref::replace_equal_field (this=0x3908828, arg=0x7fecfcc6b3c0 "\230\226\220\003") at 10.0-exists2in/sql/item.cc:8051
 
#8849 0x00000000007f8538 in Item_ref::transform (this=0x3908828, transformer=&virtual Item::replace_equal_field(unsigned char*), arg=0x7fecfcc6b3c0 "\230\226\220\003") at 10.0-exists2in/sql/item.cc:7006
 
#8850 0x000000000080bb53 in Item_in_optimizer::transform (this=0x3908628, transformer=&virtual table offset 824, argument=0x7fecfcc6b3c0 "\230\226\220\003") at 10.0-exists2in/sql/item_cmpfunc.cc:1837
 
#8851 0x0000000000814e26 in Item_cond::transform (this=0x3909258, transformer=&virtual table offset 824, arg=0x7fecfcc6b3c0 "\230\226\220\003") at 10.0-exists2in/sql/item_cmpfunc.cc:4456
 
#8852 0x0000000000832546 in Item_func::transform (this=0x3918098, transformer=&virtual table offset 824, argument=0x7fecfcc6b3c0 "\230\226\220\003") at 10.0-exists2in/sql/item_func.cc:360
 
#8853 0x0000000000663255 in substitute_for_best_equal_field (context_tab=0x1, cond=0x3918098, cond_equal=0x39030a8, table_join_idx=0x3909fd8) at 10.0-exists2in/sql/sql_select.cc:12490
 
#8854 0x0000000000662f4c in substitute_for_best_equal_field (context_tab=0x1, cond=0x3902fb8, cond_equal=0x39030a8, table_join_idx=0x3909fd8) at 10.0-exists2in/sql/sql_select.cc:12443
 
#8855 0x0000000000646a8a in JOIN::optimize_inner (this=0x3905948) at 10.0-exists2in/sql/sql_select.cc:1361
 
#8856 0x0000000000645414 in JOIN::optimize (this=0x3905948) at 10.0-exists2in/sql/sql_select.cc:994
 
#8857 0x000000000064d0f2 in mysql_select (thd=0x38cb8a0, rref_pointer_array=0x38cec68, tables=0x38facd8, wild_num=1, fields=..., conds=0x3902fb8, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x3914648, unit=0x38ce2c0, select_lex=0x38ce9b8) at 10.0-exists2in/sql/sql_select.cc:3193
 
#8858 0x000000000064387e in handle_select (thd=0x38cb8a0, lex=0x38ce210, result=0x3914648, setup_tables_done_option=0) at 10.0-exists2in/sql/sql_select.cc:362
 
#8859 0x000000000061b13b in execute_sqlcom_select (thd=0x38cb8a0, all_tables=0x38facd8) at 10.0-exists2in/sql/sql_parse.cc:4937
 
#8860 0x00000000006137a8 in mysql_execute_command (thd=0x38cb8a0) at 10.0-exists2in/sql/sql_parse.cc:2421
 
#8861 0x000000000061d9f6 in mysql_parse (thd=0x38cb8a0, rawbuf=0x3759598 "SELECT * FROM t1, v WHERE NOT EXISTS ( SELECT * FROM t3 WHERE c = b ) AND a = b", length=79, parser_state=0x7fecfcc6c4e0) at 10.0-exists2in/sql/sql_parse.cc:6056
 
#8862 0x000000000061099a in dispatch_command (command=COM_QUERY, thd=0x38cb8a0, packet=0x38d5ab1 "SELECT * FROM t1, v WHERE NOT EXISTS ( SELECT * FROM t3 WHERE c = b ) AND a = b", packet_length=79) at 10.0-exists2in/sql/sql_parse.cc:1216
 
#8863 0x000000000060fb93 in do_command (thd=0x38cb8a0) at 10.0-exists2in/sql/sql_parse.cc:945
 
#8864 0x000000000071a57a in do_handle_one_connection (thd_arg=0x38cb8a0) at 10.0-exists2in/sql/sql_connect.cc:1254
 
#8865 0x0000000000719f4a in handle_one_connection (arg=0x38cb8a0) at 10.0-exists2in/sql/sql_connect.cc:1168
 

branch: 10.0-exists2in
 
revision-id: psergey@askmonty.org-20121123063639-1jqbkjuxevqe1vb6
 
date: 2012-11-23 10:36:39 +0400
 
revno: 3482

Reproducible with MyISAM, Aria.

Minimal optimizer_switch: in_to_exists=on,exists_to_in=on
Full optimizer_switch (default + exists_to_in=on):

index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off,exists_to_in=on

EXPLAIN causes the same problem.

Test case:

SET optimizer_switch = 'exists_to_in=on';
 
 
 
CREATE TABLE t1 (a INT) ENGINE=MyISAM;
 
INSERT INTO t1 VALUES (1),(7);
 
 
 
CREATE TABLE t2 (b INT) ENGINE=MyISAM;
 
INSERT INTO t2 VALUES (8);
 
CREATE ALGORITHM=MERGE VIEW v AS SELECT * FROM t2;
 
 
 
CREATE TABLE t3 (c INT) ENGINE=MyISAM;
 
INSERT INTO t3 VALUES (4),(6);
 
 
 
SELECT * FROM t1, v WHERE NOT EXISTS ( SELECT * FROM t3 WHERE c = b ) AND a = b;
 
 

Variation of the test case (subquery instead of the view):

SET optimizer_switch = 'exists_to_in=on';
 
 
 
CREATE TABLE t1 (a INT) ENGINE=MyISAM;
 
INSERT INTO t1 VALUES (1),(7);
 
 
 
CREATE TABLE t2 (b INT) ENGINE=MyISAM;
 
INSERT INTO t2 VALUES (8);
 
 
 
CREATE TABLE t3 (c INT) ENGINE=MyISAM;
 
INSERT INTO t3 VALUES (4),(6);
 
 
 
SELECT * FROM t1, ( SELECT * FROM t2 ) alias WHERE NOT EXISTS ( SELECT * FROM t3 WHERE c = b ) AND a = b;
 
 



 Comments   
Comment by Oleksandr Byelkin [ 2012-11-27 ]

It crashes in calling virtual method of Item_view_direct_ref created for view field. According to the debug log and data in the fields of the object it should be valid.

Comment by Oleksandr Byelkin [ 2012-11-27 ]

Valgrind did not show something wrong.

Comment by Elena Stepanova [ 2012-11-27 ]

Just to make sure – have you noticed that it's, first of all, a loop? The stack trace contains almost 9K frames, most of which are in Item_ref::real_item.

Comment by Oleksandr Byelkin [ 2012-11-28 ]

Replacing equal fileds replace field where the Item_direct_view_ref points with pointers on itself. sot *(Item_ref::ref) points back on the Item.

Comment by Oleksandr Byelkin [ 2012-11-28 ]

Item_direct_view_ref is best choice to replace bacause it is a constant (8 in the example).

Comment by Oleksandr Byelkin [ 2012-11-28 ]

This patch is wrong because changes results:
=== modified file 'sql/item_cmpfunc.h'
— sql/item_cmpfunc.h 2012-11-12 20:14:16 +0000
+++ sql/item_cmpfunc.h 2012-11-28 19:27:20 +0000
@@ -1718,7 +1718,8 @@ class Item_equal: public Item_bool_func
Item_equal(Item *f1, Item *f2, bool with_const_item);
Item_equal(Item_equal *item_equal);
/* Currently the const item is always the first in the list of equal items */

  • inline Item* get_const() { return with_const ? equal_items.head() : NULL; }

    + inline Item* get_const()
    +

    { return with_const ? equal_items.head()->real_item() : NULL; }

    void add_const(Item *c, Item *f = NULL);
    /** Add a non-constant item to the multiple equality */
    void add(Item *f)

    { equal_items.push_back(f); }

Let's try to check that items are the same when build multy equality...

Comment by Oleksandr Byelkin [ 2012-11-29 ]

The problem was in unwrapping view/subquery-in-the-FROM-clase fields in substitution.

Comment by Igor Babaev [ 2023-09-06 ]

Reopened by mistake. I was confused by the attached commit from MDEV-22534 that has nothing to do with this bug.

Generated at Thu Feb 08 06:51:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.