Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
10.11
-
None
Description
A reference to a column of a merged derived table or view shares the item of the column's expression. With nested derived tables/views referring to columns several times, the item tree becomes a DAG with exponentially many paths.
MDEV-31130 makes the select list, GROUP BY, HAVING, DISTINCT, ORDER BY, window functions, INSERT and UPDATE traverse such expressions once. When the column is used in WHERE, ON, IN/EXISTS subqueries or a multi-table DELETE condition, the query is still exponential in the nesting depth, and with 20 levels it never finishes.
Test case (mysqltest)
create table t1 (f1 int, f2 int, f3 int, f4 int, f5 int);
|
insert into t1 values (1,2,3,4,5),(5,4,3,2,1),(3,3,3,3,3);
|
|
|
--disable_query_log
|
let $levels= 20;
|
let $q= select f1 as a, f1 or f2 as s1 from t1;
|
let $i= 1;
|
while ($i < $levels)
|
{
|
let $j= $i;
|
inc $j;
|
let $q= select a, s$i or s$i or s$i as s$j from ($q) dt$i;
|
inc $i;
|
}
|
let $s= s$levels;
|
|
|
--echo # fast
|
eval select count(*) from ($q) x;
|
--echo # each of these never finishes
|
eval select count(*) from ($q) x where $s > 0;
|
eval select count(*) from ($q) x join t1 on x.$s = t1.f3;
|
eval select count(*) from t1 left join ($q) x on x.$s = t1.f3;
|
eval select count(*) from t1 where f1 in (select $s from ($q) x);
|
eval select count(*) from t1 where exists (select 1 from ($q) x where x.$s = t1.f1);
|
eval delete t1 from t1, ($q) x where t1.f1 = x.a and x.$s < -1000000;
|
--enable_query_log
|
|
|
drop table t1;
|
With $levels= 6 all queries succeed with these results: 3, 3, 0, 3, 1, 1.
Test case (plain SQL)
create table t1 (f1 int, f2 int, f3 int, f4 int, f5 int); |
insert into t1 values (1,2,3,4,5),(5,4,3,2,1),(3,3,3,3,3); |
-- never finishes; without "where s20 > 0" it takes 0.1 s
|
select count(*) from (select a, s19 or s19 or s19 as s20 from (select a, s18 or s18 or s18 as s19 from (select a, s17 or s17 or s17 as s18 from (select a, s16 or s16 or s16 as s17 from (select a, s15 or s15 or s15 as s16 from (select a, s14 or s14 or s14 as s15 from (select a, s13 or s13 or s13 as s14 from (select a, s12 or s12 or s12 as s13 from (select a, s11 or s11 or s11 as s12 from (select a, s10 or s10 or s10 as s11 from (select a, s9 or s9 or s9 as s10 from (select a, s8 or s8 or s8 as s9 from (select a, s7 or s7 or s7 as s8 from (select a, s6 or s6 or s6 as s7 from (select a, s5 or s5 or s5 as s6 from (select a, s4 or s4 or s4 as s5 from (select a, s3 or s3 or s3 as s4 from (select a, s2 or s2 or s2 as s3 from (select a, s1 or s1 or s1 as s2 from (select f1 as a, f1 or f2 as s1 from t1) dt1) dt2) dt3) dt4) dt5) dt6) dt7) dt8) dt9) dt10) dt11) dt12) dt13) dt14) dt15) dt16) dt17) dt18) dt19) x where s20 > 0; |
The same happens with arithmetic, CASE, NULLIF, LIKE, IS NULL and IN() instead of OR.
Time per nesting depth
Debug build, 3 rows, OR with 3 references per level:
| depth | WHERE | ON | LEFT JOIN | IN | EXISTS | DELETE |
|---|---|---|---|---|---|---|
| 9 | 89 ms | 80 ms | 397 ms | 85 ms | 94 ms | 262 ms |
| 11 | 220 ms | 144 ms | 160 ms | 135 ms | 188 ms | 1251 ms |
| 13 | 651 ms | 600 ms | 829 ms | 636 ms | 841 ms | 10638 ms |
| 14 | 1746 ms | |||||
| 15 | 5208 ms | |||||
| 16 | 18662 ms |
Where the time goes
- WHERE / ON / EXISTS: walk(eval_not_null_tables) from st_select_lex::update_used_tables()
- LEFT JOIN: Item_func::fix_after_pullout() from mysql_derived_merge()
- IN subquery: Item_func::fix_after_pullout() from convert_subq_to_sj()
- DELETE: Item_func::transform() from Item::top_level_transform() / JOIN::transform_in_predicates_into_equalities()
walk() of eval_not_null_tables and fix_after_pullout() can be deduplicated like in MDEV-31130 (Item_traversal), but then the query still hangs in transform(). transform() probably cannot just skip a visited item, because it returns the item that replaces it. It would have to transform each shared item once and reuse the result for the other references.
Evaluating the merged expression per row is exponential as well. This is inherent to merging and is not in the scope of this bug.
Attachments
Issue Links
- relates to
-
MDEV-31130 INSERT-SELECT with many when/case/if conditions running forever
-
- In Review
-