Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.3.3-1
Description
Currently if MariaDB checks whether possibility to use range access based on [NOT] IN predicate with a long list of values it build SEL_TREEs taking huge amount of memory.
Meanwhile using CTE specified by table value constructor such a predicate could be transformed into an equivalent [NOT] IN subquery:
SELECT ... WHERE ... (expr1, ...) [NOT] IN (value_list) ...;
|
=>
|
WITH t(col1, ...) AS (SELECT * FROM VALUES value_list)
|
SELECT ... WHERE ... (expr1, ...) [NOT] IN (SELECT * FROM t)...;
|
Attachments
Issue Links
- causes
-
MDEV-14281 Wrong result from query with NOT IN predicate in WHERE
-
- Closed
-
-
MDEV-17795 Query with long IN(...) list in WHERE about 40 times slower
-
- Closed
-
-
MDEV-20105 Case for bringing in_subquery_conversion_threshold back in next possible release
-
- Closed
-
-
MDEV-20900 IN predicate to IN subquery conversion causes performance regression
-
- Closed
-
-
MDEV-33560 Spider: query with long IN clause has performance issues
-
- Closed
-
- is blocked by
-
MDEV-12172 Implement tables specified by table value constructors.
-
- Closed
-
Final evaluation results:
( see the the list of commits here https://github.com/MariaDB/server/compare/10.3...shagalla:10.3-mdev12172 )
Optimization that transforms IN predicate into IN subquery that uses TVC was made.
It works this way:
SELECT ... WHERE (expr1, ...) [NOT] IN (value_list) ...;
=>
SELECT ... WHERE (expr1, ...) [NOT] IN (SELECT * FROM (VALUES value_list) AS tvc_name) ...;
Optimization works when IN predicate is used in WHERE clause of the query or when it is ON expression of some join. It also works when IN predicate is in derived tables, CTEs and subqueries.
What's done:
1. global variable in_subquery_conversion_threshold that controls optimization working
Optimization works only when the total number of scalar values used in IN predicate is more than this variable
2. transformer Item_func_in::in_predicate_to_in_subs_transformer that transforms IN predicate in IN subselect
3. creation of TVC specified by values from IN predicate
4. creation of new statement
SELECT * FROM (VALUES value_list) AS tvc_name
5. new test file: opt_tvc.test
To clone this project from github you need to do:
git clone https://github.com/shagalla/server 10.3-mdev12172
cd 10.3-mdev12172
git branch 10.3-mdev12172
git checkout 10.3-mdev12172
maria-git/10.3-mdev12172> git pull https://github.com/shagalla/server 10.3-mdev12172