Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.1(EOL), 10.2(EOL), 10.3(EOL), 10.4(EOL)
Description
The dataset used
create table t0 (a int); |
INSERT INTO t0 VALUES (0),(0),(0),(0),(2),(0),(0),(1),(1),(0); |
|
CREATE TABLE t1 ( |
a int(11) DEFAULT NULL, |
b int(11) DEFAULT NULL, |
d int(11) DEFAULT NULL, |
KEY b_3 (a,d), |
KEY b_4 (a,b) |
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; |
insert into t1 select A.a , B.a, C.a from t0 A, t0 B, t0 C, t0 D; |
MariaDB [test]> select count(*) from t1;
|
+----------+
|
| count(*) |
|
+----------+
|
| 10000 |
|
+----------+
|
Now running the query without any limit
MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b;
|
+------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+
|
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |
|
+------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+
|
| 1 | SIMPLE | t1 | ref | b_3,b_4 | b_3 | 10 | const,const | 200 | 200.00 | 100.00 | 100.00 | Using where; Using filesort |
|
+------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+
|
Now running the query with limit= NUMBER OF TABLE RECORDS
MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 10000;
|
+------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+
|
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |
|
+------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+
|
| 1 | SIMPLE | t1 | range | b_3,b_4 | b_4 | 5 | NULL | 2000 | 2000.00 | 10.00 | 10.00 | Using where |
|
+------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+
|
This looks incorrect, we definitely have a problem with the cost model that changes ref access -> index(or range) by which we can do the ORDERING.
With limit = 2x table_records
MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 20000;
|
+------+-------------+-------+-------+---------------+------+---------+-------------+-------+----------+----------+------------+-------------+
|
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |
|
+------+-------------+-------+-------+---------------+------+---------+-------------+-------+----------+----------+------------+-------------+
|
| 1 | SIMPLE | t1 | index | b_3,b_4 | b_4 | 10 | const,const | 10266 | 10000.00 | 1.95 | 2.00 | Using where |
|
+------+-------------+-------+-------+---------------+------+---------+-------------+-------+----------+----------+------------+-------------+
|
Attachments
Issue Links
- relates to
-
MDEV-8306 Complete cost-based optimization for ORDER BY with LIMIT
-
- Stalled
-
-
MDEV-17761 Odd optimizer choice with ORDER BY LIMIT and condition selectivity
-
- Stalled
-
Activity
Field | Original Value | New Value |
---|---|---|
Description |
The dataset used
{code:sql} create table t0 (a int); INSERT INTO t0 VALUES (0),(0),(0),(0),(2),(0),(0),(1),(1),(0); CREATE TABLE t1 ( a int(11) DEFAULT NULL, b int(11) DEFAULT NULL, d int(11) DEFAULT NULL, KEY b_3 (a,d), KEY b_4 (a,b) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; insert into t1 select A.a , B.a, C.a from t0 A, t0 B, t0 C, t0 D; {code} MariaDB [test]> select count(*) from t1; +----------+ | count(*) | +----------+ | 10000 | +----------+ Now running the query without any limit MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b; +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | 1 | SIMPLE | t1 | ref | b_3,b_4 | b_3 | 10 | const,const | 200 | 200.00 | 100.00 | 100.00 | Using where; Using filesort | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ Now running the query with limit= NUMBER OF TABLE RECORDS MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 10000; +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | 1 | SIMPLE | t1 | range | b_3,b_4 | b_4 | 5 | NULL | 2000 | 2000.00 | 10.00 | 10.00 | Using where | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ This looks incorrect, we definitely have a problem with the cost model that changes ref access -> index(or range) by which we can do the ORDERING. |
Description |
The dataset used
{code:sql} create table t0 (a int); INSERT INTO t0 VALUES (0),(0),(0),(0),(2),(0),(0),(1),(1),(0); CREATE TABLE t1 ( a int(11) DEFAULT NULL, b int(11) DEFAULT NULL, d int(11) DEFAULT NULL, KEY b_3 (a,d), KEY b_4 (a,b) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; insert into t1 select A.a , B.a, C.a from t0 A, t0 B, t0 C, t0 D; {code} MariaDB [test]> select count(*) from t1; +----------+ | count(*) | +----------+ | 10000 | +----------+ Now running the query without any limit MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b; +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | 1 | SIMPLE | t1 | ref | b_3,b_4 | b_3 | 10 | const,const | 200 | 200.00 | 100.00 | 100.00 | Using where; Using filesort | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ Now running the query with limit= NUMBER OF TABLE RECORDS MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 10000; +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | 1 | SIMPLE | t1 | range | b_3,b_4 | b_4 | 5 | NULL | 2000 | 2000.00 | 10.00 | 10.00 | Using where | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ This looks incorrect, we definitely have a problem with the cost model that changes ref access -> index(or range) by which we can do the ORDERING. |
The dataset used
{code:sql} create table t0 (a int); INSERT INTO t0 VALUES (0),(0),(0),(0),(2),(0),(0),(1),(1),(0); CREATE TABLE t1 ( a int(11) DEFAULT NULL, b int(11) DEFAULT NULL, d int(11) DEFAULT NULL, KEY b_3 (a,d), KEY b_4 (a,b) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; insert into t1 select A.a , B.a, C.a from t0 A, t0 B, t0 C, t0 D; {code} {noformat} MariaDB [test]> select count(*) from t1; +----------+ | count(*) | +----------+ | 10000 | +----------+ {noformat} Now running the query without any limit {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b; +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | 1 | SIMPLE | t1 | ref | b_3,b_4 | b_3 | 10 | const,const | 200 | 200.00 | 100.00 | 100.00 | Using where; Using filesort | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ {noformat} Now running the query with limit= NUMBER OF TABLE RECORDS {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 10000; +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | 1 | SIMPLE | t1 | range | b_3,b_4 | b_4 | 5 | NULL | 2000 | 2000.00 | 10.00 | 10.00 | Using where | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ {noformat} This looks incorrect, we definitely have a problem with the cost model that changes ref access -> index(or range) by which we can do the ORDERING. |
Description |
The dataset used
{code:sql} create table t0 (a int); INSERT INTO t0 VALUES (0),(0),(0),(0),(2),(0),(0),(1),(1),(0); CREATE TABLE t1 ( a int(11) DEFAULT NULL, b int(11) DEFAULT NULL, d int(11) DEFAULT NULL, KEY b_3 (a,d), KEY b_4 (a,b) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; insert into t1 select A.a , B.a, C.a from t0 A, t0 B, t0 C, t0 D; {code} {noformat} MariaDB [test]> select count(*) from t1; +----------+ | count(*) | +----------+ | 10000 | +----------+ {noformat} Now running the query without any limit {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b; +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | 1 | SIMPLE | t1 | ref | b_3,b_4 | b_3 | 10 | const,const | 200 | 200.00 | 100.00 | 100.00 | Using where; Using filesort | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ {noformat} Now running the query with limit= NUMBER OF TABLE RECORDS {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 10000; +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | 1 | SIMPLE | t1 | range | b_3,b_4 | b_4 | 5 | NULL | 2000 | 2000.00 | 10.00 | 10.00 | Using where | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ {noformat} This looks incorrect, we definitely have a problem with the cost model that changes ref access -> index(or range) by which we can do the ORDERING. |
The dataset used
{code:sql} create table t0 (a int); INSERT INTO t0 VALUES (0),(0),(0),(0),(2),(0),(0),(1),(1),(0); CREATE TABLE t1 ( a int(11) DEFAULT NULL, b int(11) DEFAULT NULL, d int(11) DEFAULT NULL, KEY b_3 (a,d), KEY b_4 (a,b) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; insert into t1 select A.a , B.a, C.a from t0 A, t0 B, t0 C, t0 D; {code} {noformat} MariaDB [test]> select count(*) from t1; +----------+ | count(*) | +----------+ | 10000 | +----------+ {noformat} Now running the query without any limit {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b; +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ | 1 | SIMPLE | t1 | ref | b_3,b_4 | b_3 | 10 | const,const | 200 | 200.00 | 100.00 | 100.00 | Using where; Using filesort | +------+-------------+-------+------+---------------+------+---------+-------------+------+--------+----------+------------+-----------------------------+ {noformat} Now running the query with limit= NUMBER OF TABLE RECORDS {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 10000; +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ | 1 | SIMPLE | t1 | range | b_3,b_4 | b_4 | 5 | NULL | 2000 | 2000.00 | 10.00 | 10.00 | Using where | +------+-------------+-------+-------+---------------+------+---------+------+------+---------+----------+------------+-------------+ {noformat} This looks incorrect, we definitely have a problem with the cost model that changes ref access -> index(or range) by which we can do the ORDERING. With limit = 2x table_records {noformat} MariaDB [test]> analyze select a,b,d from t1 where a=1 and d=2 order by b limit 20000; +------+-------------+-------+-------+---------------+------+---------+-------------+-------+----------+----------+------------+-------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra | +------+-------------+-------+-------+---------------+------+---------+-------------+-------+----------+----------+------------+-------------+ | 1 | SIMPLE | t1 | index | b_3,b_4 | b_4 | 10 | const,const | 10266 | 10000.00 | 1.95 | 2.00 | Using where | +------+-------------+-------+-------+---------------+------+---------+-------------+-------+----------+----------+------------+-------------+ {noformat} |
Labels | order-by-optimization |
Link | This issue relates to MDEV-17761 [ MDEV-17761 ] |
Status | Open [ 1 ] | In Progress [ 3 ] |
Support case ID | 27078 |
Assignee | Varun Gupta [ varun ] | Sergei Petrunia [ psergey ] |
Status | In Progress [ 3 ] | In Review [ 10002 ] |
Assignee | Sergei Petrunia [ psergey ] | Varun Gupta [ varun ] |
Status | In Review [ 10002 ] | Stalled [ 10000 ] |
Fix Version/s | 10.1.42 [ 23407 ] | |
Fix Version/s | 10.2.28 [ 23910 ] | |
Fix Version/s | 10.3.19 [ 23908 ] | |
Fix Version/s | 10.4.9 [ 23906 ] | |
Fix Version/s | 10.2 [ 14601 ] | |
Fix Version/s | 10.1 [ 16100 ] | |
Fix Version/s | 10.3 [ 22126 ] | |
Fix Version/s | 10.4 [ 22408 ] | |
Resolution | Fixed [ 1 ] | |
Status | Stalled [ 10000 ] | Closed [ 6 ] |
Workflow | MariaDB v3 [ 91420 ] | MariaDB v4 [ 155415 ] |
Some numbers from the optimizer trace
{
"reconsidering_access_paths_for_index_ordering": {
"clause": "ORDER BY",
"fanout": 1,
"read_time": 82.001,
"table_name": "t1",
"rows_estimation": 200,
"possible_keys": [
{
"index": "b_3",
"can_resolve_order": false,
"cause": "not_usable_index_for_the query"
},
{
"index": "b_4",
"can_resolve_order": true,
"updated_limit": 10266,
"range_scan_time": 82,
"index_scan_time": 82,
"records": 2000,
"chosen": true
}
]
}
}