[MDEV-20576] A new assertion added to check validity of calculated selectivity values fails Created: 2019-09-12  Updated: 2019-09-24  Resolved: 2019-09-15

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.2.28, 10.1.42, 10.3.19, 10.4.9, 10.5.0

Type: Bug Priority: Major
Reporter: Igor Babaev Assignee: Igor Babaev
Resolution: Fixed Votes: 0
Labels: eits

Issue Links:
Blocks
blocks MDEV-20519 Query plan regression with optimizer_... Closed
Relates
relates to MDEV-20519 Query plan regression with optimizer_... Closed
relates to MDEV-20595 Assertion `0 < sel && sel <= 2.0' fai... Stalled

 Description   

After adding the assertion

 
DBUG_ASSERT(0.0 < sel && sel <= 1.0);

to check the validity of selectivity values returned by the function
table_cond_selectivity() the following test case from selectivity.test fails:

CREATE TABLE t1 (a varchar(16), b int, PRIMARY KEY(a), KEY(b));
INSERT INTO t1 VALUES
  ('USAChinese',10), ('USAEnglish',20), ('USAFrench',30);
 
CREATE TABLE t2 (i int);
INSERT INTO t2 VALUES
  (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(1),(2),(3),(4);
ANALYZE TABLE t1, t2;
 
set use_stat_tables='preferably';
set optimizer_use_condition_selectivity=3;
 
EXPLAIN EXTENDED
SELECT * FROM t1, t2
  WHERE  a <> 'USARussian' AND b IS NULL;

The added assertion also fails for the test cases:

set @@optimizer_use_condition_selectivity=2;
 
set names utf8;
 
CREATE DATABASE world;
 
use world;
 
--source include/world_schema.inc
 
--disable_query_log
--disable_result_log
--disable_warnings
--source include/world.inc
--enable_warnings
--enable_result_log
--enable_query_log
 
CREATE INDEX Name ON City(Name);
CREATE INDEX CountryPopulation ON City(Country,Population);
CREATE INDEX CountryName ON City(Country,Name);
 
--disable_query_log
--disable_result_log
--disable_warnings
ANALYZE TABLE City;
--enable_warnings
--enable_result_log
--enable_query_log
 
set @@optimizer_use_condition_selectivity=2;
 
EXPLAIN
SELECT * FROM City WHERE Country='FIN';
 
DROP DATABASE world;
 
CREATE TABLE t1 (
  a INT,
  b INT NOT NULL,
  c char(100),
  KEY (b, c),
  KEY (b, a, c)
)
DEFAULT CHARSET = utf8;
 
INSERT INTO t1 VALUES 
(1,  1, 1),
(2,  2, 2),
(3,  3, 3),
(4,  4, 4),
(5,  5, 5),
(6,  6, 6),
(7,  7, 7),
(8,  8, 8),
(9,  9, 9);
 
INSERT INTO t1 SELECT a + 10,  b, c FROM t1;
INSERT INTO t1 SELECT a + 20,  b, c FROM t1;
INSERT INTO t1 SELECT a + 40,  b, c FROM t1;
INSERT INTO t1 SELECT a + 80,  b, c FROM t1;
INSERT INTO t1 SELECT a + 160, b, c FROM t1;
INSERT INTO t1 SELECT a + 320, b, c FROM t1;
INSERT INTO t1 SELECT a + 640, b, c FROM t1;
INSERT INTO t1 SELECT a + 1280, b, c FROM t1 LIMIT 80;
 
EXPLAIN
SELECT a FROM t1 WHERE b = 1 ORDER BY c DESC LIMIT 9;
SELECT a FROM t1 WHERE b = 1 ORDER BY c DESC LIMIT 9;
 
set optimizer_use_condition_selectivity=@save_optimizer_use_condition_selectivity;
 
DROP TABLE t1;

The latter two test cases are variants of the test cases taken from range_vs_index_merge.test and order_by.test correspondingly.
They has to be run with the setting

set @@optimizer_use_condition_selectivity=2;

to make the added assertion fire.



 Comments   
Comment by Igor Babaev [ 2019-09-15 ]

A fix for this bug was pushed into 10.1

Comment by Marko Mäkelä [ 2019-09-23 ]

The assertion is failing in a merge to 10.4 in at least 33 tests, which is reported as MDEV-20595. I will comment out the assertions, to be re-enabled when merging the fix:

diff --git a/sql/sql_select.cc b/sql/sql_select.cc
index 57a66b7fdb1..f4ec49257af 100644
--- a/sql/sql_select.cc
+++ b/sql/sql_select.cc
@@ -9087,7 +9087,7 @@ double table_cond_selectivity(JOIN *join, uint idx, JOIN_TAB *s,
           something went wrong.
 	*/
         sel /= (double)table->quick_rows[key] / (double) table->stat_records();
-        DBUG_ASSERT(0 < sel && sel <= 2.0);
+        // MDEV-20595 FIXME: DBUG_ASSERT(0 < sel && sel <= 2.0);
         set_if_smaller(sel, 1.0);
         used_range_selectivity= true;
       }
@@ -9194,7 +9194,7 @@ double table_cond_selectivity(JOIN *join, uint idx, JOIN_TAB *s,
           if (field->cond_selectivity > 0)
 	  {
             sel/= field->cond_selectivity;
-            DBUG_ASSERT(0 < sel && sel <= 2.0);  
+            // MDEV-20595 FIXME: DBUG_ASSERT(0 < sel && sel <= 2.0);
             set_if_smaller(sel, 1.0);
           }
           break;

Generated at Thu Feb 08 09:00:32 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.