[MDEV-19266] Crash in EITS code when enabling 128 indexes Created: 2019-04-17  Updated: 2019-04-18  Resolved: 2019-04-18

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 10.4
Fix Version/s: 10.4.5

Type: Bug Priority: Major
Reporter: Sergei Petrunia Assignee: Sergei Petrunia
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MDEV-19235 MariaDB Server compiled for 128 Index... Closed

 Description   

Got this from Wlad: range_vs_index_merge test fails in 128-index build.

Can be reproduced on bb-10.4-wlad with 

cmake . -DCMAKE_BUILD_TYPE=Debug -DWITHOUT_MROONGA:bool=1 -DWITHOUT_TOKUDB:bool=1 -DMAX_INDEXES=128

The stack trace:

  #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
 
  #1  0x00007ffff52b2801 in __GI_abort () at abort.c:79
 
  #2  0x00007ffff52a239a in __assert_fail_base (fmt=0x7ffff54297d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x55555689b584 "bit < (map)->n_bits", file=file@entry=0x55555689b550 "/home/psergey/dev-git/10.4-wlad/include/my_bitmap.h", line=line@entry=115, function=function@entry=0x55555689ec60 <_ZZ14bitmap_set_bitE19__PRETTY_FUNCTION__> "void bitmap_set_bit(MY_BITMAP*, uint)") at assert.c:92
 
  #3  0x00007ffff52a2412 in __GI___assert_fail (assertion=0x55555689b584 "bit < (map)->n_bits", file=0x55555689b550 "/home/psergey/dev-git/10.4-wlad/include/my_bitmap.h", line=115, function=0x55555689ec60 <_ZZ14bitmap_set_bitE19__PRETTY_FUNCTION__> "void bitmap_set_bit(MY_BITMAP*, uint)") at assert.c:101
 
  #4  0x0000555555c3a3f7 in bitmap_set_bit (map=0x7fffdc104f88, bit=32767) at /home/psergey/dev-git/10.4-wlad/include/my_bitmap.h:115
 
  #5  0x0000555555c544a4 in Bitmap<128u>::set_bit (this=0x7fffdc104f88, n=32767) at /home/psergey/dev-git/10.4-wlad/sql/sql_bitmap.h:46
 
  #6  0x00005555561940b6 in and_range_trees (param=0x7fffee1c1ff0, tree1=0x7fffdc1200c8, tree2=0x7fffdc1203e8, result=0x7fffdc120308) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:9047
 
  #7  0x000055555617defd in SEL_IMERGE::and_sel_tree (this=0x7fffdc120058, param=0x7fffee1c1ff0, tree=0x7fffdc11f7c8, new_imerge=0x7fffdc120298) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:605
 
  #8  0x000055555617ed35 in imerge_list_and_tree (param=0x7fffee1c1ff0, merges=0x7fffdc11ffd0, tree=0x7fffdc11f7c8, replace=true) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:1148
 
  #9  0x00005555561942a6 in tree_and (param=0x7fffee1c1ff0, tree1=0x7fffdc11f7c8, tree2=0x7fffdc11ff78) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:9148
 
  #10 0x0000555556191024 in Item_cond_and::get_mm_tree (this=0x7fffdc01a3d0, param=0x7fffee1c1ff0, cond_ptr=0x7fffdc01a610) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:8220
 
  #11 0x000055555619120f in Item_cond::get_mm_tree (this=0x7fffdc01a508, param=0x7fffee1c1ff0, cond_ptr=0x7fffdc01b9e0) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:8258
 
  #12 0x0000555556184b94 in calculate_cond_selectivity_for_table (thd=0x7fffdc000d60, table=0x7fffdc1036a0, cond=0x7fffdc01b9e0) at /home/psergey/dev-git/10.4-wlad/sql/opt_range.cc:3412
 
  #13 0x0000555555d30184 in make_join_statistics (join=0x7fffdc01b5d8, tables_list=..., keyuse_array=0x7fffdc01b8c8) at /home/psergey/dev-git/10.4-wlad/sql/sql_select.cc:5393
 
  #14 0x0000555555d249f8 in JOIN::optimize_inner (this=0x7fffdc01b5d8) at /home/psergey/dev-git/10.4-wlad/sql/sql_select.cc:2191
 
  #15 0x0000555555d225a0 in JOIN::optimize (this=0x7fffdc01b5d8) at /home/psergey/dev-git/10.4-wlad/sql/sql_select.cc:1561
 
  #16 0x0000555555d2d44a in mysql_select (thd=0x7fffdc000d60, tables=0x7fffdc018270, wild_num=1, fields=..., conds=0x7fffdc01a508, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748612, result=0x7fffdc01af60, unit=0x7fffdc004c90, select_lex=0x7fffdc017c10) at /home/psergey/dev-git/10.4-wlad/sql/sql_select.cc:4588
 
  #17 0x0000555555d6822f in mysql_explain_union (thd=0x7fffdc000d60, unit=0x7fffdc004c90, result=0x7fffdc01af60) at /home/psergey/dev-git/10.4-wlad/sql/sql_select.cc:26541
 
  #18 0x0000555555ce6f6b in execute_sqlcom_select (thd=0x7fffdc000d60, all_tables=0x7fffdc018270) at /home/psergey/dev-git/10.4-wlad/sql/sql_parse.cc:6541
 
  #19 0x0000555555cdc60d in mysql_execute_command (thd=0x7fffdc000d60) at /home/psergey/dev-git/10.4-wlad/sql/sql_parse.cc:3891
 
  #20 0x0000555555ceb0f7 in mysql_parse (thd=0x7fffdc000d60, rawbuf=0x7fffdc017a98 "EXPLAIN\nSELECT * FROM t1 FORCE KEY (PRIMARY,f3,f4)\nWHERE ( f3 = 1 OR f1 = 7 ) AND f1 < 10\nOR f3 BETWEEN 2 AND 2 AND ( f3 = 1 OR f4 != 1 )", length=137, parser_state=0x7fffee1c4160, is_com_multi=false, is_next_command=false) at /home/psergey/dev-git/10.4-wlad/sql/sql_parse.cc:8154
 
  #21 0x0000555555cd67bf in dispatch_command (command=COM_QUERY, thd=0x7fffdc000d60, packet=0x7fffdc00ac51 "", packet_length=137, is_com_multi=false, is_next_command=false) at /home/psergey/dev-git/10.4-wlad/sql/sql_parse.cc:1831
 
  #22 0x0000555555cd4f7e in do_command (thd=0x7fffdc000d60) at /home/psergey/dev-git/10.4-wlad/sql/sql_parse.cc:1364
 
  #23 0x0000555555e4eeb3 in do_handle_one_connection (connect=0x555558106b60) at /home/psergey/dev-git/10.4-wlad/sql/sql_connect.cc:1398
 
  #24 0x0000555555e4ec17 in handle_one_connection (arg=0x555558106b60) at /home/psergey/dev-git/10.4-wlad/sql/sql_connect.cc:1301
 
  #25 0x000055555677a08e in pfs_spawn_thread (arg=0x55555814ef90) at /home/psergey/dev-git/10.4-wlad/storage/perfschema/pfs.cc:1862
 
  #26 0x00007ffff61ad6db in start_thread (arg=0x7fffee1c5700) at pthread_create.c:463
 
  #27 0x00007ffff539388f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

The crash happens because param->real_keynr[n] has garbage values for all values except 0.

 Comments   
Comment by Sergei Petrunia [ 2019-04-17 ]

EITS code works as follows:

  • create pseudo-indexes for each of the column we are getting the estimates for (check out create_key_parts_for_pseudo_indexes)
  • run the range analyzer

However, calculate_cond_selectivity_for_table only sets the first element in param.real_keynr:

param.real_keynr[0]= 0;

Comment by Sergei Petrunia [ 2019-04-17 ]

The stack trace shows it is trying to construct an index merge plan. Considering that the EITS estimator is only able to use individual range scans, does this make any sense?

But this question is orthogonal to the issue of param->real_keynr[i] not being set. It is suspicious that they need real_keynr value, EITS estimator uses pseudo indexes...

Comment by Sergei Petrunia [ 2019-04-17 ]

Attempt at fix #1 that I have provided to Wlad:

diff --git a/sql/opt_range.cc b/sql/opt_range.cc
 
index d7bbdeb82a4..d7e8637a9e6 100644
 
--- a/sql/opt_range.cc
 
+++ b/sql/opt_range.cc
 
@@ -3060,7 +3060,10 @@ bool calculate_cond_selectivity_for_table(THD *thd, TABLE *table, Item **cond)
 
     param.prev_tables= param.read_tables= 0;
 
     param.current_table= table->map;
 
     param.using_real_indexes= FALSE;
 
-    param.real_keynr[0]= 0;
 
+
 
+    for (uint keyno=0; keyno < param.keys; keyno++)
 
+      param.real_keynr[keyno]= keyno;
 
+
 
     param.alloced_sel_args= 0;
 
 
     thd->no_errors=1;

Comment by Sergei Petrunia [ 2019-04-17 ]

.... and this patch is probably incorrect.

The assert happens here:

      if (key && key->type == SEL_ARG::IMPOSSIBLE)
 
      {
 
	result->type= SEL_TREE::IMPOSSIBLE;
 
        param->table->with_impossible_ranges.set_bit(param->real_keynr[key_no]);
 
        DBUG_RETURN(1);
 
      }

with the above patch, param->real_keynr[key_no] is "the number of column that we do EITS estimation for".
it is not relevant to any index number.

Got to check what is with_impossible_ranges.

Comment by Sergei Petrunia [ 2019-04-17 ]

with_impossible_ranges was added by:

commit 658128af43b4d7c6db445164f8ed25ed4d1e3109
 
Author:	Igor Babaev <igor@askmonty.org>  Mon Feb  4 00:56:12 2019
 
Committer:	Igor Babaev <igor@askmonty.org>  Mon Feb  4 00:56:12 2019
 
 
 
MDEV-16188 Use in-memory PK filters built from range index scans

Comment by Sergei Petrunia [ 2019-04-17 ]

An updated fix:

diff --git a/sql/opt_range.cc b/sql/opt_range.cc
 
index 5ab3d70214d..ed9cd541f70 100644
 
--- a/sql/opt_range.cc
 
+++ b/sql/opt_range.cc
 
@@ -9044,7 +9044,11 @@ int and_range_trees(RANGE_OPT_PARAM *param, SEL_TREE *tree1, SEL_TREE *tree2,
 
       if (key && key->type == SEL_ARG::IMPOSSIBLE)
 
       {
 
        result->type= SEL_TREE::IMPOSSIBLE;
 
-        param->table->with_impossible_ranges.set_bit(param->real_keynr[key_no]);
 
+        if (param->using_real_indexes)
 
+        {
 
+          param->table->with_impossible_ranges.set_bit(param->
 
+                                                       real_keynr[key_no]);
 
+        }
 
         DBUG_RETURN(1);
 
       }
 
       result_keys.set_bit(key_no);

Comment by Sergei Petrunia [ 2019-04-17 ]

The whole with_impossible_ranges code looks odd to me. So one can add something like " AND (t.key>10 AND t.key < 5)" to a query, and that would affect the query plan somehow?

Comment by Sergei Petrunia [ 2019-04-17 ]

http://lists.askmonty.org/pipermail/commits/2019-April/013649.html

Comment by Sergei Petrunia [ 2019-04-17 ]

Igor, please review

Comment by Igor Babaev [ 2019-04-17 ]

Ok to push into 10.4

Comment by Igor Babaev [ 2019-04-17 ]

Sergei, Wlad
The last patch is good only for 10.4, because with_impossible_ranges was introduced in 10.4.
The above stack was shown for 10.4.
Do you really have problems with 10.2, 10.3?

Comment by Vladislav Vaintroub [ 2019-04-17 ]

I only tried with 10.4, I do not have problems with 10.2

Comment by Sergei Petrunia [ 2019-04-18 ]

I've set affects-versions to include 10.2 and 10.3 based on the first variant of the fix (which was in EITS code so I assumed earlier versions would be affected as well).

Generated at Thu Feb 08 08:50:19 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.