Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11321

MariaRocks: type_binary_indexes, type_blob_indexes fail due to different index statistics

Details

    Description

      They fail like this

      rocksdb.type_binary_indexes              [ fail ]
              Test ended at 2016-11-20 22:55:50
       
      CURRENT_TEST: rocksdb.type_binary_indexes
      --- /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_binary_indexes.result       2016-10-06 17:30:25.595958776 +0000
      +++ /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_binary_indexes.reject       2016-11-20 22:55:49.679353957 +0000
      @@ -39,7 +39,7 @@
       SHOW INDEX IN t1;
       Table  Non_unique      Key_name        Seq_in_index    Column_name     Collation       Cardinality     Sub_part        Packed  Null    Index_type  Comment Index_comment
       t1     0       PRIMARY 1       pk      A       1000    NULL    NULL            LSMTREE
      -t1     1       v16     1       v16     A       500     10      NULL    YES     LSMTREE
      +t1     1       v16     1       v16     A       1000    10      NULL    YES     LSMTREE
       INSERT INTO t1 (b,b20,v16,v128,pk) VALUES ('a','char1','varchar1a','varchar1b',1),('a','char2','varchar2a','varchar2b',2),('b','char3','varchar1a','varchar1b',3),('c','char4','varchar3a','varchar3b',4),('d','char5','varchar4a','varchar3b',5),('e','char6','varchar2a','varchar3b',6);
       INSERT INTO t1 (b,b20,v16,v128,pk) SELECT b,b20,v16,v128,pk+100 FROM t1;
       EXPLAIN SELECT HEX(SUBSTRING(v16,0,3)) FROM t1 WHERE v16 LIKE 'varchar%';
      

      ...

      rocksdb.type_blob_indexes                w2 [ fail ]
              Test ended at 2016-11-18 22:41:49
       
      CURRENT_TEST: rocksdb.type_blob_indexes
      --- /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_blob_indexes.result	2016-10-06 17:30:25.595958776 +0000
      +++ /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_blob_indexes.reject	2016-11-18 22:41:49.527347338 +0000
      @@ -105,7 +105,7 @@
       SHOW INDEX IN t1;
       Table	Non_unique	Key_name	Seq_in_index	Column_name	Collation	Cardinality	Sub_part	Packed	Null	Index_type	Comment	Index_comment
       t1	0	PRIMARY	1	pk	A	1000	NULL	NULL		LSMTREE		
      -t1	1	m	1	m	A	500	128	NULL	YES	LSMTREE		
      +t1	1	m	1	m	A	1000	128	NULL	YES	LSMTREE		
       INSERT INTO t1 (b,t,m,l) VALUES
       ('','','',''),
       ('a','b','c','d'),
      

      The difference in results doesn't look like a failure, but we need to investigate why it happens.

      Attachments

        Issue Links

          Activity

            Note that these are not real stats.
            Both tests start the server with

            --rocksdb_debug_optimizer_n_rows=1000 --rocksdb_records_in_range=50
            

            and then display stats for empty tables:

            SET GLOBAL ROCKSDB_PAUSE_BACKGROUND_WORK = 1;
            DROP TABLE IF EXISTS t1;
            CREATE TABLE t1 (b BINARY,
            b20 BINARY(20) PRIMARY KEY,
            v16 VARBINARY(16),
            v128 VARBINARY(128)
            ) ENGINE=rocksdb;
            SHOW INDEX IN t1;
            Table	Non_unique	Key_name	Seq_in_index	Column_name	Collation	Cardinality	Sub_part	Packed	Null	Index_type	Comment	Index_comment
            t1	0	PRIMARY	1	b20	A	1000	NULL	NULL		LSMTREE		
            

            psergei Sergei Petrunia added a comment - Note that these are not real stats. Both tests start the server with --rocksdb_debug_optimizer_n_rows=1000 --rocksdb_records_in_range=50 and then display stats for empty tables: SET GLOBAL ROCKSDB_PAUSE_BACKGROUND_WORK = 1; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (b BINARY, b20 BINARY(20) PRIMARY KEY, v16 VARBINARY(16), v128 VARBINARY(128) ) ENGINE=rocksdb; SHOW INDEX IN t1; Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment Index_comment t1 0 PRIMARY 1 b20 A 1000 NULL NULL LSMTREE

            The stat numbers are calculated here in ha_rocksdb.cc

                      // Fake cardinality implementation. For example, (idx1, idx2, idx3) index
                      // will have rec_per_key for (idx1)=4, (idx1,2)=2, and (idx1,2,3)=1.
                      // rec_per_key for the whole index is 1, and multiplied by 2^n if
                      // n suffix columns of the index are not used.
                      x = 1 << (k->ext_key_parts-j-1);
            

            In both servers, j=0.

            The issue is that MariaDB has k->ext_key_parts==1, while MySQL has k->actual_key_parts==2

            psergei Sergei Petrunia added a comment - The stat numbers are calculated here in ha_rocksdb.cc // Fake cardinality implementation. For example, (idx1, idx2, idx3) index // will have rec_per_key for (idx1)=4, (idx1,2)=2, and (idx1,2,3)=1. // rec_per_key for the whole index is 1, and multiplied by 2^n if // n suffix columns of the index are not used. x = 1 << (k->ext_key_parts-j-1); In both servers, j=0. The issue is that MariaDB has k->ext_key_parts==1 , while MySQL has k->actual_key_parts==2

            MariaDB:

            (gdb) p *k
              $73 = {key_length = 13, flags = 104, user_defined_key_parts = 1,
             usable_key_parts = 1, ext_key_parts = 1, ext_key_flags = 104, 
            ext_key_part_map = 0, block_size = 0, name_length = 3, algorithm = HA_KEY_ALG_UNDEF, 
            is_statistics_from_stat_tables = false, {parser = 0x0, parser_name = 0x0}, 
            key_part = 0x7fffeb708ef8, name = 0x7fffeb705359 "v16", 
            cache_name = 0x7fffeb7054e0 "test", rec_per_key = 0x7fffeb705368, 
            read_stats = 0x0, collected_stats = 0x0, handler = {bdb_return_if_eq = 0}, table = 0x7ffff4d7bc70,
             comment = {str = 0x0, length = 0}, option_list = 0x0, option_struct = 0x0}
            

            MySQL:

            (gdb) p *k
              $57 = {key_length = 13, flags = 104, actual_flags = 105, user_defined_key_parts = 1, 
            actual_key_parts = 2, unused_key_parts = 0, usable_key_parts = 2, block_size = 0, 
            algorithm = HA_KEY_ALG_UNDEF, {parser = 0x0, parser_name = 0x0}, 
            key_part = 0x7fffe0019b58, contains_document_key_part = false, name = 0x7fffe0017441 "v16",
             rec_per_key = 0x7fffe0017450, handler = {bdb_return_if_eq = 0}, table = 0x7fffe00181d0, 
            comment = {str = 0x0, length = 0}}
            

            So, MariaDB thinks that the key is not "extended" with PK.

            psergei Sergei Petrunia added a comment - MariaDB: (gdb) p *k $73 = {key_length = 13, flags = 104, user_defined_key_parts = 1, usable_key_parts = 1, ext_key_parts = 1, ext_key_flags = 104, ext_key_part_map = 0, block_size = 0, name_length = 3, algorithm = HA_KEY_ALG_UNDEF, is_statistics_from_stat_tables = false, {parser = 0x0, parser_name = 0x0}, key_part = 0x7fffeb708ef8, name = 0x7fffeb705359 "v16", cache_name = 0x7fffeb7054e0 "test", rec_per_key = 0x7fffeb705368, read_stats = 0x0, collected_stats = 0x0, handler = {bdb_return_if_eq = 0}, table = 0x7ffff4d7bc70, comment = {str = 0x0, length = 0}, option_list = 0x0, option_struct = 0x0} MySQL: (gdb) p *k $57 = {key_length = 13, flags = 104, actual_flags = 105, user_defined_key_parts = 1, actual_key_parts = 2, unused_key_parts = 0, usable_key_parts = 2, block_size = 0, algorithm = HA_KEY_ALG_UNDEF, {parser = 0x0, parser_name = 0x0}, key_part = 0x7fffe0019b58, contains_document_key_part = false, name = 0x7fffe0017441 "v16", rec_per_key = 0x7fffe0017450, handler = {bdb_return_if_eq = 0}, table = 0x7fffe00181d0, comment = {str = 0x0, length = 0}} So, MariaDB thinks that the key is not "extended" with PK.
            psergei Sergei Petrunia added a comment - I already saw this thing, this is MariaDB not using extended keys for partially-covered columns: https://jira.mariadb.org/browse/MDEV-9658?focusedCommentId=87896&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-87896

            Debugged this - MyRocks already has the code to handle unique secondary indexes. For these, SQL layer also thinks they are not "extended". In MariaRocks, that code will handle the keys with partially-covered columns.

            psergei Sergei Petrunia added a comment - Debugged this - MyRocks already has the code to handle unique secondary indexes. For these, SQL layer also thinks they are not "extended". In MariaRocks, that code will handle the keys with partially-covered columns.

            MDEV-11477 was also caused by this extended keys property.

            psergei Sergei Petrunia added a comment - MDEV-11477 was also caused by this extended keys property.

            People

              psergei Sergei Petrunia
              psergei Sergei Petrunia
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.