[MDEV-11321] MariaRocks: type_binary_indexes, type_blob_indexes fail due to different index statistics Created: 2016-11-21  Updated: 2016-12-02  Resolved: 2016-12-02

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - RocksDB
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Sergei Petrunia Assignee: Sergei Petrunia
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MDEV-9658 Make MyRocks in MariaDB stable Closed
Relates
relates to MDEV-11477 MariaRocks: rocksdb.type_varchar failure Closed

 Description   

They fail like this

rocksdb.type_binary_indexes              [ fail ]
 
        Test ended at 2016-11-20 22:55:50
 
 
 
CURRENT_TEST: rocksdb.type_binary_indexes
 
--- /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_binary_indexes.result       2016-10-06 17:30:25.595958776 +0000
 
+++ /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_binary_indexes.reject       2016-11-20 22:55:49.679353957 +0000
 
@@ -39,7 +39,7 @@
 
 SHOW INDEX IN t1;
 
 Table  Non_unique      Key_name        Seq_in_index    Column_name     Collation       Cardinality     Sub_part        Packed  Null    Index_type  Comment Index_comment
 
 t1     0       PRIMARY 1       pk      A       1000    NULL    NULL            LSMTREE
 
-t1     1       v16     1       v16     A       500     10      NULL    YES     LSMTREE
 
+t1     1       v16     1       v16     A       1000    10      NULL    YES     LSMTREE
 
 INSERT INTO t1 (b,b20,v16,v128,pk) VALUES ('a','char1','varchar1a','varchar1b',1),('a','char2','varchar2a','varchar2b',2),('b','char3','varchar1a','varchar1b',3),('c','char4','varchar3a','varchar3b',4),('d','char5','varchar4a','varchar3b',5),('e','char6','varchar2a','varchar3b',6);
 
 INSERT INTO t1 (b,b20,v16,v128,pk) SELECT b,b20,v16,v128,pk+100 FROM t1;
 
 EXPLAIN SELECT HEX(SUBSTRING(v16,0,3)) FROM t1 WHERE v16 LIKE 'varchar%';

...

rocksdb.type_blob_indexes                w2 [ fail ]
 
        Test ended at 2016-11-18 22:41:49
 
 
 
CURRENT_TEST: rocksdb.type_blob_indexes
 
--- /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_blob_indexes.result	2016-10-06 17:30:25.595958776 +0000
 
+++ /home/psergey/dev-git/10.2-mariarocks/storage/rocksdb/mysql-test/rocksdb/r/type_blob_indexes.reject	2016-11-18 22:41:49.527347338 +0000
 
@@ -105,7 +105,7 @@
 
 SHOW INDEX IN t1;
 
 Table	Non_unique	Key_name	Seq_in_index	Column_name	Collation	Cardinality	Sub_part	Packed	Null	Index_type	Comment	Index_comment
 
 t1	0	PRIMARY	1	pk	A	1000	NULL	NULL		LSMTREE		
-t1	1	m	1	m	A	500	128	NULL	YES	LSMTREE		
+t1	1	m	1	m	A	1000	128	NULL	YES	LSMTREE		
 INSERT INTO t1 (b,t,m,l) VALUES
 
 ('','','',''),
 
 ('a','b','c','d'),

The difference in results doesn't look like a failure, but we need to investigate why it happens.

 Comments   
Comment by Sergei Petrunia [ 2016-11-21 ]

Note that these are not real stats.
Both tests start the server with

--rocksdb_debug_optimizer_n_rows=1000 --rocksdb_records_in_range=50

and then display stats for empty tables:

SET GLOBAL ROCKSDB_PAUSE_BACKGROUND_WORK = 1;
 
DROP TABLE IF EXISTS t1;
 
CREATE TABLE t1 (b BINARY,
 
b20 BINARY(20) PRIMARY KEY,
 
v16 VARBINARY(16),
 
v128 VARBINARY(128)
 
) ENGINE=rocksdb;
 
SHOW INDEX IN t1;
 
Table	Non_unique	Key_name	Seq_in_index	Column_name	Collation	Cardinality	Sub_part	Packed	Null	Index_type	Comment	Index_comment
 
t1	0	PRIMARY	1	b20	A	1000	NULL	NULL		LSMTREE

Comment by Sergei Petrunia [ 2016-11-21 ]

The stat numbers are calculated here in ha_rocksdb.cc

          // Fake cardinality implementation. For example, (idx1, idx2, idx3) index
 
          // will have rec_per_key for (idx1)=4, (idx1,2)=2, and (idx1,2,3)=1.
 
          // rec_per_key for the whole index is 1, and multiplied by 2^n if
 
          // n suffix columns of the index are not used.
 
          x = 1 << (k->ext_key_parts-j-1);

In both servers, j=0.

The issue is that MariaDB has k->ext_key_parts==1, while MySQL has k->actual_key_parts==2

Comment by Sergei Petrunia [ 2016-11-21 ]

MariaDB:

(gdb) p *k
 
  $73 = {key_length = 13, flags = 104, user_defined_key_parts = 1,
 
 usable_key_parts = 1, ext_key_parts = 1, ext_key_flags = 104, 
ext_key_part_map = 0, block_size = 0, name_length = 3, algorithm = HA_KEY_ALG_UNDEF, 
is_statistics_from_stat_tables = false, {parser = 0x0, parser_name = 0x0}, 
key_part = 0x7fffeb708ef8, name = 0x7fffeb705359 "v16", 
cache_name = 0x7fffeb7054e0 "test", rec_per_key = 0x7fffeb705368, 
read_stats = 0x0, collected_stats = 0x0, handler = {bdb_return_if_eq = 0}, table = 0x7ffff4d7bc70,
 
 comment = {str = 0x0, length = 0}, option_list = 0x0, option_struct = 0x0}

MySQL:

(gdb) p *k
 
  $57 = {key_length = 13, flags = 104, actual_flags = 105, user_defined_key_parts = 1, 
actual_key_parts = 2, unused_key_parts = 0, usable_key_parts = 2, block_size = 0, 
algorithm = HA_KEY_ALG_UNDEF, {parser = 0x0, parser_name = 0x0}, 
key_part = 0x7fffe0019b58, contains_document_key_part = false, name = 0x7fffe0017441 "v16",
 
 rec_per_key = 0x7fffe0017450, handler = {bdb_return_if_eq = 0}, table = 0x7fffe00181d0, 
comment = {str = 0x0, length = 0}}

So, MariaDB thinks that the key is not "extended" with PK.

Comment by Sergei Petrunia [ 2016-11-21 ]

I already saw this thing, this is MariaDB not using extended keys for partially-covered columns:

https://jira.mariadb.org/browse/MDEV-9658?focusedCommentId=87896&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-87896

Comment by Sergei Petrunia [ 2016-12-02 ]

Debugged this - MyRocks already has the code to handle unique secondary indexes. For these, SQL layer also thinks they are not "extended". In MariaRocks, that code will handle the keys with partially-covered columns.

Comment by Sergei Petrunia [ 2016-12-02 ]

MDEV-11477 was also caused by this extended keys property.

Generated at Thu Feb 08 07:49:02 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.