[MDEV-14372] Fix and enable rocksdb.information_schema test - Jira

Details

Type: Task
Status: Open (View Workflow)
Priority: Major
Resolution: Unresolved
Fix Version/s: None
Component/s: Storage Engine - RocksDB, Tests
Labels:
None

Description

This is about enabling rocksdb.information_schema test. Currently, it is disabled because it fails like so:

CURRENT_TEST: rocksdb.information_schema

--- /home/psergey/dev-git/10.2/storage/rocksdb/mysql-test/rocksdb/r/information_schema.result   2017-09-21 10:40:03.019962115 +0000

+++ /home/psergey/dev-git/10.2/storage/rocksdb/mysql-test/rocksdb/r/information_schema.reject   2017-11-13 03:52:04.509919990 +0000

@@ -3,26 +3,22 @@

 DROP TABLE IF EXISTS t3;

 select * from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 TYPE   NAME    VALUE

-MAX_INDEX_ID   MAX_INDEX_ID    max_index_id

 CF_FLAGS       0       default [0]

 CF_FLAGS       1       __system__ [0]

 select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 count(*)

-3

+2

 select VALUE into @keysIn from INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS where CF_NAME = 'default' and LEVEL = 'Sum' and TYPE = 'KeyIn';

 CREATE TABLE t1 (i1 INT, i2 INT, PRIMARY KEY (i1)) ENGINE = ROCKSDB;

 INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3);

 select * from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 TYPE   NAME    VALUE

-BINLOG FILE    master-bin.000001

-BINLOG POS     1066

-BINLOG GTID    uuid:5

-MAX_INDEX_ID   MAX_INDEX_ID    max_index_id

+MAX_INDEX_ID   MAX_INDEX_ID    256

 CF_FLAGS       0       default [0]

 CF_FLAGS       1       __system__ [0]

 select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 count(*)

-6

+3

 set global rocksdb_force_flush_memtable_now = true;

 set global rocksdb_compact_cf='default';

 select case when VALUE-@keysIn >= 3 then 'true' else 'false' end from INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS where CF_NAME = 'default' and LEVEL = 'Sum' and TYPE = 'KeyIn';

@@ -66,7 +62,7 @@

 Variable_name  Value

 rocksdb_pause_background_work  ON

 DROP TABLE t3;

-cf_id:0,index_id:267

+cf_id:0,index_id:263

 SET GLOBAL ROCKSDB_PAUSE_BACKGROUND_WORK=0;

 SHOW GLOBAL VARIABLES LIKE 'ROCKSDB_PAUSE_BACKGROUND_WORK';

 Variable_name  Value

Attachments

Issue Links

relates to

MDEV-9658 Make MyRocks in MariaDB stable

Closed

MDEV-14892 Porting Rdb_binlog_manager to MariaDB

Open

Activity

Ascending order - Click to sort in descending order

Sergei Petrunia created issue - 2017-11-13 03:54

Sergei Petrunia made changes - 2017-11-13 03:54

Field	Original Value	New Value
Link		This issue relates to ~~MDEV-9658~~ [ ~~MDEV-9658~~ ]

Sergei Petrunia added a comment - 2017-11-13 03:55

The interesting part is here:

-BINLOG FILE    master-bin.000001

-BINLOG POS     1066

-BINLOG GTID    uuid:5

-MAX_INDEX_ID   MAX_INDEX_ID    max_index_id

+MAX_INDEX_ID   MAX_INDEX_ID    256

Sergei Petrunia added a comment - 2017-11-13 03:55 The interesting part is here: -BINLOG FILE master-bin.000001 -BINLOG POS 1066 -BINLOG GTID uuid:5 -MAX_INDEX_ID MAX_INDEX_ID max_index_id +MAX_INDEX_ID MAX_INDEX_ID 256

Sergei Petrunia added a comment - 2017-11-13 03:57

See also: https://jira.percona.com/browse/MYR-78 .
Percona had this as well.

Sergei Petrunia added a comment - 2017-11-13 03:57 See also: https://jira.percona.com/browse/MYR-78 . Percona had this as well.

Sergei Petrunia made changes - 2017-11-13 03:57

Assignee

Sergei Petrunia [ psergey ]

Sergei Petrunia made changes - 2018-01-12 13:09

Link

This issue relates to MDEV-14892 [ MDEV-14892 ]

Sergei Petrunia added a comment - 2018-01-12 16:46

Pushed a patch that fixes the above (and also a crash) but now it causes failures like this one:
http://buildbot.askmonty.org/buildbot/builders/kvm-deb-stretch-amd64/builds/2297/steps/mtr/logs/stdio

rocksdb.information_schema               w1 [ fail ]

        Test ended at 2018-01-12 10:06:03

CURRENT_TEST: rocksdb.information_schema

--- /usr/share/mysql/mysql-test/plugin/rocksdb/rocksdb/r/information_schema.result	2018-01-12 08:11:10.000000000 -0500

+++ /dev/shm/var/1/log/information_schema.reject	2018-01-12 10:06:03.800691646 -0500

@@ -8,21 +8,23 @@

 MAX_INDEX_ID	MAX_INDEX_ID	max_index_id

 CF_FLAGS	0	default [0]

 CF_FLAGS	1	__system__ [0]

-DDL_DROP_INDEX_ONGOING	cf_id:0,index_id:max_index_id

 select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 count(*)

-4

+3

 select VALUE into @keysIn from INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS where CF_NAME = 'default' and LEVEL = 'Sum' and TYPE = 'KeyIn';

 CREATE TABLE t1 (i1 INT, i2 INT, PRIMARY KEY (i1)) ENGINE = ROCKSDB;

 INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3);

 select * from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 TYPE	NAME	VALUE

+BINLOG	FILE	./master-bin.000002

+BINLOG	POS	1528

+BINLOG	GTID

 MAX_INDEX_ID	MAX_INDEX_ID	max_index_id

 CF_FLAGS	0	default [0]

 CF_FLAGS	1	__system__ [0]

 select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 count(*)

-3

+6

 set global rocksdb_force_flush_memtable_now = true;

 set global rocksdb_compact_cf='default';

Sergei Petrunia added a comment - 2018-01-12 16:46 Pushed a patch that fixes the above (and also a crash) but now it causes failures like this one: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-stretch-amd64/builds/2297/steps/mtr/logs/stdio rocksdb.information_schema w1 [ fail ] Test ended at 2018-01-12 10:06:03 CURRENT_TEST: rocksdb.information_schema --- /usr/share/mysql/mysql-test/plugin/rocksdb/rocksdb/r/information_schema.result 2018-01-12 08:11:10.000000000 -0500 +++ /dev/shm/var/1/log/information_schema.reject 2018-01-12 10:06:03.800691646 -0500 @@ -8,21 +8,23 @@ MAX_INDEX_ID MAX_INDEX_ID max_index_id CF_FLAGS 0 default [0] CF_FLAGS 1 __system__ [0] -DDL_DROP_INDEX_ONGOING cf_id:0,index_id:max_index_id select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO; count(*) -4 +3 select VALUE into @keysIn from INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS where CF_NAME = 'default' and LEVEL = 'Sum' and TYPE = 'KeyIn'; CREATE TABLE t1 (i1 INT, i2 INT, PRIMARY KEY (i1)) ENGINE = ROCKSDB; INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3); select * from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO; TYPE NAME VALUE +BINLOG FILE ./master-bin.000002 +BINLOG POS 1528 +BINLOG GTID MAX_INDEX_ID MAX_INDEX_ID max_index_id CF_FLAGS 0 default [0] CF_FLAGS 1 __system__ [0] select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO; count(*) -3 +6 set global rocksdb_force_flush_memtable_now = true; set global rocksdb_compact_cf='default';

Sergei Petrunia added a comment - 2018-01-12 16:51

On 10.2, there is also this failure (the cause is most likely that DDL_DROP_INDEX_ONGOING is finished before the query finishes):
http://buildbot.askmonty.org/buildbot/builders/winx64-packages/builds/6487/steps/test/logs/stdio

rocksdb.information_schema               w4 [ fail ]

        Test ended at 2018-01-12 14:38:10

CURRENT_TEST: rocksdb.information_schema

--- D:/winx64-packages/build/src/storage/rocksdb/mysql-test/rocksdb/r/information_schema.result	2018-01-12 13:52:18.000000000 +0000

+++ D:\winx64-packages\build\src\storage\rocksdb\mysql-test\rocksdb\r\information_schema.reject	2018-01-12 14:38:10.199026800 +0000

@@ -11,7 +11,7 @@

 DDL_DROP_INDEX_ONGOING	cf_id:0,index_id:max_index_id

 select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO;

 count(*)

-4

+3

 select VALUE into @keysIn from INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS where CF_NAME = 'default' and LEVEL = 'Sum' and TYPE = 'KeyIn';

 CREATE TABLE t1 (i1 INT, i2 INT, PRIMARY KEY (i1)) ENGINE = ROCKSDB;

 INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3);

mysqltest: Result content mismatch

Re-try run causes a different kind of failure:

rocksdb.information_schema               w4 [ retry-fail ]

        Test ended at 2018-01-12 14:38:15

CURRENT_TEST: rocksdb.information_schema

--- D:/winx64-packages/build/src/storage/rocksdb/mysql-test/rocksdb/r/information_schema.result	2018-01-12 13:52:18.000000000 +0000

+++ D:\winx64-packages\build\src\storage\rocksdb\mysql-test\rocksdb\r\information_schema.reject	2018-01-12 14:38:15.132863500 +0000

@@ -8,10 +8,21 @@

 MAX_INDEX_ID	MAX_INDEX_ID	max_index_id

 CF_FLAGS	0	default [0]

 CF_FLAGS	1	__system__ [0]

+CF_FLAGS	2	cf_a [0]

+CF_FLAGS	3	cf_b [0]

+CF_FLAGS	4	cf_c [0]

+CF_FLAGS	5	rev:cf_d [1]

 DDL_DROP_INDEX_ONGOING	cf_id:0,index_id:max_index_id

+DDL_DROP_INDEX_ONGOING	cf_id:0,index_id:257

+DDL_DROP_INDEX_ONGOING	cf_id:4,index_id:262

+DDL_DROP_INDEX_ONGOING	cf_id:2,index_id:260

+DDL_DROP_INDEX_ONGOING	cf_id:0,index_id:258

+DDL_DROP_INDEX_ONGOING	cf_id:0,index_id:259

+DDL_DROP_INDEX_ONGOING	cf_id:5,index_id:263

+DDL_DROP_INDEX_ONGOING	cf_id:3,index_id:261

It seems the test is re-ran on the same datadir, and that causes it to fail as it is not suitable for running on a non-clean data directory (why does it run on a clean datadir to begin with? The outcome of discussion on Slack is that it is by chance and not by intent?)

Sergei Petrunia added a comment - 2018-01-12 16:51 On 10.2, there is also this failure (the cause is most likely that DDL_DROP_INDEX_ONGOING is finished before the query finishes): http://buildbot.askmonty.org/buildbot/builders/winx64-packages/builds/6487/steps/test/logs/stdio rocksdb.information_schema w4 [ fail ] Test ended at 2018-01-12 14:38:10 CURRENT_TEST: rocksdb.information_schema --- D:/winx64-packages/build/src/storage/rocksdb/mysql-test/rocksdb/r/information_schema.result 2018-01-12 13:52:18.000000000 +0000 +++ D:\winx64-packages\build\src\storage\rocksdb\mysql-test\rocksdb\r\information_schema.reject 2018-01-12 14:38:10.199026800 +0000 @@ -11,7 +11,7 @@ DDL_DROP_INDEX_ONGOING cf_id:0,index_id:max_index_id select count(*) from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO; count(*) -4 +3 select VALUE into @keysIn from INFORMATION_SCHEMA.ROCKSDB_COMPACTION_STATS where CF_NAME = 'default' and LEVEL = 'Sum' and TYPE = 'KeyIn'; CREATE TABLE t1 (i1 INT, i2 INT, PRIMARY KEY (i1)) ENGINE = ROCKSDB; INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3); mysqltest: Result content mismatch Re-try run causes a different kind of failure: rocksdb.information_schema w4 [ retry-fail ] Test ended at 2018-01-12 14:38:15 CURRENT_TEST: rocksdb.information_schema --- D:/winx64-packages/build/src/storage/rocksdb/mysql-test/rocksdb/r/information_schema.result 2018-01-12 13:52:18.000000000 +0000 +++ D:\winx64-packages\build\src\storage\rocksdb\mysql-test\rocksdb\r\information_schema.reject 2018-01-12 14:38:15.132863500 +0000 @@ -8,10 +8,21 @@ MAX_INDEX_ID MAX_INDEX_ID max_index_id CF_FLAGS 0 default [0] CF_FLAGS 1 __system__ [0] +CF_FLAGS 2 cf_a [0] +CF_FLAGS 3 cf_b [0] +CF_FLAGS 4 cf_c [0] +CF_FLAGS 5 rev:cf_d [1] DDL_DROP_INDEX_ONGOING cf_id:0,index_id:max_index_id +DDL_DROP_INDEX_ONGOING cf_id:0,index_id:257 +DDL_DROP_INDEX_ONGOING cf_id:4,index_id:262 +DDL_DROP_INDEX_ONGOING cf_id:2,index_id:260 +DDL_DROP_INDEX_ONGOING cf_id:0,index_id:258 +DDL_DROP_INDEX_ONGOING cf_id:0,index_id:259 +DDL_DROP_INDEX_ONGOING cf_id:5,index_id:263 +DDL_DROP_INDEX_ONGOING cf_id:3,index_id:261 It seems the test is re-ran on the same datadir, and that causes it to fail as it is not suitable for running on a non-clean data directory (why does it run on a clean datadir to begin with? The outcome of discussion on Slack is that it is by chance and not by intent?)

Sergei Golubchik made changes - 2021-12-06 21:21

Workflow

MariaDB v3 [ 83765 ]

MariaDB v4 [ 130745 ]

Ralf Gebhardt added a comment - 2023-10-12 10:58

I have removed the fixVersion for this task as it is one of 5.5, 10.1, 10.2, 10.3, which are EOLed.

Please add a new fixVersion if you plan to work on this task for a not EOLed versions.

Ralf Gebhardt added a comment - 2023-10-12 10:58 I have removed the fixVersion for this task as it is one of 5.5, 10.1, 10.2, 10.3, which are EOLed. Please add a new fixVersion if you plan to work on this task for a not EOLed versions.

Ralf Gebhardt made changes - 2023-10-12 10:58

Fix Version/s

10.2 [ 14601 ]

People

Assignee:: Sergei Petrunia

Reporter:: Sergei Petrunia

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 2017-11-13 03:54

Updated:: 2023-10-12 10:58

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server