[MDEV-23952] RocksDB: Failed to get column family info from index id, SIGABRT in in myrocks::Rdb_dict_manager::log_start_drop_index Created: 2020-10-13  Updated: 2023-04-27

Status: Open
Project: MariaDB Server
Component/s: Storage Engine - RocksDB
Affects Version/s: 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.4, 10.5

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Sergei Petrunia
Resolution: Unresolved Votes: 0
Labels: rr-profile


 Description   

10.3 469a249a

2020-10-14  2:27:55 35 [ERROR] RocksDB: Failed to get column family info from index id (0,290). MyRocks data dictionary may get corrupted.
201014  2:27:55 [ERROR] mysqld got signal 6 ;
 
#5  0x00007fb01356f859 in __GI_abort () at abort.c:79
#6  0x00007fb00e078559 in myrocks::Rdb_dict_manager::log_start_drop_index (this=0x7fb00e87e200 <myrocks::dict_manager>, gl_index_id=..., log_action=0x7fb00e53213a "Begin") at /data/src/10.3/storage/rocksdb/rdb_datadic.cc:5263
#7  0x00007fb00e077e6d in myrocks::Rdb_dict_manager::add_drop_index (this=0x7fb00e87e200 <myrocks::dict_manager>, gl_index_ids=Python Exception <class 'gdb.error'> No type named std::__detail::_Hash_node<struct myrocks::_gl_index_id_s, true>.: 
std::unordered_set with 1 element, batch=0x7fafc8080e70) at /data/src/10.3/storage/rocksdb/rdb_datadic.cc:5130
#8  0x00007fb00dfe57ab in myrocks::ha_rocksdb::commit_inplace_alter_table (this=0x7fafbc8bea98, altered_table=0x7fafc807a3a0, ha_alter_info=0x7fb00c15eae0, commit=true) at /data/src/10.3/storage/rocksdb/ha_rocksdb.cc:12988
#9  0x0000555c971d01d8 in handler::ha_commit_inplace_alter_table (this=0x7fafbc8bea98, altered_table=0x7fafc807a3a0, ha_alter_info=0x7fb00c15eae0, commit=true) at /data/src/10.3/sql/handler.cc:4596
#10 0x0000555c96f8308c in mysql_inplace_alter_table (thd=0x7fafc8000d90, table_list=0x7fafc80117f8, table=0x7fafbc022b20, altered_table=0x7fafc807a3a0, ha_alter_info=0x7fb00c15eae0, inplace_supported=HA_ALTER_INPLACE_COPY_LOCK, target_mdl_request=0x7fb00c15ec60, alter_ctx=0x7fb00c15f210) at /data/src/10.3/sql/sql_table.cc:7701
#11 0x0000555c96f89b1f in mysql_alter_table (thd=0x7fafc8000d90, new_db=0x7fafc8011810, new_name=0x7fafc8011820, create_info=0x7fb00c15ffc0, table_list=0x7fafc80117f8, alter_info=0x7fb00c15ff00, order_num=0, order=0x0, ignore=false) at /data/src/10.3/sql/sql_table.cc:9946
#12 0x0000555c96e9e3e0 in mysql_execute_command (thd=0x7fafc8000d90) at /data/src/10.3/sql/sql_parse.cc:4089
#13 0x0000555c96eab2ce in mysql_parse (thd=0x7fafc8000d90, rawbuf=0x7fafc80116f8 "DROP INDEX idx ON t3", length=20, parser_state=0x7fb00c1605c0, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:7810
#14 0x0000555c96e97a8f in dispatch_command (command=COM_QUERY, thd=0x7fafc8000d90, packet=0x7fafc8008f11 "DROP INDEX idx ON t3", packet_length=20, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:1847
#15 0x0000555c96e9642f in do_command (thd=0x7fafc8000d90) at /data/src/10.3/sql/sql_parse.cc:1393
#16 0x0000555c97013c44 in do_handle_one_connection (connect=0x555c9a808620) at /data/src/10.3/sql/sql_connect.cc:1403
#17 0x0000555c970139a0 in handle_one_connection (arg=0x555c9a808620) at /data/src/10.3/sql/sql_connect.cc:1308
#18 0x0000555c979d9925 in pfs_spawn_thread (arg=0x555c9a946860) at /data/src/10.3/storage/perfschema/pfs.cc:1869
#19 0x00007fb013a92609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#20 0x00007fb01366c293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Reproducible on 10.2-10.5.
Release, and debug and ASAN builds fail the same way.
rr profile is available.

Here is a concurrent MTR test case (probably not minimal) which I used to reproduce the failure. It is non-deterministic and needs to be run with --repeat=N. It fails quite frequently for me, usually within 5-10 attempts, but it can vary a lot on different machines and builds.

# Run with --mem  --mysqld=--plugin-load-add=ha_rocksdb --repeat=15
 
CREATE TABLE t1 (a varchar(10), b varchar(1024), pk integer auto_increment, c date, primary key (pk), key (c)) ENGINE=RocksDB;
 
CREATE TABLE t2 (a INT) ENGINE=RocksDB;
INSERT INTO t2 VALUES (1);
 
CREATE TABLE t3 (pk integer auto_increment, d int, e varchar(10), primary key (pk), key (e)) ENGINE=rocksdb;
 
--connect (con1,localhost,root,,test)
--connect (con2,localhost,root,,test)
--connect (con3,localhost,root,,test)
 
SET AUTOCOMMIT=OFF;
CREATE PROCEDURE sp() CREATE TABLE t4 (tcol2 DATE NOT NULL);
 
--connection con1
SET max_statement_time= 1;
--send
  ALTER TABLE t1 ADD INDEX test_idx USING HASH (b(37));
 
--connection con3
CALL sp;
 
--connection con2
--send
  ALTER TABLE t3 ADD INDEX idx USING HASH (pk,d);
 
--connection con3
--error ER_BAD_FIELD_ERROR
SELECT x FROM t4 JOIN t2 JOIN t1 JOIN t3;
 
--connection con1
--error 0,ER_STATEMENT_TIMEOUT
--reap
 
--connection con3
--error 0,ER_CANT_DROP_FIELD_OR_KEY
DROP INDEX idx ON t3;
 
# Cleanup
--connection con2
--reap
--disconnect con2
--disconnect con1
--disconnect con3
--connection default
 
DROP PROCEDURE sp;
DROP TABLE t1, t2, t3, t4;


Generated at Thu Feb 08 09:26:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.