Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23952

RocksDB: Failed to get column family info from index id, SIGABRT in in myrocks::Rdb_dict_manager::log_start_drop_index

    XMLWordPrintable

    Details

      Description

      10.3 469a249a

      2020-10-14  2:27:55 35 [ERROR] RocksDB: Failed to get column family info from index id (0,290). MyRocks data dictionary may get corrupted.
      201014  2:27:55 [ERROR] mysqld got signal 6 ;
       
      #5  0x00007fb01356f859 in __GI_abort () at abort.c:79
      #6  0x00007fb00e078559 in myrocks::Rdb_dict_manager::log_start_drop_index (this=0x7fb00e87e200 <myrocks::dict_manager>, gl_index_id=..., log_action=0x7fb00e53213a "Begin") at /data/src/10.3/storage/rocksdb/rdb_datadic.cc:5263
      #7  0x00007fb00e077e6d in myrocks::Rdb_dict_manager::add_drop_index (this=0x7fb00e87e200 <myrocks::dict_manager>, gl_index_ids=Python Exception <class 'gdb.error'> No type named std::__detail::_Hash_node<struct myrocks::_gl_index_id_s, true>.: 
      std::unordered_set with 1 element, batch=0x7fafc8080e70) at /data/src/10.3/storage/rocksdb/rdb_datadic.cc:5130
      #8  0x00007fb00dfe57ab in myrocks::ha_rocksdb::commit_inplace_alter_table (this=0x7fafbc8bea98, altered_table=0x7fafc807a3a0, ha_alter_info=0x7fb00c15eae0, commit=true) at /data/src/10.3/storage/rocksdb/ha_rocksdb.cc:12988
      #9  0x0000555c971d01d8 in handler::ha_commit_inplace_alter_table (this=0x7fafbc8bea98, altered_table=0x7fafc807a3a0, ha_alter_info=0x7fb00c15eae0, commit=true) at /data/src/10.3/sql/handler.cc:4596
      #10 0x0000555c96f8308c in mysql_inplace_alter_table (thd=0x7fafc8000d90, table_list=0x7fafc80117f8, table=0x7fafbc022b20, altered_table=0x7fafc807a3a0, ha_alter_info=0x7fb00c15eae0, inplace_supported=HA_ALTER_INPLACE_COPY_LOCK, target_mdl_request=0x7fb00c15ec60, alter_ctx=0x7fb00c15f210) at /data/src/10.3/sql/sql_table.cc:7701
      #11 0x0000555c96f89b1f in mysql_alter_table (thd=0x7fafc8000d90, new_db=0x7fafc8011810, new_name=0x7fafc8011820, create_info=0x7fb00c15ffc0, table_list=0x7fafc80117f8, alter_info=0x7fb00c15ff00, order_num=0, order=0x0, ignore=false) at /data/src/10.3/sql/sql_table.cc:9946
      #12 0x0000555c96e9e3e0 in mysql_execute_command (thd=0x7fafc8000d90) at /data/src/10.3/sql/sql_parse.cc:4089
      #13 0x0000555c96eab2ce in mysql_parse (thd=0x7fafc8000d90, rawbuf=0x7fafc80116f8 "DROP INDEX idx ON t3", length=20, parser_state=0x7fb00c1605c0, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:7810
      #14 0x0000555c96e97a8f in dispatch_command (command=COM_QUERY, thd=0x7fafc8000d90, packet=0x7fafc8008f11 "DROP INDEX idx ON t3", packet_length=20, is_com_multi=false, is_next_command=false) at /data/src/10.3/sql/sql_parse.cc:1847
      #15 0x0000555c96e9642f in do_command (thd=0x7fafc8000d90) at /data/src/10.3/sql/sql_parse.cc:1393
      #16 0x0000555c97013c44 in do_handle_one_connection (connect=0x555c9a808620) at /data/src/10.3/sql/sql_connect.cc:1403
      #17 0x0000555c970139a0 in handle_one_connection (arg=0x555c9a808620) at /data/src/10.3/sql/sql_connect.cc:1308
      #18 0x0000555c979d9925 in pfs_spawn_thread (arg=0x555c9a946860) at /data/src/10.3/storage/perfschema/pfs.cc:1869
      #19 0x00007fb013a92609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #20 0x00007fb01366c293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      

      Reproducible on 10.2-10.5.
      Release, and debug and ASAN builds fail the same way.
      rr profile is available.

      Here is a concurrent MTR test case (probably not minimal) which I used to reproduce the failure. It is non-deterministic and needs to be run with --repeat=N. It fails quite frequently for me, usually within 5-10 attempts, but it can vary a lot on different machines and builds.

      # Run with --mem  --mysqld=--plugin-load-add=ha_rocksdb --repeat=15
       
      CREATE TABLE t1 (a varchar(10), b varchar(1024), pk integer auto_increment, c date, primary key (pk), key (c)) ENGINE=RocksDB;
       
      CREATE TABLE t2 (a INT) ENGINE=RocksDB;
      INSERT INTO t2 VALUES (1);
       
      CREATE TABLE t3 (pk integer auto_increment, d int, e varchar(10), primary key (pk), key (e)) ENGINE=rocksdb;
       
      --connect (con1,localhost,root,,test)
      --connect (con2,localhost,root,,test)
      --connect (con3,localhost,root,,test)
       
      SET AUTOCOMMIT=OFF;
      CREATE PROCEDURE sp() CREATE TABLE t4 (tcol2 DATE NOT NULL);
       
      --connection con1
      SET max_statement_time= 1;
      --send
        ALTER TABLE t1 ADD INDEX test_idx USING HASH (b(37));
       
      --connection con3
      CALL sp;
       
      --connection con2
      --send
        ALTER TABLE t3 ADD INDEX idx USING HASH (pk,d);
       
      --connection con3
      --error ER_BAD_FIELD_ERROR
      SELECT x FROM t4 JOIN t2 JOIN t1 JOIN t3;
       
      --connection con1
      --error 0,ER_STATEMENT_TIMEOUT
      --reap
       
      --connection con3
      --error 0,ER_CANT_DROP_FIELD_OR_KEY
      DROP INDEX idx ON t3;
       
      # Cleanup
      --connection con2
      --reap
      --disconnect con2
      --disconnect con1
      --disconnect con3
      --connection default
       
      DROP PROCEDURE sp;
      DROP TABLE t1, t2, t3, t4;
      

        Attachments

          Activity

            People

            Assignee:
            psergei Sergei Petrunia
            Reporter:
            elenst Elena Stepanova
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: