Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26677

SIGABRT in myrocks::rdb_i_s_global_info_fill_table / [ERROR] RocksDB: Failed to get column family flags from CF with id = 2. MyRocks data dictionary may be corrupted.

    XMLWordPrintable

    Details

      Description

      The crash is highly sporadic. The "MyRocks data dictionary may be corrupted." message is very lightly sporadic and shows almost always. The testcase for both the message and the crash is identical. To make the crash a bit less sporadic, I added a SELECT SLEEP(10); to the end of the testcase. CLI Based replay crash reproducibility.

      INSTALL SONAME 'ha_rocksdb';
      SET GLOBAL rocksdb_update_cf_options='DEFAULT={write_buffer_size=8m};';
      SELECT COUNT(*) FROM information_schema.rocksdb_global_info;
      SELECT SLEEP(10);
      

      This leads to either the error message below, plus a crash, or just the error message without (or with a delayed/upcoming) crash:

      10.7.0 d552e092c9f3e20da078d1b62b976f629f73d3a4 (Optimized)

      Version: '10.7.0-MariaDB'  socket: '/test/MD180921-mariadb-10.7.0-linux-x86_64-opt/socket.sock'  port: 38673  MariaDB Server
      2021-09-25 14:09:25 4 [Note] RocksDB: Got ENOENT when listing column families
      2021-09-25 14:09:25 4 [Note] RocksDB:   assuming that we're creating a new database
      2021-09-25 14:09:25 4 [Note] RocksDB: Column Families at start:
      2021-09-25 14:09:25 4 [Note]   cf=default
      2021-09-25 14:09:25 4 [Note]     write_buffer_size=67108864
      2021-09-25 14:09:25 4 [Note]     target_file_size_base=67108864
      2021-09-25 14:09:25 4 [Note] RocksDB: creating a column family __system__
      2021-09-25 14:09:25 4 [Note]     write_buffer_size=67108864
      2021-09-25 14:09:25 4 [Note]     target_file_size_base=67108864
      2021-09-25 14:09:25 4 [Note] RocksDB: Table_store: loaded DDL data for 0 tables
      2021-09-25 14:09:25 4 [Note] RocksDB: global statistics using get_sched_indexer_t indexer
      2021-09-25 14:09:25 4 [Note] MyRocks storage engine plugin has been successfully initialized.
      2021-09-25 14:09:25 4 [Note] RocksDB: creating a column family DEFAULT
      2021-09-25 14:09:25 4 [Note]     write_buffer_size=67108864
      2021-09-25 14:09:25 4 [Note]     target_file_size_base=67108864
      2021-09-25 14:09:25 4 [Note] MyRocks: options for column family 'DEFAULT' have been successfully updated.
      2021-09-25 14:09:25 4 [ERROR] RocksDB: Failed to get column family flags from CF with id = 2. MyRocks data dictionary may be corrupted.
      

      10.7.0 d552e092c9f3e20da078d1b62b976f629f73d3a4 (Optimized)

      Core was generated by `/test/MD180921-mariadb-10.7.0-linux-x86_64-opt/bin/mysqld --no-defaults --core-'.
      Program terminated with signal SIGABRT, Aborted.
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      [Current thread is 1 (Thread 0x1538a8cc2700 (LWP 3844856))]
      (gdb) bt
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      #1  0x00001538b410e859 in __GI_abort () at abort.c:79
      #2  0x000015387e27e181 in myrocks::rdb_i_s_global_info_fill_table (thd=0x153830000c58, tables=0x153830010f08, cond=<optimized out>) at /test/10.7_opt/storage/rocksdb/rdb_i_s.cc:803
      #3  0x0000564b5c0dec3e in get_schema_tables_result (join=join@entry=0x1538300120e8, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /test/10.7_opt/sql/sql_show.cc:8830
      #4  0x0000564b5c0c22df in JOIN::exec_inner (this=0x1538300120e8) at /test/10.7_opt/sql/sql_select.cc:4691
      #5  0x0000564b5c0c2b38 in JOIN::exec (this=this@entry=0x1538300120e8) at /test/10.7_opt/sql/sql_select.cc:4513
      #6  0x0000564b5c0c0bda in mysql_select (thd=0x153830000c58, tables=0x153830010f08, fields=<optimized out>, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=<optimized out>, result=0x1538300120c0, unit=0x153830004df0, select_lex=0x153830010868) at /test/10.7_opt/sql/sql_select.cc:4991
      #7  0x0000564b5c0c13a7 in handle_select (thd=thd@entry=0x153830000c58, lex=lex@entry=0x153830004d28, result=result@entry=0x1538300120c0, setup_tables_done_option=setup_tables_done_option@entry=0) at /test/10.7_opt/sql/sql_select.cc:545
      #8  0x0000564b5c044811 in execute_sqlcom_select (thd=0x153830000c58, all_tables=0x153830010f08) at /test/10.7_opt/sql/sql_parse.cc:6253
      #9  0x0000564b5c0525a5 in mysql_execute_command (thd=0x153830000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.7_opt/sql/sql_parse.cc:3944
      #10 0x0000564b5c03f826 in mysql_parse (thd=0x153830000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.7_opt/sql/sql_parse.cc:8028
      #11 0x0000564b5c04b705 in dispatch_command (command=COM_QUERY, thd=0x153830000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.7_opt/sql/sql_class.h:1358
      #12 0x0000564b5c04d5d7 in do_command (thd=0x153830000c58, blocking=blocking@entry=true) at /test/10.7_opt/sql/sql_parse.cc:1402
      #13 0x0000564b5c168ab7 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.7_opt/sql/sql_connect.cc:1418
      #14 0x0000564b5c168dfd in handle_one_connection (arg=arg@entry=0x564b5f55a6e8) at /test/10.7_opt/sql/sql_connect.cc:1312
      #15 0x0000564b5c4bba18 in pfs_spawn_thread (arg=0x564b5f4eaa38) at /test/10.7_opt/storage/perfschema/pfs.cc:2201
      #16 0x00001538b461d609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #17 0x00001538b420b293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
      

      Bug confirmed present in:
      MariaDB: 10.2.41 (dbg), 10.2.41 (opt), 10.3.32 (dbg), 10.3.32 (opt), 10.4.22 (dbg), 10.4.22 (opt), 10.5.13 (dbg), 10.5.13 (opt), 10.6.5 (dbg), 10.6.5 (opt), 10.7.0 (dbg), 10.7.0 (opt)

        Attachments

          Activity

            People

            Assignee:
            psergei Sergei Petrunia
            Reporter:
            Roel Roel Van de Paar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:

                Git Integration