Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-18204

RocksDB failed to start due to problems validating data dictionary against .frm files after restoring incremental backup

Details

    Description

      10.4 30da40bb8c3

      		 
      2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
      		 
      2019-01-10 19:10:10 0 [Note] RocksDB:   assuming that we're creating a new database
      		 
      2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
      		 
      2019-01-10 19:10:10 0 [Note]   cf=default
      		 
      2019-01-10 19:10:10 0 [Note]     write_buffer_size=67108864
      		 
      2019-01-10 19:10:10 0 [Note]     target_file_size_base=67108864
      		 
      2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
      		 
      2019-01-10 19:10:11 0 [Note]     write_buffer_size=67108864
      		 
      2019-01-10 19:10:11 0 [Note]     target_file_size_base=67108864
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
      		 
      2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
      		 
      2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
      		 
      2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
      		 
      2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.

      Backup dirs are attached.
      backup_before_prepare_0 is the initial full backup
      backup_before_prepare_1 is the 1st incremental backup
      backup_before_prepare_2 is the 1st incremental backup
      backup_0 is the prepared backup after all prepare were executed on it.

      To reproduce the failure, restore backup_0.tar.gz and start the server with

      --plugin-load-add=ha_rocksdb --binlog-format=row

      other options can be default.

      If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

      mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).


      Notes on the origin of the report

      Please be aware it's not necessarily reproducible the same way.

      elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d

      		 
      perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental

      The test log is attached as trial.log

      On 10.2: https://travis-ci.org/elenst/travis-tests/jobs/481810951

      Attachments

        1. backup_0.tar.gz
          229 kB
        2. backup_before_prepare_0.tar.gz
          346 kB
        3. backup_before_prepare_1.tar.gz
          770 kB
        4. backup_before_prepare_2.tar.gz
          1.94 MB
        5. mbackup_backup_0.log
          19 kB
        6. mbackup_backup_1.log
          19 kB
        7. mbackup_backup_2.log
          19 kB
        8. mbackup_prepare_0.log
          2 kB
        9. mbackup_prepare_1.log
          18 kB
        10. mbackup_prepare_2.log
          18 kB
        11. mbackup_restore.log
          16 kB
        12. mysql.err
          5 kB
        13. trial.log
          32 kB

        Activity

          Ascending order - Click to sort in descending order
          elenst Elena Stepanova created issue -
          elenst Elena Stepanova made changes -
          Field Original Value New Value
          Assignee Vladislav Vaintroub [ wlad ]
          elenst Elena Stepanova made changes -
          Fix Version/s 10.3 [ 22126 ]
          Affects Version/s 10.3 [ 22126 ]
          Description {noformat:title=10.4 30da40bb8c3}
          2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
          2019-01-10 19:10:10 0 [Note] RocksDB: assuming that we're creating a new database
          2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
          2019-01-10 19:10:10 0 [Note] cf=default
          2019-01-10 19:10:10 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:10 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
          2019-01-10 19:10:11 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:11 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
          {noformat}

          Backup dirs are attached.
          backup_before_prepare_0 is the initial full backup
          backup_before_prepare_1 is the 1st incremental backup
          backup_before_prepare_2 is the 1st incremental backup
          backup_0 is the prepared backup after all prepare were executed on it.

          To reproduce the failure, restore [^backup_0.tar.gz] and start the server with
          {noformat}
          --plugin-load-add=ha_rocksdb --binlog-format=row
          {noformat}
          other options can be default.

          If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

          mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).

          *I have no information yet whether it affects previous versions.*

          ----
          {color:gray}
          {panel:title=Notes on the origin of the report|bgColor=#EEEEEE}
          Please be aware it's not necessarily reproducible the same way.

          {noformat:title=elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d}
          perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental
          {noformat}

          The test log is attached as [^trial.log]
          {panel}
          {color}           		{noformat:title=10.4 30da40bb8c3}
          2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
          2019-01-10 19:10:10 0 [Note] RocksDB: assuming that we're creating a new database
          2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
          2019-01-10 19:10:10 0 [Note] cf=default
          2019-01-10 19:10:10 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:10 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
          2019-01-10 19:10:11 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:11 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
          {noformat}

          Backup dirs are attached.
          backup_before_prepare_0 is the initial full backup
          backup_before_prepare_1 is the 1st incremental backup
          backup_before_prepare_2 is the 1st incremental backup
          backup_0 is the prepared backup after all prepare were executed on it.

          To reproduce the failure, restore [^backup_0.tar.gz] and start the server with
          {noformat}
          --plugin-load-add=ha_rocksdb --binlog-format=row
          {noformat}
          other options can be default.

          If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

          mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).

          *I have no information yet whether it affects 10.2.*

          ----
          {color:gray}
          {panel:title=Notes on the origin of the report|bgColor=#EEEEEE}
          Please be aware it's not necessarily reproducible the same way.

          {noformat:title=elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d}
          perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental
          {noformat}

          The test log is attached as [^trial.log]
          {panel}
          {color}
          elenst Elena Stepanova made changes -
          Fix Version/s 10.2 [ 14601 ]
          Affects Version/s 10.2 [ 14601 ]
          Description {noformat:title=10.4 30da40bb8c3}
          2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
          2019-01-10 19:10:10 0 [Note] RocksDB: assuming that we're creating a new database
          2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
          2019-01-10 19:10:10 0 [Note] cf=default
          2019-01-10 19:10:10 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:10 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
          2019-01-10 19:10:11 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:11 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
          {noformat}

          Backup dirs are attached.
          backup_before_prepare_0 is the initial full backup
          backup_before_prepare_1 is the 1st incremental backup
          backup_before_prepare_2 is the 1st incremental backup
          backup_0 is the prepared backup after all prepare were executed on it.

          To reproduce the failure, restore [^backup_0.tar.gz] and start the server with
          {noformat}
          --plugin-load-add=ha_rocksdb --binlog-format=row
          {noformat}
          other options can be default.

          If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

          mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).

          *I have no information yet whether it affects 10.2.*

          ----
          {color:gray}
          {panel:title=Notes on the origin of the report|bgColor=#EEEEEE}
          Please be aware it's not necessarily reproducible the same way.

          {noformat:title=elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d}
          perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental
          {noformat}

          The test log is attached as [^trial.log]
          {panel}
          {color}           		{noformat:title=10.4 30da40bb8c3}
          2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
          2019-01-10 19:10:10 0 [Note] RocksDB: assuming that we're creating a new database
          2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
          2019-01-10 19:10:10 0 [Note] cf=default
          2019-01-10 19:10:10 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:10 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
          2019-01-10 19:10:11 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:11 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
          {noformat}

          Backup dirs are attached.
          backup_before_prepare_0 is the initial full backup
          backup_before_prepare_1 is the 1st incremental backup
          backup_before_prepare_2 is the 1st incremental backup
          backup_0 is the prepared backup after all prepare were executed on it.

          To reproduce the failure, restore [^backup_0.tar.gz] and start the server with
          {noformat}
          --plugin-load-add=ha_rocksdb --binlog-format=row
          {noformat}
          other options can be default.

          If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

          mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).

          ----
          {color:gray}
          {panel:title=Notes on the origin of the report|bgColor=#EEEEEE}
          Please be aware it's not necessarily reproducible the same way.

          {noformat:title=elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d}
          perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental
          {noformat}

          The test log is attached as [^trial.log]
          {panel}
          {color}
          elenst Elena Stepanova made changes -
          Description {noformat:title=10.4 30da40bb8c3}
          2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
          2019-01-10 19:10:10 0 [Note] RocksDB: assuming that we're creating a new database
          2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
          2019-01-10 19:10:10 0 [Note] cf=default
          2019-01-10 19:10:10 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:10 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
          2019-01-10 19:10:11 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:11 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
          {noformat}

          Backup dirs are attached.
          backup_before_prepare_0 is the initial full backup
          backup_before_prepare_1 is the 1st incremental backup
          backup_before_prepare_2 is the 1st incremental backup
          backup_0 is the prepared backup after all prepare were executed on it.

          To reproduce the failure, restore [^backup_0.tar.gz] and start the server with
          {noformat}
          --plugin-load-add=ha_rocksdb --binlog-format=row
          {noformat}
          other options can be default.

          If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

          mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).

          ----
          {color:gray}
          {panel:title=Notes on the origin of the report|bgColor=#EEEEEE}
          Please be aware it's not necessarily reproducible the same way.

          {noformat:title=elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d}
          perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental
          {noformat}

          The test log is attached as [^trial.log]
          {panel}
          {color}           		{noformat:title=10.4 30da40bb8c3}
          2019-01-10 19:10:10 0 [Note] RocksDB: Got ENOENT when listing column families
          2019-01-10 19:10:10 0 [Note] RocksDB: assuming that we're creating a new database
          2019-01-10 19:10:10 0 [Note] RocksDB: Column Families at start:
          2019-01-10 19:10:10 0 [Note] cf=default
          2019-01-10 19:10:10 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:10 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Note] RocksDB: creating a column family __system__
          2019-01-10 19:10:11 0 [Note] write_buffer_size=67108864
          2019-01-10 19:10:11 0 [Note] target_file_size_base=67108864
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t8, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t7, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t2, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t6, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t3, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t1, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t9, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t4, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [Warning] RocksDB: Schema mismatch - A .frm file exists for table test.t5, but that table is not registered in RocksDB
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
          2019-01-10 19:10:11 0 [ERROR] RocksDB: Failed to initialize DDL manager.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' init function returned error.
          2019-01-10 19:10:11 0 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
          {noformat}

          Backup dirs are attached.
          backup_before_prepare_0 is the initial full backup
          backup_before_prepare_1 is the 1st incremental backup
          backup_before_prepare_2 is the 1st incremental backup
          backup_0 is the prepared backup after all prepare were executed on it.

          To reproduce the failure, restore [^backup_0.tar.gz] and start the server with
          {noformat}
          --plugin-load-add=ha_rocksdb --binlog-format=row
          {noformat}
          other options can be default.

          If you have doubts about the correctness of prepare, you can run it on non-prepared backups.

          mariabackup logs from all stages are attached with hopefully self-explanatory names (mbackup*.log).

          ----
          {color:gray}
          {panel:title=Notes on the origin of the report|bgColor=#EEEEEE}
          Please be aware it's not necessarily reproducible the same way.

          {noformat:title=elenst-dev 0ebbe452a69f mariadb-toolbox 17ab77fe268d}
          perl ./runall-new.pl --basedir=/home/travis/server/10.4 --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1547101908 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--plugin-load-add=ha_rocksdb --mysqld=--binlog-format=row --engine=RocksDB --scenario=MariaBackupIncremental
          {noformat}

          The test log is attached as [^trial.log]

          On 10.2: https://travis-ci.org/elenst/travis-tests/jobs/481810951
          {panel}
          {color}
          wlad Vladislav Vaintroub made changes -
          Fix Version/s 10.4.3 [ 23230 ]
          Fix Version/s 10.3.13 [ 23215 ]
          Fix Version/s 10.2.23 [ 23307 ]
          Fix Version/s 10.2 [ 14601 ]
          Fix Version/s 10.3 [ 22126 ]
          Fix Version/s 10.4 [ 22408 ]
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Closed [ 6 ]
          serg Sergei Golubchik made changes -
          Workflow MariaDB v3 [ 91630 ] MariaDB v4 [ 155490 ]

          People

            wlad Vladislav Vaintroub
            elenst Elena Stepanova
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.