Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-18324

MariaBackup-related failures in DDL scenarios on 10.1

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • 10.1(EOL)
    • 10.2(EOL)
    • mariabackup
    • None

    Description

      This item represents an (incomplete) collection of symptoms observed upon full and incremental MariaBackup performed on 10.1 running a mix of DML and DDL. These 10.1-only issues are considered low priority problems caused by known deficiencies in implementation of DDL backup on 10.1. They are unlikely to be fixed, unless circumstances change, but we need them to be searchable in JIRA in case we get complaints about similar problems.

      Assertion failure upon CHECK TABLE after restoring from full backup

      https://travis-ci.org/elenst/travis-tests/jobs/481813549 (Trial 3)

      Backup, logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-1.tar.gz

      10.1 1d72db45a

      2019-01-20 17:14:43 140492144884480 [ERROR] InnoDB: tablespace id is 511 in the data dictionary but in file ./test/t3.ibd it is 330!
      2019-01-20 17:14:43 7fc6e063bb00  InnoDB: Assertion failure in thread 140492144884480 in file buf0buf.cc line 3112
      InnoDB: Failing assertion: zip_size == fil_space_get_zip_size(space)
       
      #3  <signal handler called>
      #4  0x00007fc6f61d7428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
      #5  0x00007fc6f61d902a in abort () from /lib/x86_64-linux-gnu/libc.so.6
      #6  0x000055834e1a6818 in buf_page_get_gen (space=511, zip_size=0, offset=0, rw_latch=2, guess=0x0, mode=10, file=0x55834e539678 "/home/travis/src/storage/xtradb/fil/fil0fil.cc", line=3923, mtr=0x7fc6e0637250, err=0x0) at /home/travis/src/storage/xtradb/buf/buf0buf.cc:3112
      #7  0x000055834e21d52b in fsp_flags_try_adjust (space_id=511, flags=0) at /home/travis/src/storage/xtradb/fil/fil0fil.cc:3921
      #8  0x000055834e22068f in fil_space_for_table_exists_in_mem (id=511, name=0x7fc6e0638410 "test/t3", print_error_if_does_not_exist=false, remove_from_data_dict_if_does_not_exist=false, adjust_space=true, heap=0x7fc690113930, table_id=525, table_flags=1) at /home/travis/src/storage/xtradb/fil/fil0fil.cc:5492
      #9  0x000055834e1f9dc8 in dict_load_table (name=0x7fc6e0638410 "test/t3", cached=1, ignore_err=DICT_ERR_IGNORE_NONE) at /home/travis/src/storage/xtradb/dict/dict0load.cc:2409
      #10 0x000055834e1dffbe in dict_table_open_on_name (table_name=0x7fc6e0638410 "test/t3", dict_locked=0, try_drop=1, ignore_err=DICT_ERR_IGNORE_NONE) at /home/travis/src/storage/xtradb/dict/dict0dict.cc:1150
      #11 0x000055834dfa5392 in ha_innobase::open (this=0x7fc6901100d8, name=0x7fc69010d6c0 "./test/t3", mode=2, test_if_locked=50) at /home/travis/src/storage/xtradb/handler/ha_innodb.cc:6368
      #12 0x000055834dc65c3e in handler::ha_open (this=0x7fc6901100d8, table_arg=0x7fc69010f490, name=0x7fc69010d6c0 "./test/t3", mode=2, test_if_locked=50) at /home/travis/src/sql/handler.cc:2534
      #13 0x000055834db1353b in open_table_from_share (thd=0x55834ff83a90, share=0x7fc69010d1e8, alias=0x7fc690008800 "t3", db_stat=39, prgflag=44, ha_open_flags=50, outparam=0x7fc69010f490, is_create_table=false) at /home/travis/src/sql/table.cc:2975
      #14 0x000055834d9b6869 in open_table (thd=0x55834ff83a90, table_list=0x7fc690008808, ot_ctx=0x7fc6e06390b0) at /home/travis/src/sql/sql_base.cc:2585
      #15 0x000055834d9b9187 in open_and_process_table (thd=0x55834ff83a90, lex=0x55834ff87500, tables=0x7fc690008808, counter=0x7fc6e0639144, flags=0, prelocking_strategy=0x7fc6e06391c0, has_prelocking_list=false, ot_ctx=0x7fc6e06390b0) at /home/travis/src/sql/sql_base.cc:4124
      #16 0x000055834d9ba280 in open_tables (thd=0x55834ff83a90, options=..., start=0x7fc6e0639128, counter=0x7fc6e0639144, flags=0, prelocking_strategy=0x7fc6e06391c0) at /home/travis/src/sql/sql_base.cc:4639
      #17 0x000055834d9bb938 in open_and_lock_tables (thd=0x55834ff83a90, options=..., tables=0x7fc690008808, derived=true, flags=0, prelocking_strategy=0x7fc6e06391c0) at /home/travis/src/sql/sql_base.cc:5385
      #18 0x000055834d9aeb48 in open_and_lock_tables (thd=0x55834ff83a90, tables=0x7fc690008808, derived=true, flags=0) at /home/travis/src/sql/sql_base.h:544
      #19 0x000055834db66374 in mysql_admin_table(THD *, TABLE_LIST *, HA_CHECK_OPT *, const char *, thr_lock_type, bool, bool, uint, int (*)(THD *, TABLE_LIST *, HA_CHECK_OPT *), struct {...}, int (*)(THD *, TABLE_LIST *, HA_CHECK_OPT *)) (thd=0x55834ff83a90, tables=0x7fc690008808, check_opt=0x55834ff88418, operator_name=0x55834e3aa7d4 "check", lock_type=TL_READ_NO_INSERT, org_open_for_modify=false, repair_table_use_frm=false, extra_open_options=32, prepare_func=0x0, operator_func=(int (handler::*)(handler * const, THD *, HA_CHECK_OPT *)) 0x55834dc69824 <handler::ha_check(THD*, st_ha_check_opt*)>, view_operator_func=0x55834db0903a <view_check(THD*, TABLE_LIST*, st_ha_check_opt*)>) at /home/travis/src/sql/sql_admin.cc:465
      #20 0x000055834db69287 in Sql_cmd_check_table::execute (this=0x7fc690008df8, thd=0x55834ff83a90) at /home/travis/src/sql/sql_admin.cc:1285
      #21 0x000055834da28b1b in mysql_execute_command (thd=0x55834ff83a90) at /home/travis/src/sql/sql_parse.cc:5701
      #22 0x000055834da2d6af in mysql_parse (thd=0x55834ff83a90, rawbuf=0x7fc6900086f8 "CHECK TABLE `test`.`t3` EXTENDED", length=32, parser_state=0x7fc6e063a670) at /home/travis/src/sql/sql_parse.cc:7468
      #23 0x000055834da1bbd3 in dispatch_command (command=COM_QUERY, thd=0x55834ff83a90, packet=0x55834ff8a161 "", packet_length=32) at /home/travis/src/sql/sql_parse.cc:1496
      #24 0x000055834da1a945 in do_command (thd=0x55834ff83a90) at /home/travis/src/sql/sql_parse.cc:1124
      #25 0x000055834db55bec in do_handle_one_connection (thd_arg=0x55834ff83a90) at /home/travis/src/sql/sql_connect.cc:1330
      #26 0x000055834db5593b in handle_one_connection (arg=0x55834ff83a90) at /home/travis/src/sql/sql_connect.cc:1242
      #27 0x00007fc6f6bfe6ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
      #28 0x00007fc6f62a941d in clone () from /lib/x86_64-linux-gnu/libc.so.6
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548004223 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --engine=InnoDB --scenario=MariaBackupFull --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      InnoDB: Could not find a valid tablespace file for 'tmp/#sql after restoring from incremental backup

      This can actually be related to a non-10.1-specific issue MDEV-18309.

      https://travis-ci.org/elenst/travis-tests/jobs/481813549 (Trial 4)

      Backup, logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-2.tar.gz

      10.1 1d72db45a

      2019-01-20 17:19:24 140328529209152 [ERROR] InnoDB: Could not find a valid tablespace file for 'tmp/#sql61a7_8_5a'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
      2019-01-20 17:19:24 140328529209152 [ERROR] InnoDB: Tablespace open failed for '"tmp"."#sql61a7_8_5a"', ignored.
      2019-01-20 17:19:24 7fa0c822a740  InnoDB: Operating system error number 2 in a file operation.
      InnoDB: The error means the system cannot find the path specified.
      InnoDB: If you are installing InnoDB, remember that you must create
      InnoDB: directories yourself, InnoDB does not create them.
      2019-01-20 17:19:24 140328529209152 [ERROR] InnoDB: Could not find a valid tablespace file for 'tmp/#sql61a7_8_96'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
      2019-01-20 17:19:24 140328529209152 [ERROR] InnoDB: Tablespace open failed for '"tmp"."#sql61a7_8_96"', ignored.
      2019-01-20 17:19:24 7fa0c822a740  InnoDB: Operating system error number 2 in a file operation.
      InnoDB: The error means the system cannot find the path specified.
      InnoDB: If you are installing InnoDB, remember that you must create
      InnoDB: directories yourself, InnoDB does not create them.
       
      etc.
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548004545 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --engine=InnoDB --scenario=MariaBackupIncremental --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      SIGSEGV upon full backup

      https://travis-ci.org/elenst/travis-tests/jobs/481813550 (Trial 3)

      Logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-3.tar.gz

      10.1 1d72db45a

      mariabackup: Generating a list of tablespaces
      190120 17:28:05 [01] Copying ibdata1 to /home/travis/logs/vardir/backup/ibdata1
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./mysql/innodb_index_stats.ibd to /home/travis/logs/vardir/backup/mysql/innodb_index_stats.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./mysql/gtid_slave_pos.ibd to /home/travis/logs/vardir/backup/mysql/gtid_slave_pos.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./mysql/innodb_table_stats.ibd to /home/travis/logs/vardir/backup/mysql/innodb_table_stats.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./test/t8.ibd to /home/travis/logs/vardir/backup/test/t8.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./test/t7.ibd to /home/travis/logs/vardir/backup/test/t7.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./test/t6.ibd to /home/travis/logs/vardir/backup/test/t6.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./test/t5.ibd to /home/travis/logs/vardir/backup/test/t5.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./test/t3.ibd to /home/travis/logs/vardir/backup/test/t3.ibd
      190120 17:28:06 [01]        ...done
      190120 17:28:06 [01] Copying ./test/t2.ibd to /home/travis/logs/vardir/backup/test/t2.ibd
      190120 17:28:06 [01]        ...done
      [01] mariabackup: error: cannot stat ./test/#sql-ib665-4215267691.ibd
      190120 17:28:06 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.1.38-MariaDB-debug
      key_buffer_size=0
      read_buffer_size=131072
      max_used_connections=0
      max_threads=1
      thread_count=0
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5416 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      Segmentation fault (core dumped)
      
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548005162 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=zlib --engine=InnoDB --scenario=MariaBackupFull --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      InnoDB: Cannot open table from the internal data dictionary of InnoDB though the .frm file for the table exists upon startup after restoring from full backup

      https://travis-ci.org/elenst/travis-tests/jobs/481813551 (Trial 3)

      Backup, logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-4.tar.gz

      10.1 1d72db45a

      2019-01-20 17:40:03 140378998168320 [Warning] InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
      2019-01-20 17:40:03 140378998168320 [Warning] InnoDB: Cannot open table test/t3 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548005730 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-encrypt-tables --mysqld=--innodb-encrypt-log --mysqld=--innodb-encryption-threads=4 --mysqld=--aria-encrypt-tables=1 --mysqld=--encrypt-tmp-disk-tables=1 --mysqld=--file-key-management --mysqld=--file-key-management-filename=/home/travis/mariadb-toolbox/data/keys.txt --mysqld=--plugin-load-add=file_key_management --mysqld=--innodb-compression-algorithm=none --engine=InnoDB --scenario=MariaBackupFull --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      Error: failed to execute query FLUSH NO_WRITE_TO_BINLOG TABLES: Query execution was interrupted (max_statement_time exceeded) during backup

      It can be considered a configuration problem, but backup should probably be smarter about such configuration parameters.

      https://travis-ci.org/elenst/travis-tests/jobs/481813551 (Trial 4)

      logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-5.tar.gz

      10.1 1d72db45a

      190120 17:43:16 Executing FLUSH NO_WRITE_TO_BINLOG TABLES...
      190120 17:43:17 >> log scanned up to (61316935)
      190120 17:43:18 >> log scanned up to (61782247)
      190120 17:43:19 >> log scanned up to (62082245)
      190120 17:43:20 >> log scanned up to (62473402)
      190120 17:43:21 >> log scanned up to (62867634)
      190120 17:43:22 >> log scanned up to (63328972)
      190120 17:43:24 >> log scanned up to (64001384)
      190120 17:43:25 >> log scanned up to (64808906)
      190120 17:43:26 >> log scanned up to (65455330)
      190120 17:43:27 >> log scanned up to (65932021)
      190120 17:43:28 >> log scanned up to (66943940)
      190120 17:43:29 >> log scanned up to (67150179)
      190120 17:43:30 >> log scanned up to (68277017)
      190120 17:43:31 >> log scanned up to (68493930)
      190120 17:43:32 >> log scanned up to (69216738)
      190120 17:43:33 >> log scanned up to (69365574)
      190120 17:43:34 >> log scanned up to (69819452)
      190120 17:43:35 >> log scanned up to (70057776)
      190120 17:43:36 >> log scanned up to (70092814)
      Error: failed to execute query FLUSH NO_WRITE_TO_BINLOG TABLES: Query execution was interrupted (max_statement_time exceeded)
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548006063 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-encrypt-tables --mysqld=--innodb-encrypt-log --mysqld=--innodb-encryption-threads=4 --mysqld=--aria-encrypt-tables=1 --mysqld=--encrypt-tmp-disk-tables=1 --mysqld=--file-key-management --mysqld=--file-key-management-filename=/home/travis/mariadb-toolbox/data/keys.txt --mysqld=--plugin-load-add=file_key_management --mysqld=--innodb-compression-algorithm=none --engine=InnoDB --scenario=MariaBackupIncremental --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      InnoDB: innodb-page-size mismatch in tablespace upon prepare from incremental backup

      https://travis-ci.org/elenst/travis-tests/jobs/481813553 (Trial 4)

      Backup, logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-6.tar.gz

      10.1 1d72db45a

      InnoDB: The universal page size of the database is set to 8192.
      mariabackup: using the following InnoDB configuration for recovery:
      mariabackup:   innodb_data_home_dir = ./
      mariabackup:   innodb_data_file_path = ibdata1:12M:autoextend
      mariabackup:   innodb_log_group_home_dir = /home/travis/logs/vardir/backup_2/
      mariabackup:   innodb_log_files_in_group = 1
      mariabackup:   innodb_log_file_size = 11206656
      mariabackup: Generating a list of tablespaces
      InnoDB: Error: Current page size 8192 !=  page size on page 16384
      2019-01-20 17:49:28 140020475705152 [ERROR] InnoDB: innodb-page-size mismatch in tablespace ./test/t8.ibd (table test/t8)
      2019-01-20 17:49:28 7f590eb73740  InnoDB: Operating system error number 2 in a file operation.
      InnoDB: The error means the system cannot find the path specified.
      InnoDB: If you are installing InnoDB, remember that you must create
      InnoDB: directories yourself, InnoDB does not create them.
      InnoDB: Error: could not open single-table tablespace file ./test/t8.ibd
      InnoDB: We do not continue the crash recovery, because the table may become
      InnoDB: corrupt if we cannot apply the log records in the InnoDB log to it.
      InnoDB: To fix the problem and start mysqld:
      InnoDB: 1) If there is a permission problem in the file and mysqld cannot
      InnoDB: open the file, you should modify the permissions.
      InnoDB: 2) If the table is not needed, or you can restore it from a backup,
      InnoDB: then you can remove the .ibd file, and InnoDB will do a normal
      InnoDB: crash recovery and ignore that table.
      InnoDB: 3) If the file system or the disk is broken, and you cannot remove
      InnoDB: the .ibd file, you can set innodb_force_recovery > 0 in my.cnf
      InnoDB: and force InnoDB to continue crash recovery here.
      InnoDB: Error: Current page size 8192 !=  page size on page 16384
      2019-01-20 17:49:28 140020475705152 [ERROR] InnoDB: innodb-page-size mismatch in tablespace ./test/t7.ibd (table test/t7)
       
      etc.
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548006364 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=8K --engine=InnoDB --scenario=MariaBackupIncremental --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      InnoDB: The combined size of ib_logfiles should be bigger than ... upon prepare from incremental backup

      This one was observed on DML-only scenario as well, so it might end up filed as a separate bug report.

      https://travis-ci.org/elenst/travis-tests/jobs/481813565 (Trial 4)
      https://travis-ci.org/elenst/travis-tests/jobs/488215039 [2668 20 2]

      Backup, logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-7.tar.gz

      10.1 1d72db45a

      2019-01-20 19:18:04 139987781945152 [Note] InnoDB: Completed initialization of buffer pool
      2019-01-20 19:18:04 139987781945152 [ERROR] InnoDB: The combined size of ib_logfiles should be bigger than
      InnoDB: 200 kB * innodb_thread_concurrency.
      2019-01-20 19:18:04 7f5172040740  InnoDB: Assertion failure in thread 139987781945152 in file ha_innodb.cc line 22037
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      190120 19:18:04 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.1.38-MariaDB-debug
      key_buffer_size=0
      read_buffer_size=131072
      max_used_connections=0
      max_threads=1
      thread_count=0
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5416 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48400
      mysys/stacktrace.c:267(my_print_stacktrace)[0x55aaf140b116]
      sql/signal_handler.cc:168(handle_fatal_signal)[0x55aaf0e3303d]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f5171c27390]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f516febe428]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f516fec002a]
      handler/ha_innodb.cc:22039(ib_logf(ib_log_level_t, char const*, ...))[0x55aaf11bc0d3]
      log/log0log.cc:887(log_calc_max_ages())[0x55aaf12161b7]
      log/log0log.cc:1135(log_group_init(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long))[0x55aaf1216f9a]
      srv/srv0start.cc:2494(innobase_start_or_create_for_mysql())[0x55aaf130fdd0]
      mariabackup/xtrabackup.cc:1801(innodb_init())[0x55aaf0a631f8]
      mariabackup/xtrabackup.cc:5982(xtrabackup_prepare_func(int, char**))[0x55aaf0a6c395]
      mariabackup/xtrabackup.cc:6830(main)[0x55aaf0a6e6be]
      /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f516fea9830]
      /home/travis/server/bin/mariabackup(_start+0x29)[0x55aaf0a5ee49]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548011660 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=64K --engine=InnoDB --scenario=MariaBackupIncremental --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      InnoDB: Tried to read 1376256 bytes at offset 0. Was only able to read 1310720 upon incremental backup

      https://travis-ci.org/elenst/travis-tests/jobs/481813567 (Trial 4)

      Backup, logs: ftp://ftp.askmonty.org/public/mdev18324/mdev18324-8.tar.gz

      10.1 1d72db45a

      190120 19:26:43 [01] Copying ./test/t7.ibd to /home/travis/logs/vardir/backup_2/test/t7.ibd.delta
      190120 19:26:43 [01]        ...done
      190120 19:26:43 [01] Copying ./test/t6.ibd to /home/travis/logs/vardir/backup_2/test/t6.ibd.delta
      2019-01-20 19:26:43 139780396857088 [ERROR] InnoDB: Tried to read 1376256 bytes at offset 0. Was only able to read 1310720.
      [01] mariabackup: Error: xtrabackup_copy_datafile() failed.
      [01] mariabackup: Error: failed to copy datafile.
      Warning: 2048 bytes lost at 0x7f211c001680, allocated by T@0 at 190120 19:26:43 >> log scanned up to (113736265)
      0x55a35d4f65c1, mysys/array.c:70, mysys/hash.c:97, mysys/thr_mutex.c:178, mysys/thr_mutex.c:320, psi/mysql_thread.h:689, sql/log.cc:8665, sql/log.cc:8717
      

      elenst-jira-refs 19e81706f Toolbox: dbef832b2

      perl ./runall-new.pl --basedir=/home/travis/server --vardir=/home/travis/logs/vardir --duration=350 --threads=4 --seed=1548012191 --reporters=Backtrace,ErrorLog,Deadlock --skip-gendata --gendata-advanced --views --grammar=conf/mariadb/generic-dml.yy --redefine=conf/mariadb/bulk_insert.yy --mysqld=--log_output=FILE --mysqld=--max-statement-time=20 --mysqld=--lock-wait-timeout=10 --mysqld=--loose-innodb-lock-wait-timeout=5 --mysqld=--loose-debug_assert_on_not_freed_memory=0 --filter=/home/travis/mariadb-toolbox/travis/10.4-combo-filter.ff --mysqld=--innodb-encrypt-tables --mysqld=--innodb-encrypt-log --mysqld=--innodb-encryption-threads=4 --mysqld=--aria-encrypt-tables=1 --mysqld=--encrypt-tmp-disk-tables=1 --mysqld=--file-key-management --mysqld=--file-key-management-filename=/home/travis/mariadb-toolbox/data/keys.txt --mysqld=--plugin-load-add=file_key_management --mysqld=--innodb-compression-algorithm=none --mysqld=--innodb-page-size=64K --engine=InnoDB --scenario=MariaBackupIncremental --redefine=conf/mariadb/alter_table.yy --redefine=conf/mariadb/modules/admin.yy
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            elenst Elena Stepanova
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.