Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24687

Mariabackup: Assertion `cursor->is_system() || srv_operation == SRV_OPERATION_RESTORE_DELTA || xb_close_files' failed

    XMLWordPrintable

    Details

      Description

      origin/10.6, 10.6 3f871b339429441ad907ecf7dfabdc414797e664 2021-01-25T19:48:09+02:00  build with debug+asan and run under "rr"
       
      Workflow:
      1. Start server and load some initial data
      2. Several sessions run concurrent DDL/DML
      3. During 2. is ongoing start Mariabackup --backup
      Around begin of the backup operation some errors like
      [00] 2021-01-25 19:08:01 mariabackup: Generating a list of tablespaces
      # 2021-01-25T19:08:23 [323579] | 2021-01-25 19:08:03 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
      # 2021-01-25T19:08:23 [323579] | 2021-01-25 19:08:03 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
      # 2021-01-25T19:08:23 [323579] | 2021-01-25 19:08:03 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not create them.
      # 2021-01-25T19:08:23 [323579] | 2021-01-25 19:08:03 0 [ERROR] InnoDB: File ./test/t5.ibd: 'open' returned OS error 71.
      2021-01-25 19:08:03 0 [ERROR] InnoDB: Operating system error number 2 in a file operation.
      2021-01-25 19:08:03 0 [ERROR] InnoDB: The error means the system cannot find the path specified.
      2021-01-25 19:08:03 0 [ERROR] InnoDB: If you are installing InnoDB, remember that you must create directories yourself, InnoDB does not create them.
      2021-01-25 19:08:03 0 [Warning] InnoDB: Cannot open './test/t5.ibd'.
       
      show up but the operation goes on end harvests finally
       
      mariabackup: /Server/10.6F/extra/mariabackup/fil_cur.cc:155: xb_fil_cur_result_t xb_fil_cur_open(xb_fil_cur_t*, xb_read_filt_t*, fil_node_t*, uint, ulonglong): Assertion `cursor->is_system() || srv_operation == SRV_OPERATION_RESTORE_DELTA || xb_close_files' failed.
      (rr) bt
      #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
      #1  0x00001c4d1c31e859 in __GI_abort () at abort.c:79
      #2  0x00001c4d1c31e729 in __assert_fail_base (fmt=0x1c4d1c4b4588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x563c9287f340 "cursor->is_system() || srv_operation == SRV_OPERATION_RESTORE_DELTA || xb_close_files", 
          file=0x563c9287f160 "/Server/10.6F/extra/mariabackup/fil_cur.cc", line=155, function=<optimized out>) at assert.c:92
      #3  0x00001c4d1c32ff36 in __GI___assert_fail (assertion=0x563c9287f340 "cursor->is_system() || srv_operation == SRV_OPERATION_RESTORE_DELTA || xb_close_files", file=0x563c9287f160 "/Server/10.6F/extra/mariabackup/fil_cur.cc", line=155, 
          function=0x563c9287f3c0 "xb_fil_cur_result_t xb_fil_cur_open(xb_fil_cur_t*, xb_read_filt_t*, fil_node_t*, uint, ulonglong)") at assert.c:101
      #4  0x0000563c903cd5b6 in xb_fil_cur_open (cursor=0x404449f525b0, read_filter=0x563c93ede500 <rf_pass_through>, node=0x60c000001cd8, thread_n=1, max_file_size=18446744073709551615) at /Server/10.6F/extra/mariabackup/fil_cur.cc:155
      #5  0x0000563c9037f5c3 in xtrabackup_copy_datafile (node=0x60c000001cd8, thread_n=1, dest_name=0x0, write_filter=..., corrupted_pages=...) at /Server/10.6F/extra/mariabackup/xtrabackup.cc:2749
      #6  0x0000563c903817bb in data_copy_thread_func (arg=0x604000005610) at /Server/10.6F/extra/mariabackup/xtrabackup.cc:3116
      #7  0x00007fc5a3cf1609 in start_thread (arg=<optimized out>) at pthread_create.c:477
      #8  0x00001c4d1c41b293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
       
      pluto:/data/Results/1611598278/TBR-832/dev/shm/vardir/1611598278/230/1/rr
      _RR_TRACE_DIR="." rr replay --mark-stdio
       
      RQG
      ====
      git clone https://github.com/mleich1/rqg --branch experimental RQG
       
      perl rqg.pl \
      --grammar=conf/mariadb/table_stress_innodb_nocopy1.yy \
      --gendata=conf/mariadb/table_stress.zz \
      --gendata_sql=conf/mariadb/table_stress.sql \
      --reporters=Mariabackup_linux \
      --mysqld=--innodb_use_native_aio=1 \
      --mysqld=--innodb_lock_schedule_algorithm=fcfs \
      --mysqld=--loose-idle_write_transaction_timeout=0 \
      --mysqld=--loose-idle_transaction_timeout=0 \
      --mysqld=--loose-idle_readonly_transaction_timeout=0 \
      --mysqld=--connect_timeout=60 \
      --mysqld=--interactive_timeout=28800 \
      --mysqld=--slave_net_timeout=60 \
      --mysqld=--net_read_timeout=30 \
      --mysqld=--net_write_timeout=60 \
      --mysqld=--loose-table_lock_wait_timeout=50 \
      --mysqld=--wait_timeout=28800 \
      --mysqld=--lock-wait-timeout=86400 \
      --mysqld=--innodb-lock-wait-timeout=50 \
      --no-mask \
      --queries=10000000 \
      --seed=random \
      --reporters=Backtrace \
      --reporters=ErrorLog \
      --reporters=Deadlock1 \
      --validators=None \
      --mysqld=--log_output=none \
      --mysqld=--log-bin \
      --mysqld=--log_bin_trust_function_creators=1 \
      --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
      --engine=InnoDB \
      --restart_timeout=120 \
      --mysqld=--plugin-load-add=file_key_management.so \
      --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
      --duration=300 \
      --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
      --mysqld=--loose-innodb-sync-debug \
      --mysqld=--innodb_stats_persistent=on \
      --mysqld=--innodb_adaptive_hash_index=off \
      --mysqld=--loose-max-statement-time=30 \
      --threads=1 \
      --mysqld=--innodb_page_size=16K \
      --mysqld=--innodb-buffer-pool-size=8M \
      --duration=300 \
      --no_mask \
      --workdir=<local settings> \
      --vardir=<local settings> \
      --mtr-build-thread=<local settings> \
      --basedir1=<local settings> \
      --script_debug=_nix_ \
      --rr=Extended \
      --rr_options=--chaos
       
       
      (rr)
      

        Attachments

          Activity

            People

            Assignee:
            vlad.lesin Vladislav Lesin
            Reporter:
            mleich Matthias Leich
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: