Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28784

mariadb-backup prepare fails with srv_start() returned 37 (Data structure corruption).

    XMLWordPrintable

    Details

      Description

      Scenario:
      1. Start the server and load some initial data.
      2. Several sessions run a DML mix
      3. During 2. is ongoing the run
          mariabackup --backup
          mariabackup --prepare on the backuped data
      The latter fails with
      # 2022-06-07T15:46:45 [3674472] | [rr 3711177 3917]2022-06-07 15:44:57 0 [Note] InnoDB: Completed initialization of buffer pool
      # 2022-06-07T15:46:45 [3674472] | [rr 3711177 4041]2022-06-07 15:44:57 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=4258151
      # 2022-06-07T15:46:45 [3674472] | [rr 3711177 4976]2022-06-07 15:45:00 0 [Note] InnoDB: Starting final batch to recover 314 pages from redo log.
      # 2022-06-07T15:46:45 [3674472] | [rr 3711177 5686]2022-06-07 15:45:03 0 [Note] InnoDB: Set innodb_force_recovery=1 to ignore corrupted pages.
      # 2022-06-07T15:46:45 [3674472] | [rr 3711177 5690][00] 2022-06-07 15:45:03 mariadb-backup: srv_start() returned 37 (Data structure corruption).
       
      origin/bb-10.8-MDEV-28708 f431888b636328b487504e24d71753cdaf5e6189 2022-06-07T12:15:27+03:00
      Marko assumes that this problem is most probably valid for all official MariaDB versions >=10.6.
       
      pluto:/data/results/1654601840/TBR-1535/home/mleich/rqg/1654601840/219/1/rr
            # rr trace of the source DB server during mariabackup --backup and later
            # That ends with RQG killing that server because of the trouble with mariabackup --prepare.
      pluto:/data/results/1654601840/TBR-1535/home/mleich/rqg/1654601840/219/1_clone/rr
            # rr trace of mariabackup --prepare ... 
            # mariabackup-0 is the rr trace of mariabackup --backup
      pluto:/data/results/1654601840/TBR-1535/home/mleich/rqg/1654601840/219/1/data_backup
             # Filecopy of the data written by  mariabackup --backup and before the --prepare
       
      RQG
      ===
      # git clone https://github.com/mleich1/rqg --branch experimental RQG
      #
      # GIT_SHOW: HEAD -> experimental, origin/experimental fcae46996ce85de6bb0844799b69ed6ee8add272 2022-06-01T20:04:31+02:00
      # rqg.pl  : Version 4.0.5 (2022-04)
      #
      # $RQG_HOME/rqg.pl \
      # --gendata=conf/mariadb/oltp.zz \
      # --max_gd_duration=900 \
      # --grammar=conf/mariadb/oltp.yy \
      # --reporters=Mariabackup_linux \
      # --mysqld=--loose-innodb_lock_schedule_algorithm=fcfs \
      # --mysqld=--loose-idle_write_transaction_timeout=0 \
      # --mysqld=--loose-idle_transaction_timeout=0 \
      # --mysqld=--loose-idle_readonly_transaction_timeout=0 \
      # --mysqld=--connect_timeout=60 \
      # --mysqld=--interactive_timeout=28800 \
      # --mysqld=--slave_net_timeout=60 \
      # --mysqld=--net_read_timeout=30 \
      # --mysqld=--net_write_timeout=60 \
      # --mysqld=--loose-table_lock_wait_timeout=50 \
      # --mysqld=--wait_timeout=28800 \
      # --mysqld=--lock-wait-timeout=86400 \
      # --mysqld=--innodb-lock-wait-timeout=50 \
      # --no-mask \
      # --queries=10000000 \
      # --seed=random \
      # --reporters=Backtrace \
      # --reporters=ErrorLog \
      # --reporters=Deadlock1 \
      # --validators=None \
      # --mysqld=--log_output=none \
      # --mysqld=--log_bin_trust_function_creators=1 \
      # --mysqld=--loose-debug_assert_on_not_freed_memory=0 \
      # --engine=InnoDB \
      # --restart_timeout=240 \
      # --mysqld=--plugin-load-add=file_key_management.so \
      # --mysqld=--loose-file-key-management-filename=$RQG_HOME/conf/mariadb/encryption_keys.txt \
      # --mysqld=--plugin-load-add=provider_lzo.so \
      # --mysqld=--plugin-load-add=provider_bzip2.so \
      # --mysqld=--plugin-load-add=provider_lzma.so \
      # --mysqld=--plugin-load-add=provider_snappy.so \
      # --mysqld=--plugin-load-add=provider_lz4.so \
      # --duration=300 \
      # --mysqld=--loose-innodb_fatal_semaphore_wait_threshold=300 \
      # --mysqld=--loose-innodb_read_only_compressed=OFF \
      # --mysqld=--innodb_stats_persistent=on \
      # --mysqld=--innodb_adaptive_hash_index=off \
      # --mysqld=--log-bin \
      # --mysqld=--sync-binlog=1 \
      # --mysqld=--loose-innodb_evict_tables_on_commit_debug=off \
      # --mysqld=--loose-max-statement-time=30 \
      # --threads=9 \
      # --mysqld=--innodb-use-native-aio=0 \
      # --mysqld=--loose-gdb \
      # --mysqld=--loose-debug-gdb \
      # --rr=Extended \
      # --rr_options=--wait \
      # --mysqld=--loose_innodb_change_buffering=purges \
      # --mysqld=--innodb_rollback_on_timeout=ON \
      # --vardir_type=slow \
      # --mysqld=--innodb_page_size=16K \
      # --mysqld=--innodb-buffer-pool-size=8M \
      # --no_mask \
      # --mtr-build-thread=948 \
      # --batch \
      # <local settings>
       
      Error pattern for RQG
      [ 'TBR-1535', 'ERROR: Reporter Mariabackup : First prepare returned.+ Executing backup: .+ Executing first prepare: .+mariadb-backup: srv_start\(\) returned 37 \(Data structure corruption\).+RESULT: The RQG run ended with status STATUS_BACKUP_FAILURE' ],
       
      
      

        Attachments

          Activity

            People

            Assignee:
            vlad.lesin Vladislav Lesin
            Reporter:
            mleich Matthias Leich
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:

                Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.