Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-13310

Preparing an incremental backup twice can corrupt data

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 10.2.7
    • 10.1.29, 10.2.10
    • Backup
    • None

    Description

      Preparing already prepared incremental backup does show error message below in percona-xtrabackup as well mariabackup 10.1.25 (to protect users from incorrect action).

      xtrabackup: This target seems to be already prepared.
      		 
      xtrabackup: error: applying incremental backup needs target prepared with --apply-log-only.

      test below passes in 10.1.25 (mariabackup exits with error code 1), but fails in 10.2.7 (mariabackup exits without error):

      call mtr.add_suppression("InnoDB: New log files created");
      		 
       
      		 
      let $basedir=$MYSQLTEST_VARDIR/tmp/backup;
      		 
      let $incremental_dir1=$MYSQLTEST_VARDIR/tmp/backup_inc1;
      		 
      let $incremental_dir2=$MYSQLTEST_VARDIR/tmp/backup_inc2;
      		 
       
      		 
      CREATE TABLE t(i INT) ENGINE INNODB;
      		 
      INSERT INTO t VALUES(1);
      		 
       
      		 
      echo # Create full backup , modify table, then create incremental/differential backup;
      		 
      --disable_result_log
      		 
      exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf  --backup --target-dir=$basedir;
      		 
      --enable_result_log
      		 
      INSERT INTO t VALUES(2);
      		 
      exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf  --backup --target-dir=$incremental_dir1 --incremental-basedir=$basedir;
      		 
       
      		 
      INSERT INTO t VALUES(3);
      		 
      exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf  --backup --target-dir=$incremental_dir2 --incremental-basedir=$basedir;
      		 
       
      		 
      echo # Prepare full backup, apply incremental one;
      		 
      exec $XTRABACKUP --prepare --apply-log-only --target-dir=$basedir;
      		 
      exec $XTRABACKUP --prepare --apply-log-only --target-dir=$basedir --incremental-dir=$incremental_dir1 ;
      		 
       
      		 
      echo # This time should succeed;
      		 
      exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir2 ;
      		 
       
      		 
      # this returns an error according to test pxb170.sh
      		 
      --error 1
      		 
      exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir2 ;
      		 
       
      		 
      # Cleanup
      		 
      drop table t;
      		 
      rmdir $basedir;
      		 
      rmdir $incremental_dir1;
      		 
      rmdir $incremental_dir2;

      10.1.25:
      Completed: All 1 tests were successful.
      10.2.7:
      mysqltest: At line 35: command "$XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir2 " succeeded - should have failed with errno 1...

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13822 Mariabackup --prepare --incremental sets file size incorrectly

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            marko Marko Mäkelä added a comment -

            First, preparing a full backup should be safe to do repeatedly in Mariabackup 10.2. It should even be safe to kill the process and rerun the prepare.

            When it comes to preparing incremental backups, we must guarantee that .delta files cannot overwrite data files on which redo log has already been applied. Currently this is not the case!

            So, it is possible that the .delta files will corrupt data files by writing older data than what was applied from the redo log on the previous invocation. Because the redo log file ib_logfile0 was deleted by the first (successful) invocation of

            exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir2 ;

            re-applying the .delta files by repeating the invocation can cause unrecoverable corruption.

            My suggested fix is to remove each .delta files after applying them.

            marko Marko Mäkelä added a comment - First, preparing a full backup should be safe to do repeatedly in Mariabackup 10.2. It should even be safe to kill the process and rerun the prepare. When it comes to preparing incremental backups, we must guarantee that .delta files cannot overwrite data files on which redo log has already been applied. Currently this is not the case! So, it is possible that the .delta files will corrupt data files by writing older data than what was applied from the redo log on the previous invocation. Because the redo log file ib_logfile0 was deleted by the first (successful) invocation of exec $XTRABACKUP --prepare --target-dir=$basedir --incremental-dir=$incremental_dir2 ; re-applying the .delta files by repeating the invocation can cause unrecoverable corruption. My suggested fix is to remove each .delta files after applying them.

            People

              wlad Vladislav Vaintroub
              anikitin Andrii Nikitin (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.