Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-22913

Error: xtrabackup_apply_delta() upon preparing incremental backup

    XMLWordPrintable

Details

    Description

      This report is about a non-debug 10.5 build. Debug builds fail with an assertion failure from MDEV-18589. However, according to MDEV-18589, on non-debug builds it didn't have the failure described here, so I'm not sure it's the same problem.

      10.5 e290e5a7

      [00] 2020-06-16 15:12:21 page size for /dev/shm/var_xbackup_apply/backup_2//ibdata1.delta is 16384 bytes
      [00] 2020-06-16 15:12:21 Applying /dev/shm/var_xbackup_apply/backup_2//ibdata1.delta to ./ibdata1...
      [00] 2020-06-16 15:12:21 page size for /dev/shm/var_xbackup_apply/backup_2//test/alt_t1.ibd.delta is 8192 bytes
      [00] 2020-06-16 15:12:21 Can't create file ./test/alt_t1.ibd
      [00] 2020-06-16 15:12:21 error: can't open ./test/alt_t1.ibd
      [00] 2020-06-16 15:12:21 Error: xtrabackup_apply_delta(): failed to apply /dev/shm/var_xbackup_apply/backup_2//test/alt_t1.ibd.delta to ./test/alt_t1.ibd.
      

      To reproduce:

      git clone https://github.com/MariaDB/randgen --branch mdev22913 rqg-mdev22913
      rqg-mdev22913
      perl ./runall-trials.pl --basedir=<basedir> --trials=5 --skip-gendata --duration=150 --threads=4 --scenario=MariaBackupIncremental --grammar=mdev22913.yy --vardir=/dev/shm/var_mdev22913
      

      Remember to set the basedir on the command line.
      For reproducing, use a 10.5 non-debug build.

      The grammar in the test above only contains single query:

      query:
        CREATE OR REPLACE TABLE { 't'.$prng->int(1,9) } (b BIT) ENGINE=InnoDB ROW_FORMAT COMPRESSED;
      

      It currently fails every time for me, but there is always room for non-determinism.
      The command line above runs the test 5 times, until it fails (in any way). Given the number of problems with incremental backup (or even any backup), it can occasionally fail with a different problem than the one described here. If you want to ignore all other ones and only focus on this one, run with the extra option

      --output="Error: xtrabackup_apply_delta"
      

      Then it will ignore other errors and automatically continue running the configured 5 trials until it hits this exact failure.

      Logs are attached.

      In the community server, this failure was only observed on 10.5 non-debug.

      10.4 and 10.5 debug and sometimes 10.4 non-debug fail with MDEV-18589 instead.
      10.4 non-debug also fails non-deterministically with

      2020-06-16 20:13:22 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
      [00] FATAL ERROR: 2020-06-16 20:13:23 mariabackup: innodb_init() returned 11 (Generic error).
      

      (to be filed separately).

      10.2 non-debug fails non-deterministically with MDEV-18550.
      10.3 non-debug fails non-deterministically with

      [00] FATAL ERROR: 2020-06-16 20:03:57 xtrabackup_copy_logfile() failed.
      

      Attachments

        1. mbackup_prepare_2.log
          14 kB
        2. mbackup_prepare_1.log
          1.65 MB
        3. mbackup_prepare_0.log
          2.22 MB
        4. mbackup_backup_3.log
          2.60 MB
        5. mbackup_backup_2.log
          568 kB
        6. mbackup_backup_1.log
          1.48 MB
        7. mbackup_backup_0.log
          2.01 MB

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.