Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-17600

Mariabackup double compresses if qpress files are present



    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 10.2.18
    • Fix Version/s: 10.2
    • Component/s: Backup
    • Labels:
    • Environment:
      ubuntu 16.04


      Due to some reason uncompressing a previously made compressed backup the qpress files (.qp extension) were still left in the data directory. For the working of MariaDB this doesn't immediately do harm as these files aren't used by the database itself. So the data directory contains files like this:
      {{rw-rr- 1 mysql root 16K Nov 2 12:36 aria_log.00000001
      rw-r---- 1 mysql root 394 Nov 2 12:36 aria_log.00000001.qp
      rw-rr- 1 mysql root 52 Nov 2 12:36 aria_log_control
      rw-r---- 1 mysql root 129 Nov 2 12:36 aria_log_control.qp}}

      If mariabackup creates a backup now, it will simply include all MariaDB specific files with the .qp extension and include these in the backup stream. Once this backup is extracted from the xbstream the (temporary) directory looks like this:

      {{rw-rr- 1 mysql root 16K Nov 2 12:36 aria_log.00000001.qp
      rw-r---- 1 mysql root 394 Nov 2 12:36 aria_log.00000001.qp.qp
      rw-rr- 1 mysql root 52 Nov 2 12:36 aria_log_control.qp
      rw-r---- 1 mysql root 129 Nov 2 12:36 aria_log_control.qp.qp}}

      Once mariabackup would get into extracting these files again, it might actually uncompress these files in the wrong order: for example first uncompress the aria_log.00000001.qp.qp and then the aria_log.00000001.qp, resulting in an older version of the aria_log file being extracted.

      Regardless of the the qpress files still left in the data directory, I personally would have expected mariabackup to ignore files with the .qp extension. Xtrabackup, upon which mariabackup has been based upon, does ignore the .qp extension and only backup files that actually are used. I think this issue is due to the addition of MariaDB specific files to xtrabackup to make it MariaDB compatible.

      My bug report is not about how these files got left there in the first place and why they had not been removed, but rather about the re-compression and inclusion of compressed files that actually should have been ignored in the first place.

      To reproduce the issue, create a backup with compression enabled:

      /usr/bin/mariabackup --defaults-file=/etc/mysql/my.cnf --backup --galera-info --parallel 1 --compress=quicklz --stream=xbstream --no-timestamp > * /some/path/to/backup.xbstream

      Then extract the backup:

      cat /some/path/to/backup.xbstream | mbstream -x -C /var/lib/mysql_new

      sudo /usr/bin/mariabackup --decompress --target-dir /var/lib/mysql_new




            vlad.lesin Vladislav Lesin
            artvidaxl Art van Scheppingen
            0 Vote for this issue
            3 Start watching this issue