Details
-
Bug
-
Status: Open (View Workflow)
-
Minor
-
Resolution: Unresolved
-
10.2.18
-
None
-
ubuntu 16.04
Description
Due to some reason uncompressing a previously made compressed backup the qpress files (.qp extension) were still left in the data directory. For the working of MariaDB this doesn't immediately do harm as these files aren't used by the database itself. So the data directory contains files like this:
{{rw-rr- 1 mysql root 16K Nov 2 12:36 aria_log.00000001
rw-r---- 1 mysql root 394 Nov 2 12:36 aria_log.00000001.qp
rw-rr- 1 mysql root 52 Nov 2 12:36 aria_log_control
rw-r---- 1 mysql root 129 Nov 2 12:36 aria_log_control.qp}}
If mariabackup creates a backup now, it will simply include all MariaDB specific files with the .qp extension and include these in the backup stream. Once this backup is extracted from the xbstream the (temporary) directory looks like this:
{{rw-rr- 1 mysql root 16K Nov 2 12:36 aria_log.00000001.qp
rw-r---- 1 mysql root 394 Nov 2 12:36 aria_log.00000001.qp.qp
rw-rr- 1 mysql root 52 Nov 2 12:36 aria_log_control.qp
rw-r---- 1 mysql root 129 Nov 2 12:36 aria_log_control.qp.qp}}
Once mariabackup would get into extracting these files again, it might actually uncompress these files in the wrong order: for example first uncompress the aria_log.00000001.qp.qp and then the aria_log.00000001.qp, resulting in an older version of the aria_log file being extracted.
Regardless of the the qpress files still left in the data directory, I personally would have expected mariabackup to ignore files with the .qp extension. Xtrabackup, upon which mariabackup has been based upon, does ignore the .qp extension and only backup files that actually are used. I think this issue is due to the addition of MariaDB specific files to xtrabackup to make it MariaDB compatible.
My bug report is not about how these files got left there in the first place and why they had not been removed, but rather about the re-compression and inclusion of compressed files that actually should have been ignored in the first place.
To reproduce the issue, create a backup with compression enabled:
/usr/bin/mariabackup --defaults-file=/etc/mysql/my.cnf --backup --galera-info --parallel 1 --compress=quicklz --stream=xbstream --no-timestamp > * /some/path/to/backup.xbstream
Then extract the backup:
cat /some/path/to/backup.xbstream | mbstream -x -C /var/lib/mysql_new
sudo /usr/bin/mariabackup --decompress --target-dir /var/lib/mysql_new
I can elaborate on that. The individual file compression does not fit well into the architecture. There are only 2 possible backup types I can imagine
There is no need to reinvent a wheel and provide builtin support for functionality that is freely available and is actively developed by someone else. There are 3rd party multithreaded compressors, there are multithreaded encryptors, so that backup can actually concentrate on its core competencies, which is copying files