Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26810

Incremental backup not compatible with compression

Details

    • Task
    • Status: Open (View Workflow)
    • Minor
    • Resolution: Unresolved
    • None
    • Backup
    • None

    Description

      The mariabackup option --compress was compatible with incremental backups. However it is deprecated, users should use streaming to 3rd-party tools. Such a compressed backup cannot be used as base-directory for an incremental backup.

      Feature request: Incorporate compression tools like gzip / 7zip into mariabackup and remove "deprecated" marker, to make incremental backups compatible with compression again.

      Attachments

        Activity

          wlad Vladislav Vaintroub added a comment - - edited

          incremental backup needs to apply diffs to baseline backup, for prepare. It can't apply diffs to compressed files (.qp, .gz, or anything else), without decompression. If you were on Windows, you could use a transparently compressed directory for base, and/or incremental directories. Maybe you can have some luck with ZFS on Linux, according to google it can have transparently compressed directories, too.

          wlad Vladislav Vaintroub added a comment - - edited incremental backup needs to apply diffs to baseline backup, for prepare. It can't apply diffs to compressed files (.qp, .gz, or anything else), without decompression. If you were on Windows, you could use a transparently compressed directory for base, and/or incremental directories. Maybe you can have some luck with ZFS on Linux, according to google it can have transparently compressed directories, too.
          JJworx Jochen added a comment -

          Thanks Vlad. This workaround might help some people, I don't really need it, I'd like to leave this feature request standing!

          The old --compress option compresses file-by-file and automatically decompresses for the diff for incremental backups. Only it uses QuickLZ, and current gzip compression has ~40% smaller results! Having some programming background I think it's possible with a small workload to incorporate gzip into mariabackup the same way it uses QuickLZ.

          JJworx Jochen added a comment - Thanks Vlad. This workaround might help some people, I don't really need it, I'd like to leave this feature request standing! The old --compress option compresses file-by-file and automatically decompresses for the diff for incremental backups. Only it uses QuickLZ, and current gzip compression has ~40% smaller results! Having some programming background I think it's possible with a small workload to incorporate gzip into mariabackup the same way it uses QuickLZ.

          People

            Unassigned Unassigned
            JJworx Jochen
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.