Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-5984

Allow MaxScale to also use zstd as a compression option for rebuilding replicas

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      I did some testing with zstd and it is yielded very fast and seemingly efficient compression results.
      On an 8 vCPU server I created a tarball of a 15GB MariaDB directory. Using all 8 vCPUs it took pigz 2 minutes & 45 seconds with a 4.8 GB output. For zstd using all 8 vCPUs it took 45 seconds for 4.7 GB output.
      I then created 2 4 vCPU servers. I loaded 8.7GB worth of data to one of the servers. I then did a streaming backup from Server A to Server B. For pigz it took 0:03:53 and averaged a transfer rate of 37.7MiB/s. For zstd it took 0:01:10 and averaged a transfer rate of 125MiB/s. For each command I used mariadb-backup --parallel=2 and told pigz and zstd to use 2 cores.
      I ran these commands 4 times each and got similar numbers. Looks like it makes sense to use zstd to speed things up.

      I feel like the inclusion of zstd could help improve the performance of the replica rebuild process.

      Attachments

        Activity

          People

            allen.herrera Allen Herrera
            kyle.hutchinson Kyle Hutchinson
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.