Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-39054

InnoDB-only, DML-only incremental backup

    XMLWordPrintable

Details

    Description

      Implement a new form of incremental backup by making use of the innodb_log_archive=ON (MDEV-37949) setting.

      This new incremental backup would work best when the data volume is large and there are relatively few writes. We already know that if the bulk of the InnoDB tables are being updated regularly, then the .delta that mariadb-backup creates will typically be much smaller than the log files.

      An important special case are workloads that modify only InnoDB tables and do not comprise any DDL operations. In this case, for an incremental backup it suffices to copy and apply only the archived log. Here is a rough workflow:

      1. Configure the server with innodb_log_archive=ON.
      2. Shut down the server and remember its LSN.
      3. Make a full file system copy of the data directory.
      4. Start the server and a non-DDL, InnoDB-only workload. The InnoDB log does not cover *.frm files (yet) or other storage engines.
      5. During the workload, determine the server's LSN and copy (or move) the ib_*.log files marked as read-only in the file system. The last 1 or 2 will be "hot" and read-write.
      6. On a remote system, apply the logs using the parameters innodb_log_recovery_start=(step2) and innodb_log_recovery_target=(step5).
      7. While the previous step is in progress, repeat the step preceding it (move the next batch of logs).
      8. Apply the next batch of logs, this time using the parameters innodb_log_recovery_start=(step5) and innodb_log_recovery_target=(step7).
      9. Repeat the last two steps several times.
      10. Shut down the server.
      11. Copy and apply the remaining log.
      12. Ensure that the backup directory exactly matches the server data directory.

      This kind of workflow will be a further optimization on top of MDEV-14992, which covers generic workloads that may involve DDL operations or writes to other storage engines than InnoDB.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.