Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33188

Enhance mariadb-dump and mariadb-import capabilities similar to MyDumper

    XMLWordPrintable

Details

    • Epic
    • Status: Stalled (View Workflow)
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Enhance mariadb-dump features similar to MyDumper

      Backup

      • Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
      • In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
      • The big tables were split into multiple data files which will help faster restore.
        • this should be done when bulk-load in multiple connection becomes faster

      Other features that could be added to mariadb-dump that would make it more useful:

      • Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).
      • Dump tables not alphabetically, but in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases). See benchmarks in MDEV-32216.

      Restore:

      • Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
      • In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
      • Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

      We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

      Description of the intended design

      • allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
        With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
        The files are almost exactly the same as before with --tab, except for directory tree
      • mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

      Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.

      Attachments

        Issue Links

          Activity

            People

              wlad Vladislav Vaintroub
              pandi.gurusamy Pandikrishnan Gurusamy
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.