Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-33188

Enhance mariadb-dump and mariadb-import capabilities similar to MyDumper

Details

    • Epic
    • Status: Stalled (View Workflow)
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Enhance mariadb-dump features similar to MyDumper

      Backup

      • Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
      • In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
      • The big tables were split into multiple data files which will help faster restore.
        • this should be done when bulk-load in multiple connection becomes faster

      Other features that could be added to mariadb-dump that would make it more useful:

      • Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).
      • Dump tables not alphabetically, but in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases). See benchmarks in MDEV-32216.

      Restore:

      • Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
      • In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
      • Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

      We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

      Description of the intended design

      • allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
        With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
        The files are almost exactly the same as before with --tab, except for directory tree
      • mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

      Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.

      Attachments

        Issue Links

          Activity

            pandi.gurusamy Pandikrishnan Gurusamy created issue -
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Field Original Value New Value
            Description Enhance mariadb-dump features similar to MyDumper


            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            *
            * Using mariadb-import can restore only one database at a time and user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.
            Enhance mariadb-dump features similar to MyDumper

            *Backup
            ** Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout


            *Restore:
            ** Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup
            ** Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout


            *Restore:
            ** Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            Enhance mariadb-dump features similar to MyDumper

            *Backup
            ** Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout


            *Restore:
            ** Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup
            ** Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout


            *Restore:
            ** Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Summary Enhance mariadb-dump capabilities similar to MyDumper Enhance mariadb-dump and mariadb-import capabilities similar to MyDumper
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables where split into multiple data files which will help faster restore.


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Assignee Mason Sharp [ JIRAUSER54052 ]
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables where split into multiple data files which will help faster restore.


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single line command backup and restore.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables where split into multiple data files which will help faster restore.


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single-line command for backup and restore.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables where split into multiple data files which will help faster restore.


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to enhance the functionality of mariadb-dump and mariadb-import similar to Mydumper/myloader as a single-line command for backup and restore.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            serg Sergei Golubchik made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.


            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump, the binlog positions are stored in a separate file. It was displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It was displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            pandi.gurusamy Pandikrishnan Gurusamy made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It was displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            ralf.gebhardt Ralf Gebhardt made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 11.5 [ 29506 ]
            masonmariadb Mason Sharp (Inactive) made changes -
            Assignee Mason Sharp [ JIRAUSER54052 ]
            masonmariadb Mason Sharp (Inactive) made changes -
            Assignee Vladislav Vaintroub [ wlad ]
            wlad Vladislav Vaintroub made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree
            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Alternatively, maybe we do not need directory tree, and have dbname.table_name.txt and dbname.table_name.txt
            wlad Vladislav Vaintroub made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree
            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Alternatively, maybe we do not need directory tree, and have dbname.table_name.txt and dbname.table_name.txt
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            wlad Vladislav Vaintroub made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            ralf.gebhardt Ralf Gebhardt made changes -
            monty Michael Widenius made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            monty Michael Widenius made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly.

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features that could be added to mariadb-dump that would make it more useful:
            * Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
            * Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features that could be added to mariadb-dump that would make it more useful:
            * Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
            * Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features that could be added to mariadb-dump that would make it more useful:
            * Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).
            * Dump tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
            * Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            serg Sergei Golubchik made changes -
            Description Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features that could be added to mariadb-dump that would make it more useful:
            * Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).
            * Dump tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
            * Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            Enhance mariadb-dump features similar to MyDumper

            *Backup*
            * Increase the parallelism, MDEV-32216 using --tab option we can able to backup only one database at a time.
            * In mariadb-dump --tab, the binlog positions are not stored in a separate file. It is displayed in stdout
            * The big tables were split into multiple data files which will help faster restore.
            ** this should be done when bulk-load in multiple connection becomes faster

            Other features that could be added to mariadb-dump that would make it more useful:
            * Compression of the dump (especially with --tab). This also implies that we would implement de-compression of the files ion the server (MDEV-28395).
            * Dump tables not alphabetically, but in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases). See benchmarks in MDEV-32216.

            *Restore:*

            * Using mariadb-import can restore only one database at a time and the user need to restore the table structure and data files separately. We should enhance mariadb-import with an option to re-create the table structure first and then start the data import.
            * In case of customers having 100+ databases mariadb-dump (with --tab) and mariadb-import are not user-friendly. (Not hard to fix)
            * Have mariadb-import in parallel mode start importing the tables in inverse size order (biggest tables first). This will help ensuring that the user does not have to wait for the big tables to finish at the end of the backup (in most cases).

            We need to improve the functionality of mariadb-dump and mariadb-import to allow for the backup and restoration of multiple databases using a single command.

            *Description of the intended design*
            - allow multiple databases with a new switch --dir (suggestions for a better name welcome, as I got no good idea at the moment)
            With this set, mariadb-dump creates a directory tree structure in the given path, <path>/dbname/ for each db, and <tablename>.txt (tab-separated data, created using SELECT INTO OUTFILE) and tablename.sql with DDL . for all tables under the corresponding db directory.
            The files are almost exactly the same as before with --tab, except for directory tree

            - mariadb-import also takes new --dir parameter, that points to directory tree created by dump, executes all *.sql files to create tables (and databases if not exist), loads the data using "LOAD DATA INFILE" for the .txt files

            Big tables won't be split into small tables now, and I do not think they should be in the future. If required, threading should be handled transparently by LOAD DATA INFILE, i.e by the server, rather than loading single table from 2 different connections. LOAD DATA is our officiaL bulk-loading interface, and we better improve it in the server, rather than building workarounds.
            wlad Vladislav Vaintroub made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            wlad Vladislav Vaintroub made changes -
            wlad Vladislav Vaintroub made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 11.6 [ 29515 ]
            Fix Version/s 11.5 [ 29506 ]
            JIraAutomate JiraAutomate made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 11.6 [ 29515 ]
            serg Sergei Golubchik made changes -
            Issue Type New Feature [ 2 ] Epic [ 5 ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 11.7 [ 29815 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 11.6 [ 29515 ]
            Fix Version/s 11.7 [ 29815 ]
            wlad Vladislav Vaintroub made changes -
            wlad Vladislav Vaintroub made changes -
            wlad Vladislav Vaintroub made changes -
            wlad Vladislav Vaintroub made changes -
            wlad Vladislav Vaintroub made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            wlad Vladislav Vaintroub made changes -
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 11.7 [ 29815 ]
            Fix Version/s 11.6(EOL) [ 29515 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 11.7 [ 29815 ]

            People

              wlad Vladislav Vaintroub
              pandi.gurusamy Pandikrishnan Gurusamy
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.