Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-37687

mariadb-dump/import: parallel partitions

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • Scripts & Clients
    • None

    Description

      Currently, mariadb-dump -T can dump multiple tables in parallel threads, but every single table is always dumped sequentially. For a big table with lots of partitions it makes sense to dump individual partitions in parallel.

      • Dump file naming: after MDEV-37483 file name is an encoded table name, as as in the server, so using #p# partition suffix — same as in the server — sounds like an obvious and safe choice.
      • Importing: two options here, mariadb-import can ignore everything after #p# suffix and load files into the same table — the server will figure out what partition the data belongs to. But a more concurrently performant approach would be to load every partition into a separate table and then use ALTER TABLE ... EXCHANGE PARTITION to put fully populated partitions in place.
      • SQL syntax to use in mariadb-dump internally:

        SELECT * FROM table_name PARTITION (partition_name)
        

      • what option to use to enable it? — don't. Simply treat all partitions in all tables as separate entities and schedule them all to be dumped in parallel according to the concurrency settings of the thread pool.

      Attachments

        Activity

          People

            wlad Vladislav Vaintroub
            serg Sergei Golubchik
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.