Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-22165

CONVERT TABLE: move in partition from existing table

Details

    Description

      Adding a new table to a partitioned table is now quite cumbersome:
      ('archive' is here a partitioned table and p1 is a normal table)

      ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
      ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
      DROP TABLE p1;

      It would be nice to be able to do this in one step:
      ALTER TABLE archive CONVERT TABLE tp1 TO PARTITION p1 VALUES LESS THAN (200);

      The operation crash safety must be protected by DDL logging.

      Attachments

        Issue Links

          Activity

            monty Michael Widenius created issue -
            monty Michael Widenius made changes -
            Field Original Value New Value
            monty Michael Widenius made changes -
            Summary Making it easier to add a new table to a partitoned table Make it easier to add a new table to a partitoned table
            serg Sergei Golubchik made changes -
            Assignee Alexey Botchkov [ holyfoot ]
            julien.fritsch Julien Fritsch made changes -
            Fix Version/s 10.6 [ 24028 ]
            Fix Version/s 10.5 [ 23123 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.6 [ 24028 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Assignee Alexey Botchkov [ holyfoot ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.7 [ 24805 ]
            serg Sergei Golubchik made changes -
            Assignee Oleksandr Byelkin [ sanja ]
            serg Sergei Golubchik made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Due Date 2021-09-14
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Dmitry Shulga [ JIRAUSER47315 ]
            shulga Dmitry Shulga made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            shulga Dmitry Shulga added a comment -

            Just to be clear, the statement like the following one
            "ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;"
            borrowed from the task description doesn't imply dropping the table 'p1'.
            In other words, after the statement be executed the table 'p1' does still exists and should be dropped explicitly if required.

            shulga Dmitry Shulga added a comment - Just to be clear, the statement like the following one "ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;" borrowed from the task description doesn't imply dropping the table 'p1'. In other words, after the statement be executed the table 'p1' does still exists and should be dropped explicitly if required.

            this is a replacement for EXCHANGE PARTITION, so it should not copy the data, but simply rename the table to become a partition. If you think ADD PARTITION is confusing, use IMPORT PARTITION for example.

            serg Sergei Golubchik added a comment - this is a replacement for EXCHANGE PARTITION, so it should not copy the data, but simply rename the table to become a partition. If you think ADD PARTITION is confusing, use IMPORT PARTITION for example.
            midenok Aleksey Midenkov made changes -

            Then MDEV-22166 maybe should use EXPORT? There are some problems with EXTRACT since it is SQL function originally: it was possible to create table "extract", now it is not.

            midenok Aleksey Midenkov added a comment - Then MDEV-22166 maybe should use EXPORT? There are some problems with EXTRACT since it is SQL function originally: it was possible to create table "extract", now it is not.

            Yes, IMPORT/EXPORT look much better than ADD/EXTRACT

            serg Sergei Golubchik added a comment - Yes, IMPORT / EXPORT look much better than ADD / EXTRACT
            midenok Aleksey Midenkov made changes -
            Summary Make it easier to add a new table to a partitoned table IMPORT PARTITION: add a new table to a partitoned table
            midenok Aleksey Midenkov added a comment - - edited

            OTOH classical meaning of IMPORT/EXPORT implies keeping the original data. If we move data instead that is not quite user-intuitive for these terms. We can be straightforward for what we do by using RENAME keyword:

            ALTER TABLE archive RENAME PARTITION p1 TO TABLE tp1;
            ALTER TABLE archive RENAME TABLE tp1 TO PARTITION p1 VALUES LESS THAN (200);

            I believe there is still no fast partition rename in MySQL/MariaDB, so we could make that as a bonus:

            ALTER TABLE archive RENAME PARTITION p1 TO p11;

            Btw, the last syntax is supported by Oracle.

            midenok Aleksey Midenkov added a comment - - edited OTOH classical meaning of IMPORT/EXPORT implies keeping the original data. If we move data instead that is not quite user-intuitive for these terms. We can be straightforward for what we do by using RENAME keyword: ALTER TABLE archive RENAME PARTITION p1 TO TABLE tp1; ALTER TABLE archive RENAME TABLE tp1 TO PARTITION p1 VALUES LESS THAN (200); I believe there is still no fast partition rename in MySQL/MariaDB, so we could make that as a bonus: ALTER TABLE archive RENAME PARTITION p1 TO p11; Btw, the last syntax is supported by Oracle.
            midenok Aleksey Midenkov made changes -
            Description Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;
            Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive IMPORT PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;
            shulga Dmitry Shulga added a comment -

            I support the Aleksey's opinion that meaning of IMPORT/EXPORT implies keeping the original data.

            On the other hand, I believe that the new syntax for adding a partition by virtue of moving a separate table to table's partition should be aligned with current syntax of ALTER TABLE in respect of adding partitions. Therefore from my point of view the following syntax would be more clear in expressing the idea of moving data from external table to table's partition:
            ALTER TABLE ... ADD PARTITION (PARTITION ... ) MIGRATE FROM TABLE ...

            For the sake of uniformity, the keyword MIGRATE could also be used in the ALTER statement for moving a partition to an external table:
            ALTER TABLE ... MIGRATE PARTITION ... TO TABLE ...

            shulga Dmitry Shulga added a comment - I support the Aleksey's opinion that meaning of IMPORT/EXPORT implies keeping the original data. On the other hand, I believe that the new syntax for adding a partition by virtue of moving a separate table to table's partition should be aligned with current syntax of ALTER TABLE in respect of adding partitions. Therefore from my point of view the following syntax would be more clear in expressing the idea of moving data from external table to table's partition: ALTER TABLE ... ADD PARTITION (PARTITION ... ) MIGRATE FROM TABLE ... For the sake of uniformity, the keyword MIGRATE could also be used in the ALTER statement for moving a partition to an external table: ALTER TABLE ... MIGRATE PARTITION ... TO TABLE ...

            Using the MIGRATE keyword looks like a nice idea. Keeping ADD PARTITION syntax compatibility is something that I also thought of. The downside of that is kind of longer syntax: two sequential PARTITION keywords. I hope we can make PARTITION inside the brackets optional.

            midenok Aleksey Midenkov added a comment - Using the MIGRATE keyword looks like a nice idea. Keeping ADD PARTITION syntax compatibility is something that I also thought of. The downside of that is kind of longer syntax: two sequential PARTITION keywords. I hope we can make PARTITION inside the brackets optional.
            midenok Aleksey Midenkov made changes -
            Summary IMPORT PARTITION: add a new table to a partitoned table ADD PARTITION: migrate partition from existing table
            midenok Aleksey Midenkov made changes -
            Description Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive IMPORT PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;
            Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive ADD PARTITION ([PARTITION] p1 VALUES LESS THAN (200)) MIGRATE [FROM] [TABLE] p1;

            As a syntax improvement we can make PARTITION optional in partition specification, as well as FROM and TABLE keywords.

            The operation crash safety must be protected by DDL logging.
            midenok Aleksey Midenkov made changes -

            What happens if the two tables are different in one of these aspects?

            • Indexes
            • Same visible indexes, but different IGNORED indexes
            • INVISIBLE columns
            • Compressed columns
            • CHECK
            • Type size (SMALLINT -> INT)
            • NULL / NOT NULL
            • Storage engine

            Indexes differences are not unlikely, in my opinion.

            What happens if the original table has triggers?

            f_razzoli Federico Razzoli added a comment - What happens if the two tables are different in one of these aspects? Indexes Same visible indexes, but different IGNORED indexes INVISIBLE columns Compressed columns CHECK Type size (SMALLINT -> INT) NULL / NOT NULL Storage engine Indexes differences are not unlikely, in my opinion. What happens if the original table has triggers?

            Somehow I don't see how the original

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;
            

            is worse or more confusing than

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) MIGRATE FROM TABLE p1;
            

            if anything, the former is shorter and only uses familiar keywords, it doesn't introduce a new keyword into the grammar

            serg Sergei Golubchik added a comment - Somehow I don't see how the original ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1; is worse or more confusing than ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) MIGRATE FROM TABLE p1; if anything, the former is shorter and only uses familiar keywords, it doesn't introduce a new keyword into the grammar

            f_razzoli, this feature introduces a convenience command to replace the following sql script:

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;
            

            it does not allow to do anything that wasn't possible before and it should generally behave exactly as the script above. Also in regards to indexes, invisible columns, constraints, etc.

            serg Sergei Golubchik added a comment - f_razzoli , this feature introduces a convenience command to replace the following sql script: ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)); ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1; DROP TABLE p1; it does not allow to do anything that wasn't possible before and it should generally behave exactly as the script above. Also in regards to indexes, invisible columns, constraints, etc.
            serg Sergei Golubchik made changes -
            Comment [ A comment with security level 'Developers' was removed. ]
            midenok Aleksey Midenkov made changes -
            Summary ADD PARTITION: migrate partition from existing table MIGRATE PARTITION: move in partition from existing table
            midenok Aleksey Midenkov made changes -
            Description Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive ADD PARTITION ([PARTITION] p1 VALUES LESS THAN (200)) MIGRATE [FROM] [TABLE] p1;

            As a syntax improvement we can make PARTITION optional in partition specification, as well as FROM and TABLE keywords.

            The operation crash safety must be protected by DDL logging.
            Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive MIGRATE PARTITION ([PARTITION] p1 VALUES LESS THAN (200)) FROM [TABLE] p1;

            As a syntax improvement we can make PARTITION optional in partition specification, as well as FROM and TABLE keywords.

            The operation crash safety must be protected by DDL logging.
            midenok Aleksey Midenkov made changes -
            Description Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive MIGRATE PARTITION ([PARTITION] p1 VALUES LESS THAN (200)) FROM [TABLE] p1;

            As a syntax improvement we can make PARTITION optional in partition specification, as well as FROM and TABLE keywords.

            The operation crash safety must be protected by DDL logging.
            Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive MIGRATE PARTITION ([PARTITION] p1 VALUES LESS THAN (200)) FROM [TABLE] p1;

            The operation crash safety must be protected by DDL logging.
            midenok Aleksey Midenkov made changes -
            Summary MIGRATE PARTITION: move in partition from existing table CONVERT PARTITION: move in partition from existing table
            midenok Aleksey Midenkov made changes -
            Summary CONVERT PARTITION: move in partition from existing table CONVERT TABLE: move in partition from existing table
            midenok Aleksey Midenkov made changes -
            Description Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive MIGRATE PARTITION ([PARTITION] p1 VALUES LESS THAN (200)) FROM [TABLE] p1;

            The operation crash safety must be protected by DDL logging.
            Adding a new table to a partitioned table is now quite cumbersome:
            ('archive' is here a partitioned table and p1 is a normal table)

            ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
            ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
            DROP TABLE p1;

            It would be nice to be able to do this in one step:
            ALTER TABLE archive CONVERT TABLE tp1 TO PARTITION p1 VALUES LESS THAN (200);

            The operation crash safety must be protected by DDL logging.
            midenok Aleksey Midenkov made changes -
            Assignee Dmitry Shulga [ JIRAUSER47315 ] Aleksey Midenkov [ midenok ]
            midenok Aleksey Midenkov made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            midenok Aleksey Midenkov added a comment - Please review bb-10.7-midenok-MDEV-22165
            midenok Aleksey Midenkov made changes -
            Assignee Aleksey Midenkov [ midenok ] Sergei Golubchik [ serg ]
            Status Stalled [ 10000 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Elena Stepanova [ elenst ]
            Status In Review [ 10002 ] Stalled [ 10000 ]

            pushed together with MDEV-22166

            serg Sergei Golubchik added a comment - pushed together with MDEV-22166
            serg Sergei Golubchik made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            Due Date 2021-09-14
            elenst Elena Stepanova made changes -
            Assignee Elena Stepanova [ elenst ] Ramesh Sivaraman [ JIRAUSER48189 ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.7.1 [ 26120 ]
            Fix Version/s 10.7 [ 24805 ]
            Assignee Ramesh Sivaraman [ JIRAUSER48189 ] Aleksey Midenkov [ midenok ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            midenok Aleksey Midenkov made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 106792 ] MariaDB v4 [ 134232 ]
            elenst Elena Stepanova made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            midenok Aleksey Midenkov made changes -
            midenok Aleksey Midenkov made changes -
            midenok Aleksey Midenkov made changes -
            serg Sergei Golubchik made changes -
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -

            People

              midenok Aleksey Midenkov
              monty Michael Widenius
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.