[MDEV-22165] CONVERT TABLE: move in partition from existing table Created: 2020-04-06  Updated: 2024-01-03  Resolved: 2021-10-26

Status: Closed
Project: MariaDB Server
Component/s: Partitioning
Fix Version/s: 10.7.1

Type: Task Priority: Critical
Reporter: Michael Widenius Assignee: Aleksey Midenkov
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
is blocked by MDEV-17567 Atomic DDL Closed
PartOf
is part of MDEV-22162 Make partitions more flexible to use Open
Problem/Incident
causes MDEV-29841 Partition by system_time can be conve... Closed
causes MDEV-31014 Database privileges are insufficient ... Closed
causes MDEV-33131 Server crash or assertion failure upo... Open
causes MDEV-33146 CONVERT TABLE TO PARTITION does not w... Open
causes MDEV-33154 Confusing error ER_PARTITION_INSTEAD_... Open
causes MDEV-33170 ASAN errors upon CONVERT TABLE TO PAR... Open
Relates
relates to MDEV-17567 Atomic DDL Closed
relates to MDEV-22164 WITHOUT VALIDATION for EXCHANGE PARTI... Closed
relates to MDEV-22166 CONVERT PARTITION: move out partition... Closed
relates to MDEV-27180 Fully atomic partitioning DDL operations Stalled
relates to MDEV-31014 Database privileges are insufficient ... Closed

 Description   

Adding a new table to a partitioned table is now quite cumbersome:
('archive' is here a partitioned table and p1 is a normal table)

ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
DROP TABLE p1;

It would be nice to be able to do this in one step:
ALTER TABLE archive CONVERT TABLE tp1 TO PARTITION p1 VALUES LESS THAN (200);

The operation crash safety must be protected by DDL logging.



 Comments   
Comment by Dmitry Shulga [ 2021-08-02 ]

Just to be clear, the statement like the following one
"ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;"
borrowed from the task description doesn't imply dropping the table 'p1'.
In other words, after the statement be executed the table 'p1' does still exists and should be dropped explicitly if required.

Comment by Sergei Golubchik [ 2021-08-02 ]

this is a replacement for EXCHANGE PARTITION, so it should not copy the data, but simply rename the table to become a partition. If you think ADD PARTITION is confusing, use IMPORT PARTITION for example.

Comment by Aleksey Midenkov [ 2021-08-03 ]

Then MDEV-22166 maybe should use EXPORT? There are some problems with EXTRACT since it is SQL function originally: it was possible to create table "extract", now it is not.

Comment by Sergei Golubchik [ 2021-08-03 ]

Yes, IMPORT/EXPORT look much better than ADD/EXTRACT

Comment by Aleksey Midenkov [ 2021-08-05 ]

OTOH classical meaning of IMPORT/EXPORT implies keeping the original data. If we move data instead that is not quite user-intuitive for these terms. We can be straightforward for what we do by using RENAME keyword:

ALTER TABLE archive RENAME PARTITION p1 TO TABLE tp1;
ALTER TABLE archive RENAME TABLE tp1 TO PARTITION p1 VALUES LESS THAN (200);

I believe there is still no fast partition rename in MySQL/MariaDB, so we could make that as a bonus:

ALTER TABLE archive RENAME PARTITION p1 TO p11;

Btw, the last syntax is supported by Oracle.

Comment by Dmitry Shulga [ 2021-08-08 ]

I support the Aleksey's opinion that meaning of IMPORT/EXPORT implies keeping the original data.

On the other hand, I believe that the new syntax for adding a partition by virtue of moving a separate table to table's partition should be aligned with current syntax of ALTER TABLE in respect of adding partitions. Therefore from my point of view the following syntax would be more clear in expressing the idea of moving data from external table to table's partition:
ALTER TABLE ... ADD PARTITION (PARTITION ... ) MIGRATE FROM TABLE ...

For the sake of uniformity, the keyword MIGRATE could also be used in the ALTER statement for moving a partition to an external table:
ALTER TABLE ... MIGRATE PARTITION ... TO TABLE ...

Comment by Aleksey Midenkov [ 2021-08-10 ]

Using the MIGRATE keyword looks like a nice idea. Keeping ADD PARTITION syntax compatibility is something that I also thought of. The downside of that is kind of longer syntax: two sequential PARTITION keywords. I hope we can make PARTITION inside the brackets optional.

Comment by Federico Razzoli [ 2021-08-10 ]

What happens if the two tables are different in one of these aspects?

  • Indexes
  • Same visible indexes, but different IGNORED indexes
  • INVISIBLE columns
  • Compressed columns
  • CHECK
  • Type size (SMALLINT -> INT)
  • NULL / NOT NULL
  • Storage engine

Indexes differences are not unlikely, in my opinion.

What happens if the original table has triggers?

Comment by Sergei Golubchik [ 2021-08-13 ]

Somehow I don't see how the original

ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) FROM TABLE p1;

is worse or more confusing than

ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200)) MIGRATE FROM TABLE p1;

if anything, the former is shorter and only uses familiar keywords, it doesn't introduce a new keyword into the grammar

Comment by Sergei Golubchik [ 2021-08-17 ]

f_razzoli, this feature introduces a convenience command to replace the following sql script:

ALTER TABLE archive ADD PARTITION (PARTITION p1 VALUES LESS THAN (200));
ALTER TABLE archive EXCHANGE PARTITION p1 WITH TABLE p1;
DROP TABLE p1;

it does not allow to do anything that wasn't possible before and it should generally behave exactly as the script above. Also in regards to indexes, invisible columns, constraints, etc.

Comment by Aleksey Midenkov [ 2021-09-10 ]

Please review bb-10.7-midenok-MDEV-22165

Comment by Sergei Golubchik [ 2021-09-13 ]

pushed together with MDEV-22166

Generated at Thu Feb 08 09:12:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.