Details
-
New Feature
-
Status: In Review (View Workflow)
-
Critical
-
Resolution: Unresolved
-
None
Description
Atomic DDL for partitioning operations covers crash-safety but it does not cover failures. F.ex. if error happens during the rename or drop of partitions the alter operation does not return the table to its original state. More details what states are possible after failed alter operation are described in handle_alter_part_error(). Let's take a couple of exerpts from it:
- Manual drop of remaining partitions is required
- The frm file is in an unknown state, and a backup is required.
Implementation of full atomicity for partitioning DDL takes into consideration the following modifications:
1. For partition drop when it is already in storage engine use DDL_LOG_DROP_TABLE_ACTION instead of DDL_LOG_DELETE_ACTION.
2. frm_action flag based on handler_name equal to reg_ext should not be used. Instead ddl_log_entry->name should contain filename with extension. So if .par file delete required we use second DDL_LOG_DELETE_ACTION.
That is: DDL_LOG_DELETE_ACTION is used only for file delete (maybe rename to DDL_LOG_RM_ACTION). For table drop DDL_LOG_DROP_TABLE_ACTION must be used.
3. Similarly, use DDL_LOG_RENAME_TABLE_ACTION instead of DDL_LOG_RENAME_ACTION. For partition rename update straight to DDL_RENAME_PHASE_TABLE because no triggers and stat processing is needed.
For DDL_LOG_REPLACE_ACTION log DDL_LOG_DROP_TABLE_ACTION first and then DDL_LOG_RENAME_TABLE_ACTION.
That is: DDL_LOG_RENAME_ACTION is used only for file rename. DDL_LOG_REPLACE_ACTION is deprecated. For file replace log DDL_LOG_RM_ACTION first.
4. mysql_drop_partitions(), mysql_rename_partitions() are not needed. This can be done via log replay. This refactoring deprecates partition_element::log_entry data.
5. Log replay must handle errors and should be crash-safe and atomic itself. That brings a couple of new modes for ddl_log_revert(): DDL_LOG_ERR_REPORT, DDL_LOG_ERR_ROLLBACK. In DDL_LOG_ERR_REPORT mode it fails log replay at the first encountered error and prints the error message. In DDL_LOG_ERR_ROLLBACK mode it also reverts back all the revertible changes it done before the error happens.
That is: DDL_LOG_ERR_ROLLBACK means DDL log replay writes new DDL log chain which covers the atomicity during the replay.
6. Implement the rest of MDEV-22166 minor refactorings (see todo.txt).
P. 2. adds new feature of removing arbitrary files via DDL log (and not just .frm and .par files). This feature is required for MDEV-16417 atomicity. It was already implemented there as a quick hack, now is the time for unified solution.
Attachments
Issue Links
- blocks
-
MDEV-16417 Store Foreign Key metadata outside of InnoDB
- In Review
-
MDEV-29558 Crash recovery: SHOW KEYS .. harvests 1696: Failed to read from the .par file
- Open
-
MDEV-31212 Range partition: allow ADD PARTITION into the middle of ranges list
- Open
-
MDEV-31213 CONVERT TABLE TO PARTITION: allow copy rows to avoid partition restrictions
- Open
- is blocked by
-
MDEV-25292 Atomic CREATE OR REPLACE TABLE
- Stalled
- is duplicated by
-
MDEV-27618 Atomic DDL is not very atomic on partitioned tables
- Confirmed
- relates to
-
MDEV-22165 CONVERT TABLE: move in partition from existing table
- Closed
-
MDEV-22166 CONVERT PARTITION: move out partition into a table
- Closed
-
MDEV-27618 Atomic DDL is not very atomic on partitioned tables
- Confirmed
-
MDEV-29566 Failed to read from the .par file upon concurrent DDL/SELECT with partition pruning
- Stalled
-
MDEV-22168 Supporting multiple engines with table partitioning
- Stalled
-
MDEV-28801 Partitioning CONVERT: implement data copy
- Open
-
MDEV-29558 Crash recovery: SHOW KEYS .. harvests 1696: Failed to read from the .par file
- Open