Details
-
Task
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
None
Description
The purpose of this task is to ensure that the following internal operations
are atomic (either all or nothing) for all storage engines:
Manage .frm
Storage engine data dictionary
Binary log
This will solve the following things:
- .frm context and storage engine dictionary is always in sync
- no #sql-xxxx files if server crashes during alter table
- binary log will always contain the DDL statement if the DDL successfully and committed to the storage engine. If the DDL
was not successful, all traces of the DDL (like temporary files) will be deleted/rolled back.
For example, if an alter table was successfully done and committed and we get a crash just before the write to the binary
log, crash recovery will write the ALTER TABLE command to the binary log. In the case of a crash in the middle of a multi-table drop, crash recovery will write to the binary log those tables that was actually dropped.
The cost of doing the above should not be more than 1-4 sync per DDL.
High level architecture
When doing a DDL, store somewhere (either trough write_ddl_log_entry() or
some new method):
- Operation
- Number of tables
- Table name
- Table id for original table
- Table id for resulting table (in case of rename)
- sql command (for binary log)
If there is no table id (for example for CSV) we would use the
timestamp of the files.
With the above information we would be able to continue from the place
where the operation failed.
Low level architecture is done for each sub project
Some requirements for a storage engine to be 'Atomic compliment':
- drop table and rename_table needs to be either atomic or can be retried if there was a crash in middle of the
operation. - Engines supporting inplace alter table, must also support the handlerton->check_version() call to allow the
ddl recovery code to check if the inline alter table succeed. This is only needed if there was a crash between
the inplace alter table commit and the rename of the .frm file.
Supported engines in 10.6 are (among others)
- InnoDB
- MyRocks
- Aria (transactional and non transactional tables)
- MyISAM
- Any engine that has only one table file (as then drop and rename will be atomic)
Attachments
Issue Links
- blocks
-
MDEV-4259 transactional DDL
-
- Open
-
-
MDEV-11633 Make the InnoDB system tablespace optional
-
- Open
-
-
MDEV-11655 Transactional data dictionary
-
- Open
-
-
MDEV-15140 Implement Partial / Filtered Indexes
-
- Open
-
-
MDEV-16417 Store Foreign Key metadata outside of InnoDB
-
- In Review
-
-
MDEV-21652 FK migration from old version
-
- In Review
-
-
MDEV-22165 CONVERT TABLE: move in partition from existing table
-
- Closed
-
-
MDEV-22166 CONVERT PARTITION: move out partition into a table
-
- Closed
-
-
MDEV-23570 deprecate keep_files_on_create
-
- Closed
-
- causes
-
MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load index but the index tree has been freed
-
- Closed
-
- includes
-
MDEV-23842 Atomic RENAME TABLE
-
- Closed
-
-
MDEV-23844 Atomic DROP TABLE (single table)
-
- Closed
-
-
MDEV-24395 Atomic DROP TRIGGER
-
- Closed
-
-
MDEV-24408 Crash-safe DROP DATABASE
-
- Closed
-
-
MDEV-24576 Atomic CREATE TABLE
-
- Closed
-
-
MDEV-24607 ATOMIC CREATE VIEW
-
- Closed
-
-
MDEV-24746 Atomic CREATE TRIGGER
-
- Closed
-
-
MDEV-25180 Atomic ALTER TABLE
-
- Closed
-
- is blocked by
-
MDEV-24589 DROP TABLE is not crash-safe
-
- Closed
-
- relates to
-
MDEV-18518 Implement atomic multi-table (or multi-partition) CREATE TABLE for InnoDB
-
- Closed
-
-
MDEV-22928 SUMMARY: AddressSanitizer: SEGV storage/innobase/include/dict0mem.h:1178 in dict_index_t::is_spatial() const
-
- Closed
-
-
MDEV-23741 InnoDB: Table ... contains <n> indexes inside InnoDB, which is different from the number of indexes <n> defined in the MariaDB
-
- Closed
-
-
MDEV-24755 Rebase FK tasks on top of Atomic DDL
-
- Closed
-
-
MDEV-25588 Atomic DDL: Binlog query event written upon recovery is corrupt
-
- Closed
-
-
MDEV-25606 Concurrent CREATE TRIGGER statements mix up in binlog and break replication
-
- Closed
-
-
MDEV-25655 Atomic DDL: InnoDB: Cannot replay rename of tablespace, error Data structure corruption
-
- Closed
-
-
MDEV-25656 Atomic DDL: Assertion `!full_crc32(flags)' failed in fil_space_t::logical_size(ulint)
-
- Closed
-
-
MDEV-25657 Atomic DDL: InnoDB: Cannot apply log, double free or corruption (fasttop)
-
- Closed
-
-
MDEV-25662 Atomic DDL: InnoDB: Expected tablespace id ... but found 0 upon crash recovery
-
- Closed
-
-
MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load index but the index tree has been freed
-
- Closed
-
-
MDEV-25705 Atomic DDL: atomic.rename_case and atomic.drop_db_long_names fail on Windows
-
- Open
-
-
MDEV-25920 Atomic DROP DATABASE
-
- Open
-
-
MDEV-26966 The parameter innodb_force_load_corrupted makes no sense
-
- Closed
-
-
MDEV-29566 Failed to read from the .par file upon concurrent DDL/SELECT with partition pruning
-
- Stalled
-
-
MDEV-6642 Server crashed with assertion failure in file ha_innodb.cc line 8619
-
- Open
-
-
MDEV-11675 Lag Free Alter On Slave
-
- Closed
-
-
MDEV-16290 ALTER TABLE ... RENAME COLUMN syntax
-
- Closed
-
-
MDEV-21053 Crash safety of foreign key DDL
-
- In Progress
-
-
MDEV-22056 Rocks db corrupts data when disk is out of space
-
- Open
-
-
MDEV-22165 CONVERT TABLE: move in partition from existing table
-
- Closed
-
-
MDEV-22166 CONVERT PARTITION: move out partition into a table
-
- Closed
-
-
MDEV-25604 Atomic DDL: Binlog event written upon recovery does not have default database
-
- Closed
-
-
MDEV-25646 Atomic DDL: InnoDB: Datafile './test/<tablename>.ibd' is corrupted. Cannot determine the space ID from the first 64 pages
-
- Closed
-
-
MDEV-25666 Atomic DDL: InnoDB: Operating system error number 2, Could not find a valid tablespace file
-
- Closed
-