[MDEV-18518] Implement atomic multi-table (or multi-partition) CREATE TABLE for InnoDB Created: 2019-02-08 Updated: 2022-04-20 Resolved: 2021-05-06 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Fix Version/s: | 10.6.1 |
| Type: | Task | Priority: | Blocker |
| Reporter: | Marko Mäkelä | Assignee: | Marko Mäkelä |
| Resolution: | Fixed | Votes: | 2 |
| Labels: | ddl, performance | ||
| Attachments: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
The way how InnoDB writes undo log about CREATE TABLE and DROP TABLE introduces the limitation that only one table may be created or dropped within a transaction. We should replace the current mechanism (around trx->table_id) and introduce new undo log records for the following:
Applying either operation must write MLOG_FILE_DELETE2 to the redo log before actually deleting any files. To avoid name clashes between DROP TABLE and subsequent CREATE TABLE by the same name (before the old files were purged), we should probably rename the files to temporary names before dropping. Also the rename operation must be undo logged and redo logged. Furthermore, during normal operation we could delete the files for the dropped table. The deletion by the purge subsystem would only be a fall-back measure for crash recovery. |
| Comments |
| Comment by Marko Mäkelä [ 2019-05-20 ] | ||||||||||||||||||||||||||||||||||||||
|
Remove-table_id.patch | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2020-07-13 ] | ||||||||||||||||||||||||||||||||||||||
|
Given our chosen simpler solution to | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-01-14 ] | ||||||||||||||||||||||||||||||||||||||
|
In | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-03 ] | ||||||||||||||||||||||||||||||||||||||
|
This turned out to be simpler than I originally thought. No new undo log record types are needed; we can simply extend the | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-04 ] | ||||||||||||||||||||||||||||||||||||||
|
No file format changes were needed for this. The only difference to earlier is that the undo log header field TRX_UNDO_TABLE_ID will always be written as 0, and the special recovery fts_drop_orphaned_tables() was removed altogether, because all internal tables for FULLTEXT INDEX should now be created in the same DDL transaction with everything else in the ALTER TABLE operation. | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-05 ] | ||||||||||||||||||||||||||||||||||||||
|
It turns out that many operations involving FULLTEXT INDEX still are not atomic. | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-05 ] | ||||||||||||||||||||||||||||||||||||||
|
I am raising this to Blocker status, because currently the feature is half finished. | ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-06 ] | ||||||||||||||||||||||||||||||||||||||
|
The following test currently leaves behind orphan FTS_ tables for the table t1:
| ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-06 ] | ||||||||||||||||||||||||||||||||||||||
|
The above test will need some improvement for the regression suite:
| ||||||||||||||||||||||||||||||||||||||
| Comment by Marko Mäkelä [ 2021-05-06 ] | ||||||||||||||||||||||||||||||||||||||
|
The ALTER TABLE operations involving FULLTEXT INDEX are now atomic inside InnoDB. The test innodb_fts.crash_recovery covers that. The innodb_fts.crash_recovery,release variant (covering a server built using cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo) would trigger many different scenarios that are expected to be fixed by The test variant innodb_fts.crash_recovery,debug kills the server during a deterministic spot of 3 different ALTER TABLE, to exercise the revised recovery that would drop any internal FTS_%INDEX_% tables along with stubs created during prepare_inplace_alter_table_dict(). |