Details
-
Bug
-
Status: In Progress (View Workflow)
-
Critical
-
Resolution: Unresolved
-
10.11, 11.4, 11.8, 12.3, 13.1
-
Can result in data loss
Description
During the testing of MDEV-14992, saahil found out that a CREATE after a RENAME is being recovered incorrectly. I can reproduce this with a patch to the test backup.backup_innodb that starts as follows:
diff --git a/mysql-test/suite/backup/backup_innodb.test b/mysql-test/suite/backup/backup_innodb.test
|
index cb1ada74985..13385d4abee 100644
|
--- a/mysql-test/suite/backup/backup_innodb.test
|
+++ b/mysql-test/suite/backup/backup_innodb.test
|
@@ -68,6 +68,7 @@ SET DEBUG_SYNC='now WAIT_FOR start';
|
}
|
--connect rename,localhost,root
|
RENAME TABLE rt TO rtt;
|
+CREATE TABLE rt(a INT PRIMARY KEY) ENGINE=InnoDB;
|
DROP TABLE dt;
|
--disconnect rename
|
--connection default |
data.tar.xz
is a 10.11 data directory that I created from a reduced version of the test. Unfortunately, that data directory is recovering fine whenever I try it.
The data directory contains the file rt.ibd that is expected to be renamed to rtt.ibd. What happens during crash recovery is that some data for the new file rt.ibd will overwrite the old file before recv_rename_files() is executed:
#0 fil_node_open_file_low
|
#1 fil_node_open_file
|
#2 fil_space_t::prepare_acquired
|
#3 fil_space_t::acquire_and_prepare
|
#4 fil_space_t::acquire
|
#5 recv_sys_t::recover_deferred (this=<recv_sys>,
|
p={first = {m_id = 0x700000000}, second = {being_processed = {m = std::atomic<signed char> = { 0xff }}, skip_read = 0x1, last_offset = 0x0, log = {head = 0x77f40c01bc78, tail = 0x77f40c01c5c8}}},
|
name, free_block=0x0)
|
#6 recv_sys_t::apply_batch (...)
|
#7 recv_sys_t::apply (this=<recv_sys>, last_batch=false)
|
#8 recv_sys_t::parse_tail<false, (recv_sys_t::store)2>
|
#9 recv_sys_t::parse<recv_buf, (recv_sys_t::store)2, 1349024115u>
|
#10 recv_sys_t::parse_mtr<(recv_sys_t::store)2, 1349024115u> (if_exists=true)
|
#11 recv_sys_t::parse_mmap<(recv_sys_t::store)2, 1349024115u> (if_exists=true)
|
#12 recv_scan_archive_store (parser=<recv_sys_t::parse_mmap<(recv_sys_t::store)2, 1349024115u>(bool)>, last_phase=true)
|
#13 recv_scan_log (last_phase=true, parser)
|
#14 recv_recovery_from_checkpoint_start ()
|
#15 srv_start (create_new_db=false)
|
Yes, this stack trace is specific to the innodb_log_archive=ON recovery. Here, we would wrongly open the old file rt.ibd for writing page (7,0) to it. In recv_rename_files() some time later, after the write was done corrupting the file, recovery would report the corruption but carry on with the startup:
2026-09-25 17:24:10 0 [Note] InnoDB: innodb_buffer_pool_size_max=8388608m, innodb_buffer_pool_size=8m
|
2026-09-25 17:24:10 0 [Note] InnoDB: Initialized memory pressure event listener
|
2026-09-25 17:24:10 0 [Note] InnoDB: Completed initialization of buffer pool
|
2026-09-25 17:24:10 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=55932
|
2026-09-25 17:24:10 0 [Note] InnoDB: Ignoring data file './test/rtt.ibd' with space ID 5. Another data file called ./test/rt.ibd exists with the same space ID.
|
2026-09-25 17:24:10 0 [Note] InnoDB: Multi-batch recovery needed at LSN 2736194
|
2026-09-25 17:24:10 0 [Note] InnoDB: End of log at LSN=3672966
|
2026-09-25 17:24:10 0 [Note] InnoDB: To recover: LSN 2736194/3672966; 303 pages
|
2026-09-25 17:24:10 0 [ERROR] InnoDB: Expected tablespace id 5 but found 7 in the file ./test/rt.ibd
|
2026-09-25 17:24:10 0 [Note] InnoDB: To recover: 106 pages
|
2026-09-25 17:24:10 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 10000 row operations to undo
|
2026-09-25 17:24:10 0 [Note] InnoDB: Trx id counter is 33
|
2026-09-25 17:24:10 0 [Note] InnoDB: 128 rollback segments are active.
|
2026-09-25 17:24:10 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
|
2026-09-25 17:24:10 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
|
2026-09-25 17:24:10 0 [Note] InnoDB: log sequence number 3672966 (memory-mapped); transaction id 33
|
After some hours of debugging, I concluded that the appearance of the corruption is dependent on some recovery parameters. Certainly, multi-batch recovery is needed. I tried different innodb_buffer_pool_size (6M, 8M, 16M), but could not reproduce the failure on 10.11 yet.
To prevent this corruption, I think that we must apply FILE_RENAME records after reading all FILE_ records, before invoking any recv_sys_t::recover_deferred().
Attachments
Issue Links
- blocks
-
MDEV-14992 BACKUP SERVER to mounted file system
-
- In Progress
-
