Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
-
10.2(EOL), 10.3(EOL), 10.4(EOL), 10.5, 10.6, 10.11, 11.4
Description
In mariadb-backup --backup there are multiple mechanisms for ensuring that a sufficient amount of the InnoDB write-ahead log (ib_logfile0) is being copied at the end of the backup.
There is a function backup_wait_for_lsn(), which will wait for the log_copying_thread() to copy a necessary amount. But, we fail to pass the target LSN to log_copying_thread() itself.
The log copying and recovery logic was rewritten in MDEV-14425 and some locking in backup was rewritten in MDEV-32932. Therefore, it is not trivial to apply this fix to earlier versions.
Attachments
Issue Links
- relates to
-
-
- Open
-
-
MDEV-14425 Change the InnoDB redo log format to reduce write amplification
-
- Closed
-
-
MDEV-32932 port backup features from MariaDB Enterprise
-
- Closed
-
Activity
Thank you for the review. Based on MDEV-5336, part of my attempted logic change seems to be incorrect, and we really have to copy the ib_logfile0 until the LSN as it is right after BACKUP STAGE BLOCK_COMMIT.
I have revised this. The logic is now somewhat clearer at least to me. All InnoDB log copying would now take place in the log_copying_thread.
One surprise to me was that executing any statement (such as SHOW ENGINE INNODB STATUS) could cause a write to the table mysql.general_log. Therefore, we must be sure to copy the file mysql/general_log.CSM before trying to determine the InnoDB log sequence number.
Thanks marko , the patch looks functionally correct to me and the refactoring is also good. I have some comments for you to consider at code level.
I still don't completely understand the requirement that we need to copy less redo logs and stay behind. Perhaps the backup could perform faster but user could pay bigger penalty in replication setup. I have tried to explain my P.O.V. in one of the comments.
I think that the point is that BACKUP STAGE BLOCK_COMMIT allows active transactions to modify InnoDB tables. Copying log records for incomplete transactions would serve no useful purpose. In fact, it would only create extra work when starting up a server on a restored backup, because then the incomplete transactions would have executed more changes that would need to be rolled back.
The server could also be busy purging the history of committed InnoDB transactions. Backup should not really care whether such modifications are included. Copying the data files would already have finished at that point, and therefore copying less log will make both backup and applying the log finish faster. The drawback might be that starting a server on the restored backup would potentially have to spend more time purging some history.
marko Sorry, I delayed review on the patch for long. After checking the details I couldn't agree with the patch and I think the current design is safer and better. I have detailed comments in pull-3370.
Also, I would strongly recommend such improvement / behaviour change be attempted as a non-functional feature improvement (perhaps only in latest version).