[MDEV-20608] innodb_log_optimize_ddl=OFF may omit some redo log Created: 2019-09-17  Updated: 2023-04-27

Status: Open
Project: MariaDB Server
Component/s: mariabackup, Storage Engine - InnoDB
Affects Version/s: 10.2.17, 10.3.9, 10.4.0
Fix Version/s: 10.4

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Unresolved Votes: 1
Labels: backup

Issue Links:
Problem/Incident
is caused by MDEV-16809 Allow full redo logging for ALTER TABLE Closed
Relates
relates to MDEV-12353 Efficient InnoDB redo log record format Closed

 Description   

MDEV-16809 introduced the configuration parameter innodb_log_optimize_ddl (ON by default, to keep existing behaviour). The intention was that with the non-default setting innodb_log_optimize_ddl=OFF, all changes to persistent data pages would be redo-logged.

Unfortunately, while the PageBulk code appears to write redo log for many operations, both in PageBulk::insert() and PageBulk::finish(), the modifications of the sparse page directory and the linked list of records are not being redo-logged.

Furthermore, PageBulk::init() is bypassing some checks by setting a flag buf_block_t::skip_flush_check, which should not exist at all. There should be no special logic for flushing ‘incomplete’ pages, and all changes to persistent pages should be redo-logged if innodb_log_optimize_ddl=OFF.

As far as I can tell, there should be no impact on InnoDB crash recovery due to this. The omitted redo logging should only affect scenarios where mariabackup --backup is run concurrently with a table-rebuilding or index-creating ALTER TABLE…ALGORITHM=INPLACE operation.

 Comments   
Comment by Marko Mäkelä [ 2019-09-17 ]

The implementation of the new physical-only redo log record format in MDEV-12353 will have to address this. My current plan is to deprecate and ignore the parameter innodb_log_optimize_ddl, and to always write redo log for modifying persistent data pages.

Comment by Marko Mäkelä [ 2019-09-17 ]

It seems that PageBulk::insert() is covered by MLOG_REC_INSERT or MLOG_COMP_REC_INSERT or MLOG_ZIP_PAGE_COMPRESS, after all. The only doubt could be around buf_block_t::skip_flush_check.

Comment by Marko Mäkelä [ 2019-09-17 ]

Note: PageBulk::finish() is writing redo log for some operations but not all of them. Apparently, the low-level operations in PageBulk::insert() must not be redo-logged, because that would lead to failure when applying MLOG_REC_INSERT or MLOG_COMP_REC_INSERT records. I should have more information soon, after I have successfully replaced the MLOG_REC_DELETE and MLOG_REC_INSERT records in the MDEV-12353 branch.

Comment by Marko Mäkelä [ 2019-09-19 ]

Even after successfully replacing the MLOG_REC_INSERT and MLOG_COMP_REC_INSERT records in my MDEV-12353 prototype, I cannot say for sure if there is a problem with PageBulk::finish(). I think that the bug would have to be repeated manually. I think that something with

ALTER TABLE t DROP INDEX b, ADD INDEX nb(b);
 
ALTER TABLE t DROP INDEX nb, ADD INDEX b(b);

while concurrently running the backup, and then executing CHECK TABLE on the copy from mariabackup --restore could do the trick. Again, this should affect innodb_log_optimize_ddl=OFF only, and probably not ROW_FORMAT=COMPRESSED tables.

Generated at Thu Feb 08 09:00:47 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.