Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20608

innodb_log_optimize_ddl=OFF may omit some redo log

Details

    Description

      MDEV-16809 introduced the configuration parameter innodb_log_optimize_ddl (ON by default, to keep existing behaviour). The intention was that with the non-default setting innodb_log_optimize_ddl=OFF, all changes to persistent data pages would be redo-logged.

      Unfortunately, while the PageBulk code appears to write redo log for many operations, both in PageBulk::insert() and PageBulk::finish(), the modifications of the sparse page directory and the linked list of records are not being redo-logged.

      Furthermore, PageBulk::init() is bypassing some checks by setting a flag buf_block_t::skip_flush_check, which should not exist at all. There should be no special logic for flushing ‘incomplete’ pages, and all changes to persistent pages should be redo-logged if innodb_log_optimize_ddl=OFF.

      As far as I can tell, there should be no impact on InnoDB crash recovery due to this. The omitted redo logging should only affect scenarios where mariabackup --backup is run concurrently with a table-rebuilding or index-creating ALTER TABLE…ALGORITHM=INPLACE operation.

      Attachments

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-16809 Allow full redo logging for ALTER TABLE

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-12353 Efficient InnoDB redo log record format

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-19747 Deprecate and ignore innodb_log_optimize_ddl

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            marko Marko Mäkelä added a comment -

            The implementation of the new physical-only redo log record format in MDEV-12353 will have to address this. My current plan is to deprecate and ignore the parameter innodb_log_optimize_ddl, and to always write redo log for modifying persistent data pages.

            marko Marko Mäkelä added a comment - The implementation of the new physical-only redo log record format in MDEV-12353 will have to address this. My current plan is to deprecate and ignore the parameter innodb_log_optimize_ddl , and to always write redo log for modifying persistent data pages.
            marko Marko Mäkelä added a comment -

            It seems that PageBulk::insert() is covered by MLOG_REC_INSERT or MLOG_COMP_REC_INSERT or MLOG_ZIP_PAGE_COMPRESS, after all. The only doubt could be around buf_block_t::skip_flush_check.

            marko Marko Mäkelä added a comment - It seems that PageBulk::insert() is covered by MLOG_REC_INSERT or MLOG_COMP_REC_INSERT or MLOG_ZIP_PAGE_COMPRESS , after all. The only doubt could be around buf_block_t::skip_flush_check .
            marko Marko Mäkelä added a comment -

            Note: PageBulk::finish() is writing redo log for some operations but not all of them. Apparently, the low-level operations in PageBulk::insert() must not be redo-logged, because that would lead to failure when applying MLOG_REC_INSERT or MLOG_COMP_REC_INSERT records. I should have more information soon, after I have successfully replaced the MLOG_REC_DELETE and MLOG_REC_INSERT records in the MDEV-12353 branch.

            marko Marko Mäkelä added a comment - Note: PageBulk::finish() is writing redo log for some operations but not all of them. Apparently, the low-level operations in PageBulk::insert() must not be redo-logged, because that would lead to failure when applying MLOG_REC_INSERT or MLOG_COMP_REC_INSERT records. I should have more information soon, after I have successfully replaced the MLOG_REC_DELETE and MLOG_REC_INSERT records in the MDEV-12353 branch.
            marko Marko Mäkelä added a comment -

            Even after successfully replacing the MLOG_REC_INSERT and MLOG_COMP_REC_INSERT records in my MDEV-12353 prototype, I cannot say for sure if there is a problem with PageBulk::finish(). I think that the bug would have to be repeated manually. I think that something with

            ALTER TABLE t DROP INDEX b, ADD INDEX nb(b);
            		 
            ALTER TABLE t DROP INDEX nb, ADD INDEX b(b);

            while concurrently running the backup, and then executing CHECK TABLE on the copy from mariabackup --restore could do the trick. Again, this should affect innodb_log_optimize_ddl=OFF only, and probably not ROW_FORMAT=COMPRESSED tables.

            marko Marko Mäkelä added a comment - Even after successfully replacing the MLOG_REC_INSERT and MLOG_COMP_REC_INSERT records in my MDEV-12353 prototype, I cannot say for sure if there is a problem with PageBulk::finish() . I think that the bug would have to be repeated manually. I think that something with ALTER TABLE t DROP INDEX b, ADD INDEX nb(b); ALTER TABLE t DROP INDEX nb, ADD INDEX b(b); while concurrently running the backup, and then executing CHECK TABLE on the copy from mariabackup --restore could do the trick. Again, this should affect innodb_log_optimize_ddl=OFF only, and probably not ROW_FORMAT=COMPRESSED tables.
            marko Marko Mäkelä added a comment -

            MDEV-19747 (MariaDB Server 10.5.1) deprecates and ignores the parameter innodb_log_optimize_ddl. Given that 10.4 is close to its end of life, it is not feasible to spend time on this.

            marko Marko Mäkelä added a comment - MDEV-19747 (MariaDB Server 10.5.1) deprecates and ignores the parameter innodb_log_optimize_ddl . Given that 10.4 is close to its end of life, it is not feasible to spend time on this.

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.