Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11782

Redefine the innodb_encrypt_log format

Details

    • 10.2.4-1, 10.2.4-2

    Description

      WL#8845 introduced a redo log format identifier in MySQL 5.7.9.

      The MariaDB extension innodb_encrypt_log should distinguish encrypted redo logs from cleartext ones by the redo log format tag. In this way, an attempt to start up MySQL 5.7 after a crash of MariaDB with innodb_encrypt_log=1 will result in a clear error message rather than some strange-looking crash.

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MDEV-12041 Implement key rotation for innodb_encrypt_log

          • Major - Major loss of function.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13061 innodb_encrypt_log recovery is spamming the error log

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13834 10.2 wrongly recognizes 10.1.10 innodb_encrypt_log=ON data as after-crash and refuses to start

          • Major - Major loss of function.
          • Closed
          includes

          Task - A task that needs to be done. MDEV-12061 Allow innodb_log_files_in_group=1

          • Major - Major loss of function.
          • Closed
          is part of

          Bug - A problem which impairs or prevents the functions of the product. MDEV-11873 Unnecessary InnoDB warnings upon bootstrap

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-12103 Reduce the time of looking for MLOG_CHECKPOINT during crash recovery

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-12288 Reset DB_TRX_ID when the history is removed, to speed up MVCC

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit boundary

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-9011 Redo log encryption does not work

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-11432 Change the informational redo log format tag to say "MariaDB 10.2.3"

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13253 After rebuilding redo logs, InnoDB can leak data from redo log buffer

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13318 Crash recovery failure after the server is killed during innodb_encrypt_log startup

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 7 older comments
            marko Marko Mäkelä added a comment -

            bb-10.2-mdev-11782

            marko Marko Mäkelä added a comment - bb-10.2-mdev-11782
            marko Marko Mäkelä added a comment -

            Revised patch with less memcpy() and consistently using a 4-byte cleartext log block header: bb-10.2-mdev-11782

            marko Marko Mäkelä added a comment - Revised patch with less memcpy() and consistently using a 4-byte cleartext log block header: bb-10.2-mdev-11782
            jplindst Jan Lindström (Inactive) added a comment -

            ok to push after addressing review comments that are open.

            jplindst Jan Lindström (Inactive) added a comment - ok to push after addressing review comments that are open.
            marko Marko Mäkelä added a comment -

            Revised patch, addressing the review comments and test failures, and also with some changes to the startup logic: We must not generate redo log before rebuilding the redo logs.
            bb-10.2-marko (includes also MDEV-12061)

            marko Marko Mäkelä added a comment - Revised patch, addressing the review comments and test failures, and also with some changes to the startup logic: We must not generate redo log before rebuilding the redo logs. bb-10.2-marko (includes also MDEV-12061 )
            marko Marko Mäkelä added a comment -

            commit 2af28a363c0ac55c9b91aa9eb26949fc9ecf043a
            Author: Marko Mäkelä <marko.makela@mariadb.com>
            Date: Fri Feb 10 12:11:42 2017 +0200

            MDEV-11782: Redefine the innodb_encrypt_log format

            Write only one encryption key to the checkpoint page.
            Use 4 bytes of nonce. Encrypt more of each redo log block,
            only skipping the 4-byte field LOG_BLOCK_HDR_NO which the
            initialization vector is derived from.

            Issue notes, not warning messages for rewriting the redo log files.

            recv_recovery_from_checkpoint_finish(): Do not generate any redo log,
            because we must avoid that before rewriting the redo log files, or
            otherwise a crash during a redo log rewrite (removing or adding
            encryption) may end up making the database unrecoverable.
            Instead, do these tasks in innobase_start_or_create_for_mysql().

            Issue a firm "Missing MLOG_CHECKPOINT" error message. Remove some
            unreachable code and duplicated error messages for log corruption.

            LOG_HEADER_FORMAT_ENCRYPTED: A flag for identifying an encrypted redo
            log format.

            log_group_t::is_encrypted(), log_t::is_encrypted(): Determine
            if the redo log is in encrypted format.

            recv_find_max_checkpoint(): Interpret LOG_HEADER_FORMAT_ENCRYPTED.

            srv_prepare_to_delete_redo_log_files(): Display NOTE messages about
            adding or removing encryption. Do not issue warnings for redo log
            resizing any more.

            innobase_start_or_create_for_mysql(): Rebuild the redo logs also when
            the encryption changes.

            innodb_log_checksums_func_update(): Always use the CRC-32C checksum
            if innodb_encrypt_log. If needed, issue a warning
            that innodb_encrypt_log implies innodb_log_checksums.

            log_group_write_buf(): Compute the checksum on the encrypted
            block contents, so that transmission errors or incomplete blocks can be
            detected without decrypting.

            Rewrite most of the redo log encryption code. Only remember one
            encryption key at a time (but remember up to 5 when upgrading from the
            MariaDB 10.1 format.)

            marko Marko Mäkelä added a comment - commit 2af28a363c0ac55c9b91aa9eb26949fc9ecf043a Author: Marko Mäkelä <marko.makela@mariadb.com> Date: Fri Feb 10 12:11:42 2017 +0200 MDEV-11782 : Redefine the innodb_encrypt_log format Write only one encryption key to the checkpoint page. Use 4 bytes of nonce. Encrypt more of each redo log block, only skipping the 4-byte field LOG_BLOCK_HDR_NO which the initialization vector is derived from. Issue notes, not warning messages for rewriting the redo log files. recv_recovery_from_checkpoint_finish(): Do not generate any redo log, because we must avoid that before rewriting the redo log files, or otherwise a crash during a redo log rewrite (removing or adding encryption) may end up making the database unrecoverable. Instead, do these tasks in innobase_start_or_create_for_mysql(). Issue a firm "Missing MLOG_CHECKPOINT" error message. Remove some unreachable code and duplicated error messages for log corruption. LOG_HEADER_FORMAT_ENCRYPTED: A flag for identifying an encrypted redo log format. log_group_t::is_encrypted(), log_t::is_encrypted(): Determine if the redo log is in encrypted format. recv_find_max_checkpoint(): Interpret LOG_HEADER_FORMAT_ENCRYPTED. srv_prepare_to_delete_redo_log_files(): Display NOTE messages about adding or removing encryption. Do not issue warnings for redo log resizing any more. innobase_start_or_create_for_mysql(): Rebuild the redo logs also when the encryption changes. innodb_log_checksums_func_update(): Always use the CRC-32C checksum if innodb_encrypt_log. If needed, issue a warning that innodb_encrypt_log implies innodb_log_checksums. log_group_write_buf(): Compute the checksum on the encrypted block contents, so that transmission errors or incomplete blocks can be detected without decrypting. Rewrite most of the redo log encryption code. Only remember one encryption key at a time (but remember up to 5 when upgrading from the MariaDB 10.1 format.)

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.