Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-12041

Implement key rotation for innodb_encrypt_log

Details

    Description

      MDEV-9422 disabled the key rotation for the encrypted InnoDB redo log, because the used approach of writing multiple key metadata to the checkpoint page did not work.

      A better approach would seem to be the following:

      • In the redo log checkpoint page, write the current key version identifier.
      • When the redo log key is going to be rotated:
      1. Write a MLOG_KEY_ROTATE record with the information of the new (rotated) key
      2. Pad the end of the log block, and encrypt this block with the old key.
      3. Starting from the next log block, encrypt using the rotated key.

      This approach has the problem that writing log involves double buffering, and the log block boundaries are not known until the final copying, which happens under mutex protection. It would be simpler to do the following:

      • In each encrypted log block, write the encryption key version. (This would reduce the 512-byte logical block payload size from 496 to 492 bytes, or from 97% to 96%.)

      Attachments

        Issue Links

          is blocked by

          Task - A task that needs to be done. MDEV-11782 Redefine the innodb_encrypt_log format

          • Major - Major loss of function.
          • Closed
          is caused by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-9422 Encrypted redo log checksum errors on restart after killing busy XtraDB instance

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13044 Do not create a redo log checkpoint at startup

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-13318 Crash recovery failure after the server is killed during innodb_encrypt_log startup

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MDEV-14425 Change the InnoDB redo log format to reduce write amplification

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-21949 key rotation for innodb_encrypt_log is not working in 10.5

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            marko Marko Mäkelä created issue -
            marko Marko Mäkelä made changes -
            Field Original Value New Value
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä added a comment -

            As a part of this work, InnoDB should periodically change the random encryption parameters that are initialized in log_crypt_init(). These parameters could be changed much more frequently than the encryption key. Maybe we should assign and store a random nonce in each redo log block? Maybe the logical redo log block size should be larger than 512 bytes?

            Starting with MDEV-13318, log_crypt_init() will only be invoked when rebuilding the redo logs.

            marko Marko Mäkelä added a comment - As a part of this work, InnoDB should periodically change the random encryption parameters that are initialized in log_crypt_init(). These parameters could be changed much more frequently than the encryption key. Maybe we should assign and store a random nonce in each redo log block? Maybe the logical redo log block size should be larger than 512 bytes? Starting with MDEV-13318 , log_crypt_init() will only be invoked when rebuilding the redo logs.
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä added a comment -

            This could be implemented as part of MDEV-14425 by writing an encryption key ID in each redo log block.

            marko Marko Mäkelä added a comment - This could be implemented as part of MDEV-14425 by writing an encryption key ID in each redo log block.
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.3 [ 22126 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Fix Version/s 10.4 [ 22408 ]
            ralf.gebhardt Ralf Gebhardt made changes -
            Epic Link PT-73 [ 68549 ]
            marko Marko Mäkelä made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            marko Marko Mäkelä made changes -
            Description MDEV-9422 disabled the key rotation for the encrypted InnoDB redo log, because the used approach of writing multiple key metadata to the checkpoint page did not work.

            A better approach would seem to be the following:
            * In the redo log checkpoint page, write the current key version identifier.
            * When the redo log key is going to be rotated:
            # Write a MLOG_KEY_ROTATE record with the information of the new (rotated) key
            # Pad the end of the log block, and encrypt this block with the old key.
            # Starting from the next log block, encrypt using the rotated key.             		MDEV-9422 disabled the key rotation for the encrypted InnoDB redo log, because the used approach of writing multiple key metadata to the checkpoint page did not work.

            A better approach would seem to be the following:
            * In the redo log checkpoint page, write the current key version identifier.
            * When the redo log key is going to be rotated:
            # Write a {{MLOG_KEY_ROTATE}} record with the information of the new (rotated) key
            # Pad the end of the log block, and encrypt this block with the old key.
            # Starting from the next log block, encrypt using the rotated key.

            This approach has the problem that writing log involves double buffering, and the log block boundaries are not known until the final copying, which happens under mutex protection. It would be simpler to do the following:
            * In each encrypted log block, write the encryption key version. (This would reduce the 512-byte logical block payload size from 496 to 492 bytes, or from 97% to 96%.)
            marko Marko Mäkelä added a comment -

            Starting with MariaDB 10.4.0, the encrypted InnoDB redo log format will reserve 4 bytes at the end of each 512-byte redo log block for the encryption key version. This allows key rotation at any time. For performance reasons, we will attempt key rotation only when writing log as part of a redo log checkpoint.

            marko Marko Mäkelä added a comment - Starting with MariaDB 10.4.0, the encrypted InnoDB redo log format will reserve 4 bytes at the end of each 512-byte redo log block for the encryption key version. This allows key rotation at any time. For performance reasons, we will attempt key rotation only when writing log as part of a redo log checkpoint.
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2018-08-13 13:09:52.0 2018-08-13 13:09:52.645
            marko Marko Mäkelä made changes -
            Fix Version/s 10.4.0 [ 23115 ]
            Fix Version/s 10.4 [ 22408 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            Labels ServiceNow
            rob.schwyzer@mariadb.com Rob Schwyzer (Inactive) made changes -
            Labels ServiceNow 76qDvLB8Gju6Hs7nk3VY3EX42G795W5z
            serg Sergei Golubchik made changes -
            Labels 76qDvLB8Gju6Hs7nk3VY3EX42G795W5z
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 79607 ] MariaDB v4 [ 133126 ]
            mariadb-jira-automation Jira Automation (IT) made changes -
            Zendesk Related Tickets 201658 137277
            Zendesk active tickets 201658

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.