Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10368

get_latest_version() called too often

Details

    • 10.1.18, 10.2.4-1

    Description

      Data at rest encryption. MariaDB calls the get_latest_version() of the *_key_management plugins millions of times if innodb_encryption_threads>1 during data scrub.

      Tested with file_key_management plugin, but it should not matter.

      The issue can be seen at each background data scrub, so for testing purposes I have set the interval to 60 seconds:
      innodb-background-scrub-data-check-interval=60

      When using innodb_encryption_threads>1 the function get_latest_version() of the plugin is called millions of times, also cpu load raises considerably.

      Using innodb_encryption_threads=1 there is no such issue.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-14398 When innodb_encryption_rotate_key_age=0 is set, server won't encrypt tablespaces

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            jplindst Jan Lindström (Inactive) added a comment -

            http://lists.askmonty.org/pipermail/commits/2016-December/010241.html

            commit 2093b7dd9807506b6f83af0969b22f27ab6c034e
            Author: Jan Lindström <jan.lindstrom@mariadb.com>
            Date: Thu Dec 8 10:24:30 2016 +0200

            MDEV-10368: get_latest_version() called too often

            Reduce the number of calls to encryption_get_key_get_latest_version
            when doing key rotation with two different methods:

            (1) We need to fetch key information when tablespace not yet
            have a encryption information, invalid keys are handled now
            differently (see below). There was extra call to detect
            if key_id is not found on key rotation.

            (2) If key_id is not found from encryption plugin, do not
            try fetching new key_version for it as it will fail anyway.
            We store return value from encryption_get_key_get_latest_version
            call and if it returns ENCRYPTION_KEY_VERSION_INVALID there
            is no need to call it again.

            jplindst Jan Lindström (Inactive) added a comment - http://lists.askmonty.org/pipermail/commits/2016-December/010241.html commit 2093b7dd9807506b6f83af0969b22f27ab6c034e Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Thu Dec 8 10:24:30 2016 +0200 MDEV-10368 : get_latest_version() called too often Reduce the number of calls to encryption_get_key_get_latest_version when doing key rotation with two different methods: (1) We need to fetch key information when tablespace not yet have a encryption information, invalid keys are handled now differently (see below). There was extra call to detect if key_id is not found on key rotation. (2) If key_id is not found from encryption plugin, do not try fetching new key_version for it as it will fail anyway. We store return value from encryption_get_key_get_latest_version call and if it returns ENCRYPTION_KEY_VERSION_INVALID there is no need to call it again.
            marko Marko Mäkelä added a comment -

            I replied to the commit email, requesting some changes mainly related to the monitoring interfaces.

            marko Marko Mäkelä added a comment - I replied to the commit email, requesting some changes mainly related to the monitoring interfaces.
            jplindst Jan Lindström (Inactive) added a comment -

            http://lists.askmonty.org/pipermail/commits/2016-December/010263.html

            I did not do full refactoring as this is GA release and in my opinion we should avoid
            large changes there.

            jplindst Jan Lindström (Inactive) added a comment - http://lists.askmonty.org/pipermail/commits/2016-December/010263.html I did not do full refactoring as this is GA release and in my opinion we should avoid large changes there.
            marko Marko Mäkelä added a comment -

            I would prefer some minor changes to the patch. See my reply on the commits list.

            marko Marko Mäkelä added a comment - I would prefer some minor changes to the patch. See my reply on the commits list.
            jplindst Jan Lindström (Inactive) added a comment -

            commit 72cc73cea2e7071277b6a88bae0236cabf11788a
            Author: Jan Lindström <jan.lindstrom@mariadb.com>
            Date: Tue Dec 13 11:51:33 2016 +0200

            MDEV-10368: get_latest_version() called too often

            Reduce the number of calls to encryption_get_key_get_latest_version
            when doing key rotation with two different methods:

            (1) We need to fetch key information when tablespace not yet
            have a encryption information, invalid keys are handled now
            differently (see below). There was extra call to detect
            if key_id is not found on key rotation.

            (2) If key_id is not found from encryption plugin, do not
            try fetching new key_version for it as it will fail anyway.
            We store return value from encryption_get_key_get_latest_version
            call and if it returns ENCRYPTION_KEY_VERSION_INVALID there
            is no need to call it again.

            jplindst Jan Lindström (Inactive) added a comment - commit 72cc73cea2e7071277b6a88bae0236cabf11788a Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Tue Dec 13 11:51:33 2016 +0200 MDEV-10368 : get_latest_version() called too often Reduce the number of calls to encryption_get_key_get_latest_version when doing key rotation with two different methods: (1) We need to fetch key information when tablespace not yet have a encryption information, invalid keys are handled now differently (see below). There was extra call to detect if key_id is not found on key rotation. (2) If key_id is not found from encryption plugin, do not try fetching new key_version for it as it will fail anyway. We store return value from encryption_get_key_get_latest_version call and if it returns ENCRYPTION_KEY_VERSION_INVALID there is no need to call it again.

            People

              jplindst Jan Lindström (Inactive)
              claudio.nanni Claudio Nanni
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.