[MDEV-10368] get_latest_version() called too often - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.1.14
Fix Version/s: 10.1.20
Component/s: Encryption, Storage Engine - InnoDB, Storage Engine - XtraDB
Labels:
Environment:
linux

Sprint:
10.1.18, 10.2.4-1

Description

Data at rest encryption. MariaDB calls the get_latest_version() of the *_key_management plugins millions of times if innodb_encryption_threads>1 during data scrub.

Tested with file_key_management plugin, but it should not matter.

The issue can be seen at each background data scrub, so for testing purposes I have set the interval to 60 seconds:
innodb-background-scrub-data-check-interval=60

When using innodb_encryption_threads>1 the function get_latest_version() of the plugin is called millions of times, also cpu load raises considerably.

Using innodb_encryption_threads=1 there is no such issue.

Attachments

Issue Links

relates to

MDEV-14398 When innodb_encryption_rotate_key_age=0 is set, server won't encrypt tablespaces

Closed

Activity

Ascending order - Click to sort in descending order

Jan Lindström (Inactive) added a comment - 2016-12-08 08:33

http://lists.askmonty.org/pipermail/commits/2016-December/010241.html

commit 2093b7dd9807506b6f83af0969b22f27ab6c034e
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Thu Dec 8 10:24:30 2016 +0200

~~MDEV-10368~~: get_latest_version() called too often

Reduce the number of calls to encryption_get_key_get_latest_version
when doing key rotation with two different methods:

(1) We need to fetch key information when tablespace not yet
have a encryption information, invalid keys are handled now
differently (see below). There was extra call to detect
if key_id is not found on key rotation.

(2) If key_id is not found from encryption plugin, do not
try fetching new key_version for it as it will fail anyway.
We store return value from encryption_get_key_get_latest_version
call and if it returns ENCRYPTION_KEY_VERSION_INVALID there
is no need to call it again.

Jan Lindström (Inactive) added a comment - 2016-12-08 08:33 http://lists.askmonty.org/pipermail/commits/2016-December/010241.html commit 2093b7dd9807506b6f83af0969b22f27ab6c034e Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Thu Dec 8 10:24:30 2016 +0200 MDEV-10368 : get_latest_version() called too often Reduce the number of calls to encryption_get_key_get_latest_version when doing key rotation with two different methods: (1) We need to fetch key information when tablespace not yet have a encryption information, invalid keys are handled now differently (see below). There was extra call to detect if key_id is not found on key rotation. (2) If key_id is not found from encryption plugin, do not try fetching new key_version for it as it will fail anyway. We store return value from encryption_get_key_get_latest_version call and if it returns ENCRYPTION_KEY_VERSION_INVALID there is no need to call it again.

Marko Mäkelä added a comment - 2016-12-08 11:44

I replied to the commit email, requesting some changes mainly related to the monitoring interfaces.

Marko Mäkelä added a comment - 2016-12-08 11:44 I replied to the commit email, requesting some changes mainly related to the monitoring interfaces.

Jan Lindström (Inactive) added a comment - 2016-12-12 10:11

http://lists.askmonty.org/pipermail/commits/2016-December/010263.html

I did not do full refactoring as this is GA release and in my opinion we should avoid
large changes there.

Jan Lindström (Inactive) added a comment - 2016-12-12 10:11 http://lists.askmonty.org/pipermail/commits/2016-December/010263.html I did not do full refactoring as this is GA release and in my opinion we should avoid large changes there.

Marko Mäkelä added a comment - 2016-12-12 11:37

I would prefer some minor changes to the patch. See my reply on the commits list.

Marko Mäkelä added a comment - 2016-12-12 11:37 I would prefer some minor changes to the patch. See my reply on the commits list.

Jan Lindström (Inactive) added a comment - 2016-12-13 09:53

commit 72cc73cea2e7071277b6a88bae0236cabf11788a
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Tue Dec 13 11:51:33 2016 +0200

~~MDEV-10368~~: get_latest_version() called too often

Reduce the number of calls to encryption_get_key_get_latest_version
when doing key rotation with two different methods:

(1) We need to fetch key information when tablespace not yet
have a encryption information, invalid keys are handled now
differently (see below). There was extra call to detect
if key_id is not found on key rotation.

(2) If key_id is not found from encryption plugin, do not
try fetching new key_version for it as it will fail anyway.
We store return value from encryption_get_key_get_latest_version
call and if it returns ENCRYPTION_KEY_VERSION_INVALID there
is no need to call it again.

Jan Lindström (Inactive) added a comment - 2016-12-13 09:53 commit 72cc73cea2e7071277b6a88bae0236cabf11788a Author: Jan Lindström <jan.lindstrom@mariadb.com> Date: Tue Dec 13 11:51:33 2016 +0200 MDEV-10368 : get_latest_version() called too often Reduce the number of calls to encryption_get_key_get_latest_version when doing key rotation with two different methods: (1) We need to fetch key information when tablespace not yet have a encryption information, invalid keys are handled now differently (see below). There was extra call to detect if key_id is not found on key rotation. (2) If key_id is not found from encryption plugin, do not try fetching new key_version for it as it will fail anyway. We store return value from encryption_get_key_get_latest_version call and if it returns ENCRYPTION_KEY_VERSION_INVALID there is no need to call it again.

MariaDB Server

get_latest_version() called too often

Details

Description

Attachments

Issue Links

Activity

People

Dates

Git Integration