Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-28330

Hashicorp: Document key caching and key version caching

Details

    Description

      According to the description in hashicorp_key_management.cnf, key cache related variables mean the following:

      --[loose-]hashicorp-key-management-caching-enabled="on"|"off"

      Enable key caching (storing key values received from
      the Hashicorp Vault server in the local memory). By default
      caching is enabled.

      --[loose-]hashicorp-key-management-cache-timeout=<timeout>

      The time (in milliseconds) after which the value of the key
      stored in the cache becomes invalid and an attempt to read this
      data causes a new request send to the vault server. By default,
      cache entries become invalid after 60,000 milliseconds (after
      one minute).

      The test below does the following:

      • configure keys 1 and 4 in the vault;
      • create a table with ENCRYPTION_KEY_ID=4, so that the key has been read;
      • reconfigure the vault to remove key 4;
      • try to create another table with ENCRYPTION_KEY_ID=4.

      Since key cache is enabled and the (default) timeout is 60 seconds, presumably it should still work and the table should be created using the key from the cache. It doesn't happen however, the statement fails due to the missing key.

      # The test presumes that the local vault is running at $VAULT_ADDR,
      		 
      # and the token is configured in $VAULT_TOKEN
      		 
       
      		 
      --source include/have_innodb.inc
      		 
       
      		 
      --exec vault secrets disable bug
      		 
      --exec vault secrets enable -path /bug -version=2 kv
      		 
      --exec vault kv put /bug/1 data=01234567890123456789012345678901 > /dev/null
      		 
      --exec vault kv put /bug/4 data=01234567890123456789012345678904 > /dev/null
      		 
       
      		 
      --let $restart_parameters= --plugin-load-add=hashicorp_key_management --hashicorp-key-management-vault-url="$VAULT_ADDR/v1/bug/" --hashicorp-key-management-token="$VAULT_TOKEN"
      		 
       
      		 
      --source include/restart_mysqld.inc
      		 
       
      		 
      CREATE TABLE t1 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
      		 
      INSERT INTO t1 VALUES ('foo'),('bar');
      		 
       
      		 
      select @@hashicorp_key_management_caching_enabled, @@hashicorp_key_management_cache_timeout;
      		 
       
      		 
      --exec vault secrets disable bug
      		 
      --exec vault secrets enable -path /bug -version=2 kv
      		 
      --exec vault kv put /bug/1 data=01234567890123456789012345678901 > /dev/null
      		 
       
      		 
      CREATE TABLE t2 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
      		 
       
      		 
      # Cleanup
      		 
      DROP TABLE IF EXISTS t1, t2;
      		 
      --exec vault secrets disable bug

      preview-10.9-MDEV-20119-misc e62a2a0615

      		 
      CREATE TABLE t1 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
      		 
      INSERT INTO t1 VALUES ('foo'),('bar');
      		 
      select @@hashicorp_key_management_caching_enabled, @@hashicorp_key_management_cache_timeout;
      		 
      @@hashicorp_key_management_caching_enabled	@@hashicorp_key_management_cache_timeout
      		 
      1	60000
      		 
       
      		 
      CREATE TABLE t2 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
      		 
       
      		 
      mysqltest: At line 24: query 'CREATE TABLE t2 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4' failed: ER_CANT_CREATE_TABLE (1005): Can't create table `test`.`t2` (errno: 140 "Wrong create options")

      There are no test cases coming with the plugin to validate any cache functionality, so there is still a room for interpretation. If expectations in the test above are false, then at least add tests to indicate how the cache is expected to act, and document it accordingly.

      Attachments

        Issue Links

          is part of

          Task - A task that needs to be done. MDEV-28494 Hashicorp plugin documentation

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-19281 Vault Key Management Plugin

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          split to

          Task - A task that needs to be done. MDEV-28528 Hashicorp Vault plugin: documentation update

          • Major - Major loss of function.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            elenst Elena Stepanova created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -
            Description According to the description in hashicorp_key_management.cnf, key cache related variables mean the following:
            {quote}
            --[loose-]hashicorp-key-management-caching-enabled="on"|"off"

              Enable key caching (storing key values received from
              the Hashicorp Vault server in the local memory). By default
              caching is enabled.

            --[loose-]hashicorp-key-management-cache-timeout=<timeout>

              The time (in milliseconds) after which the value of the key
              stored in the cache becomes invalid and an attempt to read this
              data causes a new request send to the vault server. By default,
              cache entries become invalid after 60,000 milliseconds (after
              one minute).
            {quote}

            The test below does the following:
            - configure keys 1 and 4 in the vault;
            - create a table with ENCRYPTION_KEY_ID=4, so that the key has been read;
            - reconfigure the vault to remove key 4;
            - try to create another table with ENCRYPTION_KEY_ID=4.

            Since key cache is enabled and the (default) timeout is 60 seconds, presumably it should still work and the table should be created using the key from the cache. It doesn't happen however, the statement fails due to the missing key.

            There are no test cases coming with the plugin to validate any cache functionality, so there is still a room for interpretation. If expectations in the test above are false, then at least add tests to indicate how the cache is expected to act, and document it accordingly.             		According to the description in hashicorp_key_management.cnf, key cache related variables mean the following:
            {quote}
            --[loose-]hashicorp-key-management-caching-enabled="on"|"off"

              Enable key caching (storing key values received from
              the Hashicorp Vault server in the local memory). By default
              caching is enabled.

            --[loose-]hashicorp-key-management-cache-timeout=<timeout>

              The time (in milliseconds) after which the value of the key
              stored in the cache becomes invalid and an attempt to read this
              data causes a new request send to the vault server. By default,
              cache entries become invalid after 60,000 milliseconds (after
              one minute).
            {quote}

            The test below does the following:
            - configure keys 1 and 4 in the vault;
            - create a table with ENCRYPTION_KEY_ID=4, so that the key has been read;
            - reconfigure the vault to remove key 4;
            - try to create another table with ENCRYPTION_KEY_ID=4.

            Since key cache is enabled and the (default) timeout is 60 seconds, presumably it should still work and the table should be created using the key from the cache. It doesn't happen however, the statement fails due to the missing key.

            {code:sql}
            # The test presumes that the local vault is running at $VAULT_ADDR,
            # and the token is configured in $VAULT_TOKEN

            --source include/have_innodb.inc

            --exec vault secrets disable bug
            --exec vault secrets enable -path /bug -version=2 kv
            --exec vault kv put /bug/1 data=01234567890123456789012345678901 > /dev/null
            --exec vault kv put /bug/4 data=01234567890123456789012345678904 > /dev/null

            --let $restart_parameters= --plugin-load-add=hashicorp_key_management --hashicorp-key-management-vault-url="$VAULT_ADDR/v1/bug/" --hashicorp-key-management-token="$VAULT_TOKEN"

            --source include/restart_mysqld.inc

            CREATE TABLE t1 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
            INSERT INTO t1 VALUES ('foo'),('bar');

            select @@hashicorp_key_management_caching_enabled, @@hashicorp_key_management_cache_timeout;

            --exec vault secrets disable bug
            --exec vault secrets enable -path /bug -version=2 kv
            --exec vault kv put /bug/1 data=01234567890123456789012345678901 > /dev/null

            CREATE TABLE t2 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;

            # Cleanup
            DROP TABLE IF EXISTS t1, t2;
            --exec vault secrets disable bug
            {code}

            {code:sql|title=preview-10.9-MDEV-20119-misc e62a2a0615}
            CREATE TABLE t1 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;
            INSERT INTO t1 VALUES ('foo'),('bar');
            select @@hashicorp_key_management_caching_enabled, @@hashicorp_key_management_cache_timeout;
            @@hashicorp_key_management_caching_enabled @@hashicorp_key_management_cache_timeout
            1 60000

            CREATE TABLE t2 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4;

            mysqltest: At line 24: query 'CREATE TABLE t2 (a VARCHAR(8)) ENGINE=InnoDB ENCRYPTED=YES ENCRYPTION_KEY_ID=4' failed: ER_CANT_CREATE_TABLE (1005): Can't create table `test`.`t2` (errno: 140 "Wrong create options")
            {code}

            There are no test cases coming with the plugin to validate any cache functionality, so there is still a room for interpretation. If expectations in the test above are false, then at least add tests to indicate how the cache is expected to act, and document it accordingly.
            sysprg Julius Goryavsky made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            sysprg Julius Goryavsky made changes -
            Assignee Julius Goryavsky [ sysprg ] Alexander Barkov [ bar ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            bar Alexander Barkov made changes -
            Assignee Alexander Barkov [ bar ] Julius Goryavsky [ sysprg ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            sysprg Julius Goryavsky made changes -
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -
            Component/s Documentation [ 10903 ]
            Priority Critical [ 2 ] Major [ 3 ]
            Summary Hashicorp: Key caching doesn't appear to be working Hashicorp: Document key caching and key version caching
            elenst Elena Stepanova made changes -
            sysprg Julius Goryavsky made changes -
            sysprg Julius Goryavsky made changes -
            issue.field.resolutiondate 2022-05-10 11:07:03.0 2022-05-10 11:07:03.829
            sysprg Julius Goryavsky made changes -
            Fix Version/s 10.9.1 [ 27114 ]
            Fix Version/s 10.9 [ 26905 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            elenst Elena Stepanova made changes -
            julien.fritsch Julien Fritsch made changes -
            julien.fritsch Julien Fritsch made changes -
            Labels Cloned
            julien.fritsch Julien Fritsch made changes -

            People

              sysprg Julius Goryavsky
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.