[MDEV-16866] InnoDB fails to start upon crash recovery with "[ERROR] InnoDB: Redo log crypto: failed to decrypt log block" - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Critical
Resolution: Fixed
Affects Version/s: 10.1.34, 10.1.39, 10.1(EOL)
Fix Version/s: 10.1.41
Component/s: Encryption, Storage Engine - InnoDB
Labels:
None

Description

The initial test flow was this:

start 10.1.34 server with the following options:

--no-defaults --file-key-management --file-key-management-filename=keys.txt --plugin-load-add=file_key_management.so --innodb-encrypt-tables --innodb-encrypt-log --innodb-encryption-threads=4 --aria-encrypt-tables=1 --encrypt-tmp-disk-tables=1 --encrypt-binlog --innodb-page-size=8K --innodb-compression-algorithm=none --loose-innodb-file-format=Barracuda --loose-innodb-file-per-table=1 --loose-max-statement-time=20 --loose-lock-wait-timeout=20 --loose-innodb-lock-wait-timeout=10 --loose-innodb_log_compressed_pages=on --loose-innodb-change-buffering=none

run a mix of DDL/DML on the server for a few minutes;
SIGKILL the server;
start the same server with the exact same options and additionally --innodb-force-recovery=3

InnoDB doesn't start:

10.1.34
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: innodb-page-size has been changed from the default value 16384 to 8192 .
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: The InnoDB memory heap is disabled
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using Linux native AIO
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using SSE crc32 instructions
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Completed initialization of buffer pool
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Highest supported file format is Barracuda.
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Starting crash recovery from checkpoint LSN=23790689
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
InnoDB: redo log checkpoint: 0 [ chk key ]: [ 4 1 ]
2018-07-31 16:08:32 140279075214848 [ERROR] InnoDB: Redo log crypto: failed to decrypt log block. Reason could be that requested key version is not found, required encryption key management plugin is not found or configured encryption algorithm and/or method does not match.
2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' init function returned error.
2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2018-07-31 16:08:32 140279075214848 [Note] Plugin 'FEEDBACK' is disabled.
2018-07-31 16:08:32 140279075214848 [ERROR] Unknown/unsupported storage engine: InnoDB
2018-07-31 16:08:32 140279075214848 [ERROR] Aborting

Error in my_thread_global_end(): 1 threads didn't exit

The key file is present and is the same for both starts.

The datadir and the key file are available, nothing is confidential, the datadir just doesn't fit into JIRA.

Attachments

Activity

Ascending order - Click to sort in descending order

Elena Stepanova created issue - 2018-07-31 13:35

Elena Stepanova made changes - 2018-07-31 13:35

Field	Original Value	New Value
Description	The initial test flow was this: - start 10.1.34 server with the following options: {noformat} --no-defaults --file-key-management --file-key-management-filename=keys.txt --plugin-load-add=file_key_management.so --innodb-encrypt-tables --innodb-encrypt-log --innodb-encryption-threads=4 --aria-encrypt-tables=1 --encrypt-tmp-disk-tables=1 --encrypt-binlog --innodb-page-size=8K --innodb-compression-algorithm=none --loose-innodb-file-format=Barracuda --loose-innodb-file-per-table=1 --loose-max-statement-time=20 --loose-lock-wait-timeout=20 --loose-innodb-lock-wait-timeout=10 --loose-innodb_log_compressed_pages=on --loose-innodb-change-buffering=none {noformat} - run a mix of DDL/DML on the server for a few minutes; - SIGKILL the server; - start the same server with the exact options and additionally {{--innodb-force-recovery=3}} InnoDB doesn't start: {noformat:title=10.1.34} 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: innodb-page-size has been changed from the default value 16384 to 8192 . 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: The InnoDB memory heap is disabled 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Compressed tables use zlib 1.2.3 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using Linux native AIO 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using SSE crc32 instructions 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Initializing buffer pool, size = 128.0M 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Completed initialization of buffer pool 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Highest supported file format is Barracuda. 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Starting crash recovery from checkpoint LSN=23790689 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer... InnoDB: redo log checkpoint: 0 [ chk key ]: [ 4 1 ] 2018-07-31 16:08:32 140279075214848 [ERROR] InnoDB: Redo log crypto: failed to decrypt log block. Reason could be that requested key version is not found, required encryption key management plugin is not found or configured encryption algorithm and/or method does not match. 2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' init function returned error. 2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2018-07-31 16:08:32 140279075214848 [Note] Plugin 'FEEDBACK' is disabled. 2018-07-31 16:08:32 140279075214848 [ERROR] Unknown/unsupported storage engine: InnoDB 2018-07-31 16:08:32 140279075214848 [ERROR] Aborting Error in my_thread_global_end(): 1 threads didn't exit {noformat} The key file is present and is the same for both starts. The datadir and the key file are available, nothing is confidential, the datadir just doesn't fit into JIRA.	The initial test flow was this: - start 10.1.34 server with the following options: {noformat} --no-defaults --file-key-management --file-key-management-filename=keys.txt --plugin-load-add=file_key_management.so --innodb-encrypt-tables --innodb-encrypt-log --innodb-encryption-threads=4 --aria-encrypt-tables=1 --encrypt-tmp-disk-tables=1 --encrypt-binlog --innodb-page-size=8K --innodb-compression-algorithm=none --loose-innodb-file-format=Barracuda --loose-innodb-file-per-table=1 --loose-max-statement-time=20 --loose-lock-wait-timeout=20 --loose-innodb-lock-wait-timeout=10 --loose-innodb_log_compressed_pages=on --loose-innodb-change-buffering=none {noformat} - run a mix of DDL/DML on the server for a few minutes; - SIGKILL the server; - start the same server with the exact same options and additionally {{--innodb-force-recovery=3}} InnoDB doesn't start: {noformat:title=10.1.34} 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: innodb-page-size has been changed from the default value 16384 to 8192 . 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: The InnoDB memory heap is disabled 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Compressed tables use zlib 1.2.3 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using Linux native AIO 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using SSE crc32 instructions 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Initializing buffer pool, size = 128.0M 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Completed initialization of buffer pool 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Highest supported file format is Barracuda. 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Starting crash recovery from checkpoint LSN=23790689 2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer... InnoDB: redo log checkpoint: 0 [ chk key ]: [ 4 1 ] 2018-07-31 16:08:32 140279075214848 [ERROR] InnoDB: Redo log crypto: failed to decrypt log block. Reason could be that requested key version is not found, required encryption key management plugin is not found or configured encryption algorithm and/or method does not match. 2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' init function returned error. 2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2018-07-31 16:08:32 140279075214848 [Note] Plugin 'FEEDBACK' is disabled. 2018-07-31 16:08:32 140279075214848 [ERROR] Unknown/unsupported storage engine: InnoDB 2018-07-31 16:08:32 140279075214848 [ERROR] Aborting Error in my_thread_global_end(): 1 threads didn't exit {noformat} The key file is present and is the same for both starts. The datadir and the key file are available, nothing is confidential, the datadir just doesn't fit into JIRA.

Elena Stepanova made changes - 2018-07-31 13:45

Assignee

Elena Stepanova [ elenst ]

Jan Lindström [ jplindst ]

Elena Stepanova made changes - 2018-07-31 13:46

Affects Version/s

10.1 [ 16100 ]

Valerii Kravchuk made changes - 2019-06-10 14:32

Affects Version/s		10.1.39 [ 23305 ]
Priority	Major [ 3 ]	Critical [ 2 ]

Valerii Kravchuk made changes - 2019-06-10 14:38

Assignee

Jan Lindström [ jplindst ]

Thirunarayanan Balathandayuthapani [ thiru ]

Thirunarayanan Balathandayuthapani made changes - 2019-06-10 18:30

Description

The initial test flow was this:

- start 10.1.34 server with the following options:
{noformat}
--no-defaults --file-key-management --file-key-management-filename=keys.txt --plugin-load-add=file_key_management.so --innodb-encrypt-tables --innodb-encrypt-log --innodb-encryption-threads=4 --aria-encrypt-tables=1 --encrypt-tmp-disk-tables=1 --encrypt-binlog --innodb-page-size=8K --innodb-compression-algorithm=none --loose-innodb-file-format=Barracuda --loose-innodb-file-per-table=1 --loose-max-statement-time=20 --loose-lock-wait-timeout=20 --loose-innodb-lock-wait-timeout=10 --loose-innodb_log_compressed_pages=on --loose-innodb-change-buffering=none
{noformat}
- run a mix of DDL/DML on the server for a few minutes;
- SIGKILL the server;
- start the same server with the exact same options and additionally {{--innodb-force-recovery=3}}

InnoDB doesn't start:
{noformat:title=10.1.34}
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: innodb-page-size has been changed from the default value 16384 to 8192 .
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: The InnoDB memory heap is disabled
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using Linux native AIO
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using SSE crc32 instructions
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Completed initialization of buffer pool
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Highest supported file format is Barracuda.
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Starting crash recovery from checkpoint LSN=23790689
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
InnoDB: redo log checkpoint: 0 [ chk key ]: [ 4 1 ]
2018-07-31 16:08:32 140279075214848 [ERROR] InnoDB: Redo log crypto: failed to decrypt log block. Reason could be that requested key version is not found, required encryption key management plugin is not found or configured encryption algorithm and/or method does not match.
2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' init function returned error.
2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2018-07-31 16:08:32 140279075214848 [Note] Plugin 'FEEDBACK' is disabled.
2018-07-31 16:08:32 140279075214848 [ERROR] Unknown/unsupported storage engine: InnoDB
2018-07-31 16:08:32 140279075214848 [ERROR] Aborting

Error in my_thread_global_end(): 1 threads didn't exit
{noformat}

The key file is present and is the same for both starts.

The datadir and the key file are available, nothing is confidential, the datadir just doesn't fit into JIRA.

The initial test flow was this:

- start 10.1.34 server with the following options:
{noformat}`
--no-defaults --file-key-management --file-key-management-filename=keys.txt --plugin-load-add=file_key_management.so --innodb-encrypt-tables --innodb-encrypt-log --innodb-encryption-threads=4 --aria-encrypt-tables=1 --encrypt-tmp-disk-tables=1 --encrypt-binlog --innodb-page-size=8K --innodb-compression-algorithm=none --loose-innodb-file-format=Barracuda --loose-innodb-file-per-table=1 --loose-max-statement-time=20 --loose-lock-wait-timeout=20 --loose-innodb-lock-wait-timeout=10 --loose-innodb_log_compressed_pages=on --loose-innodb-change-buffering=none
{noformat}
- run a mix of DDL/DML on the server for a few minutes;
- SIGKILL the server;
- start the same server with the exact same options and additionally {{--innodb-force-recovery=3}}

InnoDB doesn't start:
{noformat:title=10.1.34}
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: innodb-page-size has been changed from the default value 16384 to 8192 .
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: The InnoDB memory heap is disabled
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Compressed tables use zlib 1.2.3
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using Linux native AIO
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Using SSE crc32 instructions
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Completed initialization of buffer pool
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Highest supported file format is Barracuda.
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Starting crash recovery from checkpoint LSN=23790689
2018-07-31 16:08:31 140279075214848 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
InnoDB: redo log checkpoint: 0 [ chk key ]: [ 4 1 ]
2018-07-31 16:08:32 140279075214848 [ERROR] InnoDB: Redo log crypto: failed to decrypt log block. Reason could be that requested key version is not found, required encryption key management plugin is not found or configured encryption algorithm and/or method does not match.
2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' init function returned error.
2018-07-31 16:08:32 140279075214848 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2018-07-31 16:08:32 140279075214848 [Note] Plugin 'FEEDBACK' is disabled.
2018-07-31 16:08:32 140279075214848 [ERROR] Unknown/unsupported storage engine: InnoDB
2018-07-31 16:08:32 140279075214848 [ERROR] Aborting

Error in my_thread_global_end(): 1 threads didn't exit
{noformat}

The key file is present and is the same for both starts.

The datadir and the key file are available, nothing is confidential, the datadir just doesn't fit into JIRA.

Thirunarayanan Balathandayuthapani made changes - 2019-06-11 10:47

Status

Open [ 1 ]

In Progress [ 3 ]

Thirunarayanan Balathandayuthapani made changes - 2019-06-11 11:52

Assignee	Thirunarayanan Balathandayuthapani [ thiru ]	Marko Mäkelä [ marko ]
Status	In Progress [ 3 ]	In Review [ 10002 ]

Marko Mäkelä made changes - 2019-06-11 12:02

Assignee	Marko Mäkelä [ marko ]	Thirunarayanan Balathandayuthapani [ thiru ]
Status	In Review [ 10002 ]	Stalled [ 10000 ]

Thirunarayanan Balathandayuthapani made changes - 2019-06-12 06:56

Fix Version/s		10.1.41 [ 23406 ]
Fix Version/s	10.1 [ 16100 ]
Resolution		Fixed [ 1 ]
Status	Stalled [ 10000 ]	Closed [ 6 ]

Sergei Golubchik made changes - 2021-12-06 21:47

Workflow

MariaDB v3 [ 88667 ]

MariaDB v4 [ 154736 ]

Jira Automation (IT) made changes - 2024-07-04 07:23

Zendesk Related Tickets

141619

People

Assignee:: Thirunarayanan Balathandayuthapani

Reporter:: Elena Stepanova

Votes:: 1 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 2018-07-31 13:35

Updated:: 2024-07-07 23:49

Resolved:: 2019-06-12 06:56

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server

Details

Description

Attachments

Activity

People

Dates

Git Integration