apologies - now attached

Thanks.

BROKEN: block: 562605447 checkpoint: 7834 7d4b6fda f0505855

BROKEN: block: 562605447 checkpoint: 7834 7d4b6fda f0505855

InnoDB: Log block no 562605447 at lsn 288053988352 has

InnoDB: ok header, but checksum field contains 4031797333, should be 2102095834

2016-03-26  9:04:52 7088 [ERROR] InnoDB: Redo log crypto: getting mysqld crypto key from key version failed. Reason could be that requested key version is not found or required encryption key management plugin is not found.

Looks related to MDEV-9422 (and MDEV-9774), only in this case we don't have a crash or, apparently, a busy instance.

amramsay, how do you update the instance? Did you run the MSI? Or did you stop the server manually and switch the binaries? It's just to make sure there was no kill/recovery performed by an automatic update in the middle (although it's highly unlikely, given the narrow time interval).

Still, might be a consequence or another representation of MDEV-9422.
Assigning to jplindst for the collection.

Without first stopping the existing instance, I ran the MSI which appeared to complete successfully. I then stop the running service to switch config and reinstate already existing data path, reset the service to point to new config, and restart - then I get the error. Doing exactly the same between 10.1.11 and 10.1.12 doesn't get the error. To go back to the previous version, I uninstalled 10.1.13 first. In all cases AFAIK there was no kill/recovery, of any kind, performed.

Hi, no need for more information, I could repeat this with original test case for MDEV-9422 by just removing the system killall and doing a nice shutdown. Problem is that InnoDB does read redo logs even on normal startup after normal shutdown and reads log blocs from earlier checkpoints so that latest checkpoint might not contain the necessary key to open the log block. What you can do for database that is created with < mariadb 10.1.13:

• 1) configure server with innodb_fast_shutdown=0
• 2) shutdown the server nicely
• 3) take backups for all database files and log files
• 4) delete ib_log* files
• 5) migrate to >= mariadb 10.1.13

commit 37a65e3335bd2dc28574b2f938d2a18b1b52f8cb
Author: Jan Lindström <jan.lindstrom@mariadb.com>
Date: Wed Mar 30 16:08:05 2016 +0300

MDEV-9793: getting mysqld crypto key from key version failed

Make sure that we read all possible encryption keys from checkpoint
and if log block checksum does not match, print all found
checkpoint encryption keys.

Rest of the bug is duplicate of MDEV-9774. Only difference is that you can recover from this bug, but no known recovery from MDEV-9774.

I will close this bug report as there is a way to recover from this and actual bug is handled on MDEV-9774 if the fix is found.