[#MDEV-8164] Server crashes in pfs_mutex_enter_func after fil_crypt_is

enc2.opt
--plugin-load-add=file_key_management.so --file_key_management_filename=/home/elenst/git/10.1/mysql-test/std_data/keys.txt

enc2.test
--source include/have_innodb.inc
SET default_storage_engine = InnoDB;

CREATE TABLE t1 (pk INT PRIMARY KEY, c VARCHAR(256));
SET GLOBAL innodb_encrypt_tables = ON;
SET GLOBAL innodb_encryption_threads = 1;
CREATE TABLE t2 AS SELECT * FROM t1;
SET GLOBAL innodb_encryption_rotation_iops = 100;
SET GLOBAL innodb_encrypt_tables = OFF;
CREATE OR REPLACE TABLE t2 AS SELECT * FROM t1;

Run with --repeat=N (depends on the machine, e.g. it fails for me on 1st or 2nd attempt on my local box, but on ~10th attempt or so on perro).

Stack trace from bb-10.1-serg commit 0dcce2bf + MDEV-8045 patch
#3 <signal handler called>
#4 0x00007f2f0cab113a in pfs_mutex_enter_func (mutex=0x78, file_name=0x7f2f0cccaf58 "storage/innobase/fil/fil0crypt.cc", line=972) at storage/innobase/include/sync0sync.ic:245
#5 0x00007f2f0cab389f in fil_crypt_is_closing (space=5) at storage/innobase/fil/fil0crypt.cc:972
#6 0x00007f2f0cab4e91 in fil_crypt_rotate_page (key_state=0x7f2efd7fedb0, state=0x7f2efd7fedc0) at storage/innobase/fil/fil0crypt.cc:1759
#7 0x00007f2f0cab541d in fil_crypt_rotate_pages (key_state=0x7f2efd7fedb0, state=0x7f2efd7fedc0) at storage/innobase/fil/fil0crypt.cc:1926
#8 0x00007f2f0cab5af3 in fil_crypt_thread (arg=0x0) at storage/innobase/fil/fil0crypt.cc:2143
#9 0x00007f2f15776b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#10 0x00007f2f1380b95d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

You might get a slighly different stack trace or an assertion failure, it happened to me. Example:

#3  <signal handler called>

#4  0x00007fa6396b113a in pfs_mutex_enter_func (mutex=0x78, file_name=0x7fa6398caf58 "storage/innobase/fil/fil0crypt.cc", line=1565) at storage/innobase/include/sync0sync.ic:245

#5  0x00007fa6396b490e in fil_crypt_find_page_to_rotate (key_state=0x7fa62a3fedb0, state=0x7fa62a3fedc0) at storage/innobase/fil/fil0crypt.cc:1565

#6  0x00007fa6396b5b31 in fil_crypt_thread (arg=0x0) at storage/innobase/fil/fil0crypt.cc:2140

#7  0x00007fa6422e1b50 in start_thread (arg=<optimized out>) at pthread_create.c:304

#8  0x00007fa64037695d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

[MDEV-8164] Server crashes in pfs_mutex_enter_func after fil_crypt_is_closing or alike Created: 2015-05-15 Updated: 2015-05-28 Resolved: 2015-05-18
Status:	Closed
Project:	MariaDB Server
Component/s:	Encryption, Storage Engine - InnoDB, Storage Engine - XtraDB
Affects Version/s:	10.1.4
Fix Version/s:	10.1.5

[MDEV-8164] Server crashes in pfs_mutex_enter_func after fil_crypt_is_closing or alike Created: 2015-05-15 Updated: 2015-05-28 Resolved: 2015-05-18