[MDEV-23447] SIGSEGV in fil_system_t::keyrotate_next Created: 2020-08-11  Updated: 2022-02-10  Resolved: 2020-08-11

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.2.35, 10.3.26, 10.4.16, 10.5.7

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: not-10.1, sporadic

Issue Links:
Relates
relates to MDEV-20399 SEGV in fil_system_t::keyrotate_next ... Closed

 Description   

SET GLOBAL innodb_encryption_threads=5;
SET GLOBAL innodb_encryption_rotate_key_age=0;
SELECT SLEEP(5);  # Somewhat delayed crash happens during sleep

Leads, sporadically, to:

10.4.15 eae968f62d285de97ed607c87bc131cd863d5d03 (Optimized)

Core was generated by `/test/MD110820-mariadb-10.4.15-linux-x86_64-opt/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  my_read (Filedes=Filedes@entry=40, 
    Buffer=Buffer@entry=0x149a739fbe70 "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=Count@entry=4096, MyFlags=MyFlags@entry=0)
    at /test/10.4_opt/mysys/my_read.c:63
[Current thread is 1 (Thread 0x149a739fe700 (LWP 586712))]
(gdb) bt
#0  my_read (Filedes=Filedes@entry=40, Buffer=Buffer@entry=0x149a739fbe70 "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=Count@entry=4096, MyFlags=MyFlags@entry=0) at /test/10.4_opt/mysys/my_read.c:63
#1  0x0000561dbc8c81dd in output_core_info () at /test/10.4_opt/sql/signal_handler.cc:66
#2  0x0000561dbc8c8610 in handle_fatal_signal (sig=11) at /test/10.4_opt/sql/signal_handler.cc:339
#3  <signal handler called>
#4  fil_system_t::keyrotate_next (encrypt=false, recheck=false, space=0x149a9888f780, this=<optimized out>) at /test/10.4_opt/storage/innobase/fil/fil0crypt.cc:1598
#5  fil_space_next (space=space@entry=0x149a9888f780, recheck=recheck@entry=false, encrypt=<optimized out>) at /test/10.4_opt/storage/innobase/fil/fil0crypt.cc:1638
#6  0x0000561dbcc75ba0 in fil_crypt_find_space_to_rotate (recheck=<optimized out>, state=<optimized out>, key_state=<optimized out>) at /test/10.4_opt/storage/innobase/fil/fil0crypt.cc:1718
#7  fil_crypt_thread () at /test/10.4_opt/storage/innobase/fil/fil0crypt.cc:2412
#8  0x0000149a9ad396db in start_thread (arg=0x149a739fe700) at pthread_create.c:463
#9  0x0000149a99eb3a3f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.5.6 1c587481966abc7a9ad5309d0a91ca920f7a5657 (Debug)

Core was generated by `/test/MD110820-mariadb-10.5.6-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055cea2a9934a in my_read (Filedes=Filedes@entry=39, 
    Buffer=Buffer@entry=0x14d5a65fc170 "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=Count@entry=4096, MyFlags=MyFlags@entry=0)
    at /test/10.5_dbg/mysys/my_read.c:63
63	      int got_errno= my_errno= errno;
[Current thread is 1 (Thread 0x14d5a65fe700 (LWP 594745))]
(gdb) bt
#0  0x000055cea2a9934a in my_read (Filedes=Filedes@entry=39, Buffer=Buffer@entry=0x14d5a65fc170 "Limit", ' ' <repeats 21 times>, "Soft Limit", ' ' <repeats 11 times>, "Hard Limit", ' ' <repeats 11 times>, "Units     \nMax cpu time", ' ' <repeats 14 times>, "unlimited", ' ' <repeats 12 times>, "unlimited", ' ' <repeats 12 times>, "seconds   \nMax file size", ' ' <repeats 13 times>, "unlimited       "..., Count=Count@entry=4096, MyFlags=MyFlags@entry=0) at /test/10.5_dbg/mysys/my_read.c:63
#1  0x000055cea22537c6 in output_core_info () at /test/10.5_dbg/sql/signal_handler.cc:68
#2  0x000055cea2253d5d in handle_fatal_signal (sig=11) at /test/10.5_dbg/sql/signal_handler.cc:326
#3  <signal handler called>
#4  fil_system_t::keyrotate_next (encrypt=false, recheck=false, space=0x14d5d4c43a18, this=<optimized out>) at /test/10.5_dbg/storage/innobase/fil/fil0crypt.cc:1471
#5  fil_space_next (space=0x14d5d4c43a18, recheck=recheck@entry=false, encrypt=encrypt@entry=false) at /test/10.5_dbg/storage/innobase/fil/fil0crypt.cc:1511
#6  0x000055cea2991f4d in fil_crypt_find_space_to_rotate (recheck=<optimized out>, state=<optimized out>, key_state=<optimized out>) at /test/10.5_dbg/storage/innobase/fil/fil0crypt.cc:1591
#7  fil_crypt_thread () at /test/10.5_dbg/storage/innobase/fil/fil0crypt.cc:2129
#8  0x000014d5d70956db in start_thread (arg=0x14d5a65fe700) at pthread_create.c:463
#9  0x000014d5d6493a3f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Bug confirmed to be present in:
MariaDB: 10.2.34 (dbg), 10.2.34 (opt), 10.3.25 (dbg), 10.3.25 (opt), 10.4.15 (dbg), 10.4.15 (opt), 10.5.6 (dbg), 10.5.6 (opt)

Bug confirmed not present in:
MariaDB: 10.1.47 (dbg), 10.1.47 (opt)
MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.47 (dbg), 5.6.47 (opt), 5.7.29 (dbg), 5.7.29 (opt), 8.0.19 (dbg), 8.0.19 (opt)



 Comments   
Comment by Marko Mäkelä [ 2020-08-11 ]

The problem was that fil_system_t::keyrotate_next() attempted to iterate fil_system.rotation_list starting from space even though space->is_in_rotation_list did not hold. That will cause the iterator to wrap an invalid pointer (in debug builds, it might be guaranteed to be a null pointer), and an attempt to dereference the invalid pointer may trigger SIGSEGV.

Comment by Roel Van de Paar [ 2020-08-11 ]

Super. Thank you Marko

Generated at Thu Feb 08 09:22:29 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.