[MDEV-23267] Assertion `!srv_read_only_mode || m_ignore_read_only' failed in SysTablespace::create_file when using innodb-force-recovery Created: 2020-07-23  Updated: 2021-12-21  Resolved: 2021-10-25

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7
Fix Version/s: 10.2.41, 10.3.32, 10.4.22, 10.5.13, 10.6.5

Type: Bug Priority: Critical
Reporter: Roel Van de Paar Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: affects-tests

Issue Links:
Relates
relates to MDEV-27332 SIGSEGV in <lambda(trx_t&)>::operator... Closed

 Description   

$ cd /test/MD140720-mariadb-10.5.5-linux-x86_64-opt
$ rm -Rf data*
$ ./scripts/mariadb-install-db --no-defaults --force --auth-root-authentication-method=normal --innodb-force-recovery=254 --basedir=${PWD} --datadir=${PWD}/data

Leads to:

10.5.5 30e7a0a866dce530d8328c6d614e48d39a264f9b (Optimized)

Core was generated by `/test/MD140720-mariadb-10.5.5-linux-x86_64-opt/bin/mysqld --no-defaults --lc-me'.
Program terminated with signal SIGABRT, Aborted.
#0  0x000014d9f75dd237 in kill () at ../sysdeps/unix/syscall-template.S:78
[Current thread is 1 (Thread 0x14d9f91ce800 (LWP 2512383))]
(gdb) bt
#0  0x000014d9f75dd237 in kill () at ../sysdeps/unix/syscall-template.S:78
#1  0x000055c9b257b037 in handle_fatal_signal (sig=6) at /test/10.5_opt/sql/signal_handler.cc:342
#2  <signal handler called>
#3  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#4  0x000014d9f75de8b1 in __GI_abort () at abort.c:79
#5  0x000055c9b290fb19 in os_file_handle_error_cond_exit (on_error_silent=false, should_abort=true, operation=0x55c9b2d0fcf7 "open", name=0x55c9b4a59078 "./ibdata1") at /test/10.5_opt/storage/innobase/os/os0file.cc:3464
#6  os_file_handle_error (operation=0x55c9b2d0fcf7 "open", name=0x55c9b4a59078 "./ibdata1") at /test/10.5_opt/storage/innobase/os/os0file.cc:212
#7  os_file_create_func (name=name@entry=0x55c9b4a59078 "./ibdata1", create_mode=create_mode@entry=52, purpose=purpose@entry=62, type=type@entry=100, read_only=read_only@entry=true, success=success@entry=0x7fffe0c8b427) at /test/10.5_opt/storage/innobase/os/os0file.cc:1424
#8  0x000055c9b2ad212b in pfs_os_file_create_func (purpose=62, type=100, src_file=0x55c9b2ef1bb8 "/test/10.5_opt/storage/innobase/fsp/fsp0file.cc", src_line=72, src_line=72, src_file=0x55c9b2ef1bb8 "/test/10.5_opt/storage/innobase/fsp/fsp0file.cc", success=0x7fffe0c8b427, read_only=true, type=100, purpose=62, create_mode=52, name=0x55c9b4a59078 "./ibdata1", key=<optimized out>) at /test/10.5_opt/storage/innobase/include/os0file.ic:168
#9  Datafile::open_or_create (this=this@entry=0x55c9b4a590a8, read_only_mode=true) at /test/10.5_opt/storage/innobase/fsp/fsp0file.cc:70
#10 0x000055c9b2ad9826 in SysTablespace::create_file (file=@0x55c9b4a590a8: {_vptr.Datafile = 0x55c9b352ca90 <vtable for Datafile+16>, m_name = 0x55c9b4a591d8 "ibdata1", m_filepath = 0x55c9b4a59078 "./ibdata1", m_filename = 0x55c9b4a5907a "ibdata1", m_handle = {m_file = -1, m_psi = 0x0}, m_open_flags = OS_FILE_CREATE, m_size = 768, m_order = 0, m_type = SRV_NOT_RAW, m_space_id = 18446744073709551615, m_flags = 21, m_exists = false, m_is_valid = false, m_first_page = 0x0, m_last_os_error = 0, m_file_info = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}}}, this=0x55c9b3ff1bc0 <srv_sys_space>) at /test/10.5_opt/storage/innobase/fsp/fsp0sysspace.cc:448
#11 SysTablespace::open_or_create (this=0x55c9b3ff1bc0 <srv_sys_space>, is_temp=is_temp@entry=false, create_new_db=create_new_db@entry=true, sum_new_sizes=sum_new_sizes@entry=0x7fffe0c8b7d0, flush_lsn=flush_lsn@entry=0x7fffe0c8b7c8) at /test/10.5_opt/storage/innobase/fsp/fsp0sysspace.cc:867
#12 0x000055c9b29b0e8b in srv_start (create_new_db=true) at /test/10.5_opt/storage/innobase/srv/srv0start.cc:1360
#13 0x000055c9b28a2c02 in innodb_init (p=<optimized out>) at /test/10.5_opt/storage/innobase/handler/ha_innodb.cc:4001
#14 0x000055c9b257df3e in ha_initialize_handlerton (plugin=0x55c9b49985b0) at /test/10.5_opt/sql/handler.cc:645
#15 0x000055c9b23991b3 in plugin_initialize (tmp_root=tmp_root@entry=0x7fffe0c90d10, plugin=plugin@entry=0x55c9b49985b0, argc=argc@entry=0x55c9b371fc80 <remaining_argc>, argv=argv@entry=0x55c9b4961e38, options_only=options_only@entry=false) at /test/10.5_opt/sql/sql_plugin.cc:1459
#16 0x000055c9b239a453 in plugin_init (argc=argc@entry=0x55c9b371fc80 <remaining_argc>, argv=<optimized out>, flags=1) at /test/10.5_opt/sql/sql_plugin.cc:1751
#17 0x000055c9b22bf7ed in init_server_components () at /test/10.5_opt/sql/mysqld.cc:4909
#18 0x000055c9b22c558a in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.5_opt/sql/mysqld.cc:5492
#19 0x000014d9f75bfb97 in __libc_start_main (main=0x55c9b2288940 <main(int, char**)>, argc=13, argv=0x7fffe0c92ca8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffe0c92c98) at ../csu/libc-start.c:310
#20 0x000055c9b22ba1fa in _start ()

10.5.5 30e7a0a866dce530d8328c6d614e48d39a264f9b (Debug)

mysqld: /test/10.5_dbg/storage/innobase/fsp/fsp0sysspace.cc:430: dberr_t SysTablespace::create_file(Datafile&): Assertion `!srv_read_only_mode || m_ignore_read_only' failed.

10.5.5 30e7a0a866dce530d8328c6d614e48d39a264f9b (Debug)

mysqld: /test/10.5_dbg/storage/innobase/fsp/fsp0sysspace.cc:430: dberr_t SysTablespace::create_file(Datafile&): Assertion `!srv_read_only_mode || m_ignore_read_only' failed.
200723 15:22:06 [ERROR] mysqld got signal 6 ;

10.5.5 30e7a0a866dce530d8328c6d614e48d39a264f9b (Debug)

Core was generated by `/test/MD140720-mariadb-10.5.5-linux-x86_64-dbg/bin/mysqld --no-defaults --lc-me'.
Program terminated with signal SIGABRT, Aborted.
#0  0x000014744422b237 in kill () at ../sysdeps/unix/syscall-template.S:78
[Current thread is 1 (Thread 0x147445e1c800 (LWP 2487656))]
(gdb) bt
#0  0x000014744422b237 in kill () at ../sysdeps/unix/syscall-template.S:78
#1  0x000055846afc171f in handle_fatal_signal (sig=6) at /test/10.5_dbg/sql/signal_handler.cc:342
#2  <signal handler called>
#3  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#4  0x000014744422c8b1 in __GI_abort () at abort.c:79
#5  0x000014744421c42a in __assert_fail_base (fmt=0x1474443a3a38 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x55846bcce808 "!srv_read_only_mode || m_ignore_read_only", file=file@entry=0x55846bcce6d8 "/test/10.5_dbg/storage/innobase/fsp/fsp0sysspace.cc", line=line@entry=430, function=function@entry=0x55846bcce5a0 <SysTablespace::create_file(Datafile&)::__PRETTY_FUNCTION__> "dberr_t SysTablespace::create_file(Datafile&)") at assert.c:92
#6  0x000014744421c4a2 in __GI___assert_fail (assertion=assertion@entry=0x55846bcce808 "!srv_read_only_mode || m_ignore_read_only", file=file@entry=0x55846bcce6d8 "/test/10.5_dbg/storage/innobase/fsp/fsp0sysspace.cc", line=line@entry=430, function=function@entry=0x55846bcce5a0 <SysTablespace::create_file(Datafile&)::__PRETTY_FUNCTION__> "dberr_t SysTablespace::create_file(Datafile&)") at assert.c:101
#7  0x000055846b712f23 in SysTablespace::create_file (this=this@entry=0x55846ce2e960 <srv_sys_space>, file=@0x55846ea6e568: {_vptr.Datafile = 0x55846c35e850 <vtable for Datafile+16>, m_name = 0x55846ea6e698 "ibdata1", m_filepath = 0x55846ea6e538 "./ibdata1", m_filename = 0x55846ea6e53a "ibdata1", m_handle = {m_file = -1, m_psi = 0x0}, m_open_flags = OS_FILE_CREATE, m_size = 768, m_order = 0, m_type = SRV_NOT_RAW, m_space_id = 18446744073709551615, m_flags = 21, m_exists = false, m_is_valid = false, m_first_page = 0x0, m_last_os_error = 0, m_file_info = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}}}) at /test/10.5_dbg/storage/innobase/fsp/fsp0sysspace.cc:430
#8  0x000055846b7135b5 in SysTablespace::open_or_create (this=0x55846ce2e960 <srv_sys_space>, is_temp=is_temp@entry=false, create_new_db=create_new_db@entry=true, sum_new_sizes=sum_new_sizes@entry=0x7ffdc8e32268, flush_lsn=flush_lsn@entry=0x7ffdc8e32260) at /test/10.5_dbg/storage/innobase/fsp/fsp0sysspace.cc:867
#9  0x000055846b5748d6 in srv_start (create_new_db=<optimized out>) at /test/10.5_dbg/storage/innobase/srv/srv0start.cc:1360
#10 0x000055846b38eb0e in innodb_init (p=<optimized out>) at /test/10.5_dbg/storage/innobase/handler/ha_innodb.cc:4001
#11 0x000055846afc5ff5 in ha_initialize_handlerton (plugin=0x55846e9a2a40) at /test/10.5_dbg/sql/handler.cc:645
#12 0x000055846ad31043 in plugin_initialize (tmp_root=tmp_root@entry=0x7ffdc8e37a50, plugin=plugin@entry=0x55846e9a2a40, argc=argc@entry=0x55846c555e80 <remaining_argc>, argv=argv@entry=0x55846e962298, options_only=<optimized out>) at /test/10.5_dbg/sql/sql_plugin.cc:1459
#13 0x000055846ad32ada in plugin_init (argc=argc@entry=0x55846c555e80 <remaining_argc>, argv=<optimized out>, flags=1) at /test/10.5_dbg/sql/sql_plugin.cc:1751
#14 0x000055846ac143c4 in init_server_components () at /test/10.5_dbg/sql/mysqld.cc:4909
#15 0x000055846ac1ad6e in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.5_dbg/sql/mysqld.cc:5492
#16 0x000055846ac0d6b3 in main (argc=<optimized out>, argv=<optimized out>) at /test/10.5_dbg/sql/main.cc:25

Same issue reproducible on 10.3 using slightly modified testcase (using mysql_install_db):

$ rm -Rf data*; ./scripts/mysql_install_db --no-defaults --force --auth-root-authentication-method=normal --innodb-force-recovery=254 --basedir=${PWD} --datadir=${PWD}/data

Bug confirmed present in:
MariaDB: 10.1.46 (dbg), 10.1.46 (opt), 10.3.24 (opt), 10.3.24 (dbg), 10.4.14 (opt), 10.4.14 (dbg), 10.5.5 (opt), 10.5.5 (dbg)

Bug confirmed not present in:
MariaDB: 10.2.33 (opt), 10.2.33 (dbg)



 Comments   
Comment by Roel Van de Paar [ 2020-07-23 ]

Unique bug ID's across versions.

SIGABRT|os_file_handle_error_cond_exit|os_file_handle_error|os_file_create_func|pfs_os_file_create_func  # 10.1(dbg+opt)+10.5(opt)
SIGABRT|ut_dbg_assertion_failed|SysTablespace::create_file|SysTablespace::open_or_create|srv_start  # 10.3(dbg+opt)+10.4(dbg+opt)
!srv_read_only_mode || m_ignore_read_only|SIGABRT|SysTablespace::create_file|SysTablespace::open_or_create|srv_start|innodb_init  # 10.5(dbg)
!srv_read_only_mode || m_ignore_read_only|SIGABRT|ut_dbg_assertion_failed|SysTablespace::create_file|SysTablespace::open_or_create|srv_start
!srv_read_only_mode|SIGABRT|fil_write_flushed_lsn|checkpoint_now_set|sys_var_pluginvar::global_update|sys_var::update
!srv_read_only_mode|SIGABRT|log_buffer_flush_to_disk|ha_innobase::extra|select_insert::prepare_eof|select_insert::send_eof
SIGABRT|ib::fatal::~fatal|log_t::file::write|log_write_buf|log_write
SIGSEGV|fetch_data_into_cache|trx_i_s_possibly_fetch_data_into_cache|trx_i_s_common_fill_table|get_schema_tables_result
SIGSEGV|<lambdaatomic|thread_safe_trx_ilist_t::for_each<fetch_data_into_cachetrx0sys.h|fetch_data_into_cache|trx_i_s_possibly_fetch_data_into_cache

Comment by Marko Mäkelä [ 2020-07-23 ]

Maybe InnoDB should simply refuse to create any files when innodb_force_recovery is set to any of the values 4, 5, 6. The maximum value is 6, and any value bigger than that should be clamped to the maximum value somewhere outside any storage engine code.

Comment by Roel Van de Paar [ 2020-07-23 ]

Yes, confirming innodb-force-recovery change

option 'innodb-force-recovery': unsigned value 254 adjusted to 6

Comment by Roel Van de Paar [ 2020-07-23 ]

Actually, just found that 10.2 has an issue also, different again

/test/MD250620-mariadb-10.2.33-linux-x86_64-dbg$ rm -Rf data*; ./scripts/mysql_install_db --no-defaults --force --auth-root-authentication-method=normal --innodb-force-recovery=254 --basedir=${PWD} --datadir=${PWD}/data
Installing MariaDB/MySQL system tables in '/test/MD250620-mariadb-10.2.33-linux-x86_64-dbg/data' ...
2020-07-23 16:02:22 22658891986880 [Warning] InnoDB: Unable to open "./ib_logfile0" to check native AIO read support.
2020-07-23 16:02:22 22658891986880 [Warning] InnoDB: Linux Native AIO disabled.
2020-07-23 16:02:22 22658891986880 [ERROR] InnoDB: Can't create file './ibdata1' when --innodb-read-only is set
2020-07-23 16:02:22 22658891986880 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[1872] with error Generic error
2020-07-23 16:02:22 22658891986880 [ERROR] Plugin 'InnoDB' init function returned error.
2020-07-23 16:02:22 22658891986880 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2020-07-23 16:02:22 22658891986880 [ERROR] Unknown/unsupported storage engine: InnoDB
2020-07-23 16:02:22 22658891986880 [ERROR] Aborting

Comment by Roel Van de Paar [ 2021-08-17 ]

Yet another stack seen with this testcase.

# mysqld options required for replay:  --innodb-force-recovery=24
INSERT INTO mysql.innodb_table_stats SELECT database_name,''AS table_name,laST_UPDATE,123 AS n_rows,clustered_index_size,sum_of_other_index_sizes FROM mysql.innodb_table_stats WHERE table_name='';

10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Debug)

mysqld: /test/10.6_dbg/storage/innobase/log/log0log.cc:841: void log_buffer_flush_to_disk(bool): Assertion `!srv_read_only_mode' failed.                                                                                                

10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Debug)

Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-dbg/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x1478c821b700 (LWP 497420))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00001478cae24859 in __GI_abort () at abort.c:79
#2  0x00001478cae24729 in __assert_fail_base (fmt=0x1478cafba588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5577c940e14f "!srv_read_only_mode", file=0x5577c9433c38 "/test/10.6_dbg/storage/innobase/log/log0log.cc", line=841, function=<optimized out>) at assert.c:92                                                                      
#3  0x00001478cae35f36 in __GI___assert_fail (assertion=assertion@entry=0x5577c940e14f "!srv_read_only_mode", file=file@entry=0x5577c9433c38 "/test/10.6_dbg/storage/innobase/log/log0log.cc", line=line@entry=841, function=function@entry=0x5577c9434648 "void log_buffer_flush_to_disk(bool)") at assert.c:101                                           
#4  0x00005577c8c93e39 in log_buffer_flush_to_disk (sync=sync@entry=true) at /test/10.6_dbg/storage/innobase/log/log0log.cc:841                                                                                                         
#5  0x00005577c8bfdde7 in ha_innobase::extra (this=0x14789016f790, operation=HA_EXTRA_END_ALTER_COPY) at /test/10.6_dbg/storage/innobase/handler/ha_innodb.cc:15435                                                                     
#6  0x00005577c84d347b in select_insert::prepare_eof (this=this@entry=0x1478900148e8) at /test/10.6_dbg/sql/sql_insert.cc:4169                                                                                                          
#7  0x00005577c84d3ae6 in select_insert::send_eof (this=0x1478900148e8) at /test/10.6_dbg/sql/sql_insert.cc:4268
#8  0x00005577c859f9e4 in do_select (procedure=<optimized out>, join=0x1478900149a8) at /test/10.6_dbg/sql/sql_select.cc:20621                                                                                                          
#9  JOIN::exec_inner (this=this@entry=0x1478900149a8) at /test/10.6_dbg/sql/sql_select.cc:4727
#10 0x00005577c859fd30 in JOIN::exec (this=this@entry=0x1478900149a8) at /test/10.6_dbg/sql/sql_select.cc:4505
#11 0x00005577c859dc98 in mysql_select (thd=thd@entry=0x147890000db8, tables=0x1478906f6218, fields=@0x1478906f5988: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x1478906f5c88, last = 0x1478906f61a8, elements = 6}, <No data fields>}, conds=0x1478906f6ac0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2202455245568, result=0x1478900148e8, unit=0x147890005130, select_lex=0x1478906f56e8) at /test/10.6_dbg/sql/sql_select.cc:4983                                                                                                             
#12 0x00005577c859df4d in handle_select (thd=thd@entry=0x147890000db8, lex=lex@entry=0x147890005068, result=result@entry=0x1478900148e8, setup_tables_done_option=setup_tables_done_option@entry=1073741824) at /test/10.6_dbg/sql/sql_select.cc:545                                                                                                        
#13 0x00005577c852083d in mysql_execute_command (thd=thd@entry=0x147890000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.6_dbg/sql/sql_parse.cc:4712                                           
#14 0x00005577c850acf5 in mysql_parse (thd=thd@entry=0x147890000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x1478c821a400) at /test/10.6_dbg/sql/sql_parse.cc:8026                            
#15 0x00005577c8519860 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x147890000db8, packet=packet@entry=0x1478909685b9 "INSERT INTO mysql.innodb_table_stats SELECT database_name, 'stats_rename2' AS table_name, last_update, 123 AS n_rows, clustered_index_size, sum_of_other_index_sizes FROM mysql.innodb_table_stats WHERE"..., packet_length=packet_length@entry=230, blocking=blocking@entry=true) at /test/10.6_dbg/sql/sql_class.h:1340        
#16 0x00005577c851cc78 in do_command (thd=0x147890000db8, blocking=blocking@entry=true) at /test/10.6_dbg/sql/sql_parse.cc:1404                                                                                                         
#17 0x00005577c86813d2 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x5577cb6c07c8, put_in_cache=put_in_cache@entry=true) at /test/10.6_dbg/sql/sql_connect.cc:1410                                              
#18 0x00005577c86819d7 in handle_one_connection (arg=arg@entry=0x5577cb6c07c8) at /test/10.6_dbg/sql/sql_connect.cc:1312                                                                                                                
#19 0x00005577c8b30776 in pfs_spawn_thread (arg=0x5577cb581608) at /test/10.6_dbg/storage/perfschema/pfs.cc:2201
#20 0x00001478cb333609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#21 0x00001478caf21293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.3.32 d9526ae60820d8b8d511f94edfff3ea2888766ca (Optimized)

InnoDB: Failing assertion: !srv_read_only_mode || m_ignore_read_only

10.3.32 d9526ae60820d8b8d511f94edfff3ea2888766ca (Optimized)

Core was generated by `/test/MD160821-mariadb-10.3.32-linux-x86_64-opt/bin/mysqld --no-defaults --lc-m'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x1496973f87c0 (LWP 260483))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00001496975b4859 in __GI_abort () at abort.c:79
#2  0x00005594b6d802da in ut_dbg_assertion_failed (expr=expr@entry=0x5594b77e6a28 "!srv_read_only_mode || m_ignore_read_only", file=file@entry=0x5594b77e6970 "/test/10.3_opt/storage/innobase/fsp/fsp0sysspace.cc", line=line@entry=428) at /test/10.3_opt/storage/innobase/ut/ut0dbg.cc:60
#3  0x00005594b6d9b1f0 in SysTablespace::create_file (this=0x5594b7e2a0e0 <srv_sys_space>, file=@0x5594b97669e0: {_vptr.Datafile = 0x5594b7bf98f0 <vtable for Datafile+16>, m_name = 0x5594b98181a0 "ibdata1", m_filepath = 0x5594b9818180 "./ibdata1", m_filename = 0x5594b9818182 "ibdata1", m_handle = {m_file = -1, m_psi = 0x0}, m_open_flags = OS_FILE_CREATE, m_size = 768, m_order = 0, m_type = SRV_NOT_RAW, m_space_id = 18446744073709551615, m_flags = 0, m_exists = false, m_is_valid = false, m_first_page_buf = 0x0, m_first_page = 0x0, m_last_os_error = 0, m_file_info = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}}}) at /test/10.3_opt/storage/innobase/fsp/fsp0sysspace.cc:428
#4  0x00005594b740d7a2 in SysTablespace::open_or_create (this=0x5594b7e2a0e0 <srv_sys_space>, is_temp=is_temp@entry=false, create_new_db=create_new_db@entry=true, sum_new_sizes=sum_new_sizes@entry=0x7fff69a44a40, flush_lsn=flush_lsn@entry=0x7fff69a44a30) at /test/10.3_opt/storage/innobase/fsp/fsp0sysspace.cc:865
#5  0x00005594b6d7778d in srv_start (create_new_db=<optimized out>) at /test/10.3_opt/storage/innobase/srv/srv0start.cc:1684
#6  0x00005594b721d657 in innodb_init (p=<optimized out>) at /test/10.3_opt/storage/innobase/handler/ha_innodb.cc:4310
#7  0x00005594b707bff0 in ha_initialize_handlerton (plugin=0x5594b9759dc8) at /test/10.3_opt/sql/handler.cc:549
#8  0x00005594b6e8f855 in plugin_initialize (tmp_root=tmp_root@entry=0x7fff69a49b00, plugin=plugin@entry=0x5594b9759dc8, argc=argc@entry=0x5594b7dac9a8 <remaining_argc>, argv=argv@entry=0x5594b9722c60, options_only=options_only@entry=false) at /test/10.3_opt/sql/sql_plugin.cc:1433
#9  0x00005594b6e90a02 in plugin_init (argc=argc@entry=0x5594b7dac9a8 <remaining_argc>, argv=0x5594b9722c60, flags=2) at /test/10.3_opt/sql/sql_plugin.cc:1715
#10 0x00005594b6dbf1cc in init_server_components () at /test/10.3_opt/sql/mysqld.cc:5467
#11 0x00005594b6dc53e0 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.3_opt/sql/mysqld.cc:6087
#12 0x00001496975b60b3 in __libc_start_main (main=0x5594b6da4590 <main(int, char**)>, argc=13, argv=0x7fff69a4a3c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff69a4a3b8) at ../csu/libc-start.c:308
#13 0x00005594b6db80ae in _start () at /test/10.3_opt/sql/sql_list.h:158

10.5.13 0268b8712288d46fbd8a43fdef6bada399b68dff (Optimized)

Core was generated by `/test/MD160821-mariadb-10.5.13-linux-x86_64-opt/bin/mysqld --no-defaults --lc-m'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14945a933800 (LWP 261456))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000014945aaef859 in __GI_abort () at abort.c:79
#2  0x00005641070f070a in os_file_handle_error_cond_exit (on_error_silent=false, should_abort=true, operation=0x564107ba1d0e "open", name=0x5641096b9218 "./ibdata1") at /test/10.5_opt/storage/innobase/os/os0file.cc:3197
#3  os_file_handle_error (operation=0x564107ba1d0e "open", name=0x5641096b9218 "./ibdata1") at /test/10.5_opt/storage/innobase/os/os0file.cc:210
#4  os_file_create_func (name=name@entry=0x5641096b9218 "./ibdata1", create_mode=create_mode@entry=52, purpose=purpose@entry=62, type=type@entry=100, read_only=read_only@entry=true, success=success@entry=0x7ffe7e7bc327) at /test/10.5_opt/storage/innobase/os/os0file.cc:1296
#5  0x0000564107966ffb in pfs_os_file_create_func (purpose=62, type=100, src_file=0x564107d73c48 "/test/10.5_opt/storage/innobase/fsp/fsp0file.cc", src_line=70, src_line=70, src_file=0x564107d73c48 "/test/10.5_opt/storage/innobase/fsp/fsp0file.cc", success=0x7ffe7e7bc327, read_only=true, type=100, purpose=62, create_mode=52, name=0x5641096b9218 "./ibdata1", key=<optimized out>) at /test/10.5_opt/storage/innobase/include/os0file.ic:167
#6  Datafile::open_or_create (this=this@entry=0x5641096b9248, read_only_mode=true) at /test/10.5_opt/storage/innobase/fsp/fsp0file.cc:70
#7  0x000056410796ad33 in SysTablespace::create_file (this=0x564108c745a0 <srv_sys_space>, file=@0x5641096b9248: {_vptr.Datafile = 0x5641081b2ab0 <vtable for Datafile+16>, m_name = 0x5641096b9368 "ibdata1", m_filepath = 0x5641096b9218 "./ibdata1", m_filename = 0x5641096b921a "ibdata1", m_handle = {m_file = -1, m_psi = 0x0}, m_open_flags = OS_FILE_CREATE, m_size = 768, m_order = 0, m_type = SRV_NOT_RAW, m_space_id = 18446744073709551615, m_flags = 21, m_exists = false, m_is_valid = false, m_first_page = 0x0, m_last_os_error = 0, m_file_info = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}}}) at /test/10.5_opt/storage/innobase/fsp/fsp0sysspace.cc:449
#8  0x000056410796b3e2 in SysTablespace::open_or_create (this=0x564108c745a0 <srv_sys_space>, is_temp=is_temp@entry=false, create_new_db=create_new_db@entry=true, sum_new_sizes=sum_new_sizes@entry=0x7ffe7e7bc608, flush_lsn=flush_lsn@entry=0x7ffe7e7bc600) at /test/10.5_opt/storage/innobase/fsp/fsp0sysspace.cc:868
#9  0x0000564107105539 in srv_start (create_new_db=<optimized out>) at /test/10.5_opt/storage/innobase/srv/srv0start.cc:1276
#10 0x00005641077774e2 in innodb_init (p=<optimized out>) at /test/10.5_opt/storage/innobase/handler/ha_innodb.cc:4023
#11 0x0000564107455022 in ha_initialize_handlerton (plugin=0x5641095f86a0) at /test/10.5_opt/sql/handler.cc:645
#12 0x000056410724d9ea in plugin_initialize (tmp_root=tmp_root@entry=0x7ffe7e7bf110, plugin=plugin@entry=0x5641095f86a0, argc=argc@entry=0x5641083a5d40 <remaining_argc>, argv=argv@entry=0x5641095bfbc8, options_only=options_only@entry=false) at /test/10.5_opt/sql/sql_plugin.cc:1459
#13 0x000056410724ecbb in plugin_init (argc=argc@entry=0x5641083a5d40 <remaining_argc>, argv=<optimized out>, flags=1) at /test/10.5_opt/sql/sql_plugin.cc:1751
#14 0x000056410716bcf7 in init_server_components () at /test/10.5_opt/sql/mysqld.cc:4889
#15 0x0000564107171819 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.5_opt/sql/mysqld.cc:5481
#16 0x000014945aaf10b3 in __libc_start_main (main=0x564107135970 <main(int, char**)>, argc=13, argv=0x7ffe7e7c1138, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe7e7c1128) at ../csu/libc-start.c:308
#17 0x000056410716671e in _start () at /test/10.5_opt/sql/mysqld.cc:4321

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Optimized)

Core was generated by `/test/MD160821-mariadb-10.7.0-linux-x86_64-opt/bin/mariadbd --no-defaults --lc-'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14c5fbd83800 (LWP 261606))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000014c5fbf3f859 in __GI_abort () at abort.c:79
#2  0x0000556939a62358 in os_file_handle_error_cond_exit (name=<optimized out>, operation=0x55693a4c6ecd "open", should_abort=<optimized out>, on_error_silent=<optimized out>) at /test/10.7_opt/storage/innobase/os/os0file.cc:3174
#3  0x000055693a158826 in os_file_handle_error (operation=0x55693a4c6ecd "open", name=0x55693c96c578 "./ibdata1") at /test/10.7_opt/storage/innobase/os/os0file.cc:1280
#4  os_file_create_func (name=name@entry=0x55693c96c578 "./ibdata1", create_mode=create_mode@entry=52, purpose=purpose@entry=62, type=type@entry=100, read_only=read_only@entry=true, success=success@entry=0x7ffc11d9fee7) at /test/10.7_opt/storage/innobase/os/os0file.cc:1280
#5  0x000055693a29c85b in pfs_os_file_create_func (purpose=62, type=100, src_file=0x55693a69e5f8 "/test/10.7_opt/storage/innobase/fsp/fsp0file.cc", src_line=53, src_line=53, src_file=0x55693a69e5f8 "/test/10.7_opt/storage/innobase/fsp/fsp0file.cc", success=0x7ffc11d9fee7, read_only=true, type=100, purpose=62, create_mode=52, name=0x55693c96c578 "./ibdata1", key=<optimized out>) at /test/10.7_opt/storage/innobase/include/os0file.ic:167
#6  Datafile::open_or_create (this=this@entry=0x55693c96c468, read_only_mode=true) at /test/10.7_opt/storage/innobase/fsp/fsp0file.cc:53
#7  0x000055693a2a0183 in SysTablespace::create_file (this=0x55693b5b4060 <srv_sys_space>, file=@0x55693c96c468: {_vptr.Datafile = 0x55693aae78b0 <vtable for Datafile+16>, m_filepath = 0x55693c96c578 "./ibdata1", m_filename = 0x55693c96c57a "ibdata1", m_handle = {m_file = -1, m_psi = 0x0}, m_open_flags = OS_FILE_CREATE, m_size = 768, m_order = 0, m_type = SRV_NOT_RAW, m_space_id = 4294967295, m_flags = 21, m_exists = false, m_is_valid = false, m_first_page = 0x0, m_last_os_error = 0, m_defer = false, m_file_info = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}}}) at /test/10.7_opt/storage/innobase/fsp/fsp0sysspace.cc:448
#8  0x000055693a2a0872 in SysTablespace::open_or_create (this=0x55693b5b4060 <srv_sys_space>, is_temp=is_temp@entry=false, create_new_db=create_new_db@entry=true, sum_new_sizes=sum_new_sizes@entry=0x7ffc11da01c8, flush_lsn=flush_lsn@entry=0x7ffc11da01c0) at /test/10.7_opt/storage/innobase/fsp/fsp0sysspace.cc:863
#9  0x0000556939a75a39 in srv_start (create_new_db=<optimized out>) at /test/10.7_opt/storage/innobase/srv/srv0start.cc:1260
#10 0x000055693a0f8752 in innodb_init (p=<optimized out>) at /test/10.7_opt/storage/innobase/handler/ha_innodb.cc:4151
#11 0x0000556939e19e52 in ha_initialize_handlerton (plugin=0x55693c8b17f0) at /test/10.7_opt/sql/handler.cc:659
#12 0x0000556939bf5528 in plugin_initialize (tmp_root=<optimized out>, plugin=0x55693c8b17f0, argc=0x55693aced0c0 <remaining_argc>, argv=<optimized out>, options_only=<optimized out>) at /test/10.7_opt/sql/sql_plugin.cc:1463
#13 0x0000556939bf66db in plugin_init (argc=argc@entry=0x55693aced0c0 <remaining_argc>, argv=<optimized out>, flags=1) at /test/10.7_opt/sql/sql_plugin.cc:1756
#14 0x0000556939ad82fc in init_server_components () at /test/10.7_opt/sql/mysqld.cc:5048
#15 0x0000556939addae3 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.7_opt/sql/mysqld.cc:5664
#16 0x000014c5fbf410b3 in __libc_start_main (main=0x556939aa2b40 <main(int, char**)>, argc=13, argv=0x7ffc11da56e8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc11da56d8) at ../csu/libc-start.c:308
#17 0x0000556939ad2b7e in _start () at /test/10.7_opt/sql/mysqld.cc:4480

10.6.5 9ac1ac006197c8979db1dc73f4e983f623e831e8 (Optimized)

Core was generated by `/test/MD160821-mariadb-10.6.5-linux-x86_64-opt/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x154fbc75c700 (LWP 269767))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000154fc7b87859 in __GI_abort () at abort.c:79
#2  0x000055d7b0d61957 in ib::fatal::~fatal (this=<optimized out>, __in_chrg=<optimized out>) at /test/10.6_opt/storage/innobase/ut/ut0ut.cc:519
#3  0x000055d7b0d41ef6 in log_t::file::write (this=0x55d7b2854780 <log_sys+192>, offset=<optimized out>, buf=<optimized out>) at /test/10.6_opt/storage/innobase/include/ut0ut.h:327
#4  0x000055d7b14299b3 in log_write_buf (new_data_offset=<optimized out>, start_lsn=8192, len=512, buf=0x154fbd7f3000 <error: Cannot access memory at address 0x154fbd7f3000>) at /test/10.6_opt/include/span.h:66
#5  log_write (rotate_key=<optimized out>) at /test/10.6_opt/storage/innobase/log/log0log.cc:755
#6  log_write_up_to (lsn=<optimized out>, flush_to_disk=<optimized out>, rotate_key=<optimized out>, callback=<optimized out>) at /test/10.6_opt/storage/innobase/log/log0log.cc:817
#7  0x000055d7b13e3ee6 in ha_innobase::extra (this=0x154f9401a970, operation=HA_EXTRA_END_ALTER_COPY) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:15414
#8  0x000055d7b0e5bb6f in select_insert::prepare_eof (this=this@entry=0x154f940131b0) at /test/10.6_opt/sql/handler.h:3545
#9  0x000055d7b0e5be26 in select_insert::send_eof (this=0x154f940131b0) at /test/10.6_opt/sql/sql_insert.cc:4268
#10 0x000055d7b0efe439 in do_select (procedure=<optimized out>, join=0x154f94013270) at /test/10.6_opt/sql/sql_select.cc:20659
#11 JOIN::exec_inner (this=0x154f94013270) at /test/10.6_opt/sql/sql_select.cc:4737
#12 0x000055d7b0efe8f8 in JOIN::exec (this=this@entry=0x154f94013270) at /test/10.6_opt/sql/sql_select.cc:4515
#13 0x000055d7b0efc99a in mysql_select (thd=0x154f94000c58, tables=0x154f94011df0, fields=<optimized out>, conds=0x154f94012680, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=<optimized out>, result=0x154f940131b0, unit=0x154f94004e00, select_lex=0x154f940112d8) at /test/10.6_opt/sql/sql_select.cc:4993
#14 0x000055d7b0efd167 in handle_select (thd=thd@entry=0x154f94000c58, lex=lex@entry=0x154f94004d38, result=result@entry=0x154f940131b0, setup_tables_done_option=setup_tables_done_option@entry=1073741824) at /test/10.6_opt/sql/sql_select.cc:545
#15 0x000055d7b0e9c155 in mysql_execute_command (thd=0x154f94000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:4711
#16 0x000055d7b0e885c6 in mysql_parse (thd=0x154f94000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8030
#17 0x000055d7b0e944c5 in dispatch_command (command=COM_QUERY, thd=0x154f94000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1357
#18 0x000055d7b0e963e7 in do_command (thd=0x154f94000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#19 0x000055d7b0fa4ef7 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1418
#20 0x000055d7b0fa523d in handle_one_connection (arg=arg@entry=0x55d7b4024608) at /test/10.6_opt/sql/sql_connect.cc:1312
#21 0x000055d7b132f778 in pfs_spawn_thread (arg=0x55d7b3be2928) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#22 0x0000154fc8096609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#23 0x0000154fc7c84293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Comment by Roel Van de Paar [ 2021-08-17 ]

A fix for this bug would be appreciated as it keeps creating different stacks (ref uniqueID list above).

Comment by Roel Van de Paar [ 2021-08-20 ]

# mysqld options required for replay:  --innodb-force-recovery=24
XA START 'a','a',0;
SELECT stat_value>0 FROM mysql.innodb_index_stats WHERE table_name LIKE 'a' IN (0);
SELECT * FROM information_schema.innodb_lock_waits;

Leads to yet another set of different crashes. This particular testcase does not crash 10.2 and may be due to a more recent change

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Debug)

Core was generated by `/test/MD160821-mariadb-10.7.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055c9b756f871 in <lambda(trx_t&)>::operator() (trx=..., 
    __closure=<optimized out>) at /usr/include/c++/9/atomic:250
250	      load(memory_order __m = memory_order_seq_cst) const noexcept
[Current thread is 1 (Thread 0x149d340e3700 (LWP 2303685))]
(gdb) bt
#0  0x000055c9b756f871 in <lambda(trx_t&)>::operator() (trx=@0x149d3419b030: {<ilist_node<void>> = {next = 0x55c9b820c780 <trx_sys+320>, prev = 0x55c9b820c780 <trx_sys+320>}, n_ref = {m_counter = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = 0}, <No data fields>}}, id = 0, mutex = {lock = {<std::__atomic_base<unsigned int>> = {static _S_alignment = 4, _M_i = 0}, <No data fields>}, static HOLDER = 2147483648}, mutex_owner = {<std::__atomic_base<unsigned long>> = {static _S_alignment = 8, _M_i = 0}, <No data fields>}, state = {m = {static _S_min_alignment = 4, static _S_alignment = 4, _M_i = TRX_STATE_ACTIVE}}, lock = {wait_lock = {m = {_M_b = {_M_p = 0x0}}}, wait_trx = 0x0, cond = pthread_cond_t = {
          Threads known to still execute a wait function = 0,
          Clock ID = CLOCK_REALTIME,
          Shared = No
        }, suspend_time = {m = {static _S_min_alignment = 8, static _S_alignment = 8, _M_i = {val = 0}}}, was_chosen_as_deadlock_victim = {m = {<std::__atomic_base<unsigned char>> = {static _S_alignment = 1, _M_i = 0 '\000'}, <No data fields>}}, rec_cached = 0 '\000', table_cached = 0 '\000', wait_thr = 0x0, rec_pool = {{lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}}, table_pool = {{trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}}, lock_heap = 0x55c9ba133878, trx_locks = {count = 0, start = 0x0, end = 0x0, node = &ib_lock_t::trx_locks, init = 51966}, table_locks = std::vector of length 0, capacity 0, evicted_tables = {count = 0, start = 0x0, end = 0x0, node = &dict_table_t::table_LRU, init = 51966}, n_rec_locks = 0}, wsrep = 0 '\000', read_view = {<ReadViewBase> = {m_low_limit_id = 0, m_up_limit_id = 0, m_ids = std::vector of length 0, capacity 0, m_low_limit_no = 0}, m_open = {_M_base = {static _S_alignment = 1, _M_i = false}}, m_mutex = {m_mutex = {global = pthread_mutex_t = {
              Type = Error check,
              Status = Not acquired,
              Robust = No,
              Shared = No,
              Protocol = None
            }, mutex = pthread_mutex_t = {
              Type = Normal,
              Status = Not acquired,
              Robust = No,
              Shared = No,
              Protocol = None
            }, file = 0x55c9b7bc0590 "/test/10.7_dbg/storage/innobase/include/read0types.h", name = 0x55c9b79ad9a1 "m_mutex", line = 219, count = 0, create_flags = 0, active_flags = 0, id = 0, thread = 0, locked_mutex = 0x0, used_mutex = 0x0, prev = 0x0, next = 0x0}, m_psi = 0x0}, m_creator_trx_id = 0}, is_recovered = false, op_info = 0x55c9b7b24b7f "", isolation_level = 0, check_foreigns = true, bulk_insert = false, is_registered = true, active_commit_ordered = false, check_unique_secondary = true, flush_log_later = false, must_flush_log_later = false, duplicates = 0, dict_operation = false, dict_operation_lock_mode = 0, start_time = 1629489348, start_time_micro = 1629489348607519, commit_lsn = 0, mysql_thd = 0x149d08000db8, mysql_log_file_name = 0x0, mysql_log_offset = 0, n_mysql_tables_in_use = 0, mysql_n_tables_locked = 0, error_state = DB_SUCCESS, error_info = 0x0, error_key_num = 18446744073709551615, graph = 0x0, trx_savepoints = {count = 0, start = 0x0, end = 0x0, node = &trx_named_savept_t::trx_savepoints, init = 51966}, undo_no = 0, last_sql_stat_start = {least_undo_no = 0}, rsegs = {m_redo = {rseg = 0x0, undo = 0x0}, m_noredo = {rseg = 0x0, undo = 0x0}}, roll_limit = 0, in_rollback = false, pages_undone = 0, n_autoinc_rows = 0, autoinc_locks = 0x149d08034958, read_only = true, auto_commit = false, will_lock = true, fts_trx = 0x0, fts_next_doc_id = 0, flush_tables = 0, start_line = 4669, start_file = 0x55c9b7be77b0 "/test/10.7_dbg/storage/innobase/row/row0sel.cc", xid = {formatID = -1, gtrid_length = 0, bqual_length = 0, data = '\000' <repeats 127 times>}, mod_tables = std::map with 0 elements, detailed_error = 0x55c9ba133758 "", rw_trx_hash_element = 0x0, rw_trx_hash_pins = 0x0, magic_n = 91118598}, __closure=<optimized out>) at /usr/include/c++/9/atomic:250
#1  thread_safe_trx_ilist_t::for_each<fetch_data_into_cache(trx_i_s_cache_t*)::<lambda(trx_t&)> > (callback=<optimized out>, this=<optimized out>) at /test/10.7_dbg/storage/innobase/include/trx0sys.h:828
#2  fetch_data_into_cache (cache=0x55c9b8b679e0 <trx_i_s_cache_static>) at /test/10.7_dbg/storage/innobase/trx/trx0i_s.cc:1214
#3  trx_i_s_possibly_fetch_data_into_cache (cache=cache@entry=0x55c9b8b679e0 <trx_i_s_cache_static>) at /test/10.7_dbg/storage/innobase/trx/trx0i_s.cc:1243
#4  0x000055c9b73d3b97 in trx_i_s_common_fill_table (thd=0x149d08000db8, tables=0x149d08014358) at /test/10.7_dbg/storage/innobase/handler/i_s.cc:973
#5  0x000055c9b6d979eb in get_schema_tables_result (join=join@entry=0x149d080155b0, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /test/10.7_dbg/sql/sql_show.cc:8837
#6  0x000055c9b6d69a77 in JOIN::exec_inner (this=this@entry=0x149d080155b0) at /test/10.7_dbg/sql/sql_string.h:682
#7  0x000055c9b6d6a718 in JOIN::exec (this=this@entry=0x149d080155b0) at /test/10.7_dbg/sql/sql_select.cc:4513
#8  0x000055c9b6d686f6 in mysql_select (thd=thd@entry=0x149d08000db8, tables=0x149d08014358, fields=@0x149d08013fe8: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x149d080142d8, last = 0x149d08016100, elements = 4}, <No data fields>}, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2685668096, result=0x149d08015588, unit=0x149d08005120, select_lex=0x149d08013d48) at /test/10.7_dbg/sql/sql_select.cc:4991
#9  0x000055c9b6d689ab in handle_select (thd=thd@entry=0x149d08000db8, lex=lex@entry=0x149d08005058, result=result@entry=0x149d08015588, setup_tables_done_option=setup_tables_done_option@entry=0) at /test/10.7_dbg/sql/sql_select.cc:545
#10 0x000055c9b6cca72e in execute_sqlcom_select (thd=thd@entry=0x149d08000db8, all_tables=0x149d08014358) at /test/10.7_dbg/sql/sql_parse.cc:6256
#11 0x000055c9b6cd75d1 in mysql_execute_command (thd=thd@entry=0x149d08000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.7_dbg/sql/sql_parse.cc:3946
#12 0x000055c9b6cc3ac3 in mysql_parse (thd=thd@entry=0x149d08000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x149d340e2400) at /test/10.7_dbg/sql/sql_parse.cc:8030
#13 0x000055c9b6cd26c8 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x149d08000db8, packet=packet@entry=0x149d0800b739 "SELECT * FROM information_schema.innodb_lock_waits", packet_length=packet_length@entry=50, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_class.h:1357
#14 0x000055c9b6cd5ae9 in do_command (thd=0x149d08000db8, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_parse.cc:1404
#15 0x000055c9b6e4bdd6 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55c9ba57cd68, put_in_cache=put_in_cache@entry=true) at /test/10.7_dbg/sql/sql_connect.cc:1418
#16 0x000055c9b6e4c3db in handle_one_connection (arg=arg@entry=0x55c9ba57cd68) at /test/10.7_dbg/sql/sql_connect.cc:1312
#17 0x000055c9b72b4ce4 in pfs_spawn_thread (arg=0x55c9ba499048) at /test/10.7_dbg/storage/perfschema/pfs.cc:2201
#18 0x0000149d35855609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#19 0x0000149d35443293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.6.5 9ac1ac006197c8979db1dc73f4e983f623e831e8 (Optimized)

Core was generated by `/test/MD160821-mariadb-10.6.5-linux-x86_64-opt/bin/mysqld --no-defaults --core-'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000556cdf0d7a74 in <lambda(trx_t&)>::operator() (
    __closure=<optimized out>, trx=...) at /usr/include/c++/9/atomic:250
250	      load(memory_order __m = memory_order_seq_cst) const noexcept
[Current thread is 1 (Thread 0x14fcc4cda700 (LWP 2311603))]
(gdb) bt
#0  0x0000556cdf0d7a74 in <lambda(trx_t&)>::operator() (__closure=<optimized out>, trx=@0x14fcc4f72030: {<ilist_node<void>> = {next = 0x556cdfb587c0 <trx_sys+192>, prev = 0x556cdfb587c0 <trx_sys+192>}, n_ref = {m_counter = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = 0}, <No data fields>}}, id = 0, mutex = {lock = {<std::__atomic_base<unsigned int>> = {static _S_alignment = 4, _M_i = 0}, <No data fields>}, static HOLDER = 2147483648}, state = {m = {static _S_min_alignment = 4, static _S_alignment = 4, _M_i = TRX_STATE_ACTIVE}}, lock = {wait_lock = {m = {_M_b = {_M_p = 0x0}}}, wait_trx = 0x0, cond = pthread_cond_t = {
          Threads known to still execute a wait function = 0,
          Clock ID = CLOCK_REALTIME,
          Shared = No
        }, suspend_time = {m = {static _S_min_alignment = 8, static _S_alignment = 8, _M_i = {val = 0}}}, was_chosen_as_deadlock_victim = {m = {<std::__atomic_base<unsigned char>> = {static _S_alignment = 1, _M_i = 0 '\000'}, <No data fields>}}, rec_cached = 0 '\000', table_cached = 0 '\000', wait_thr = 0x0, rec_pool = {{lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}, {lock = {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, pad = '\000' <repeats 255 times>}}, table_pool = {{trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}, {trx = 0x0, trx_locks = {prev = 0x0, next = 0x0}, index = 0x0, hash = 0x0, requested_time = 0, wait_time = 0, un_member = {tab_lock = {table = 0x0, locks = {prev = 0x0, next = 0x0}}, rec_lock = {page_id = {m_id = 0}, n_bits = 0}}, type_mode = 0}}, lock_heap = 0x556ce2369f38, trx_locks = {count = 0, start = 0x0, end = 0x0, node = &ib_lock_t::trx_locks}, table_locks = std::vector of length 0, capacity 0, evicted_tables = {count = 0, start = 0x0, end = 0x0, node = &dict_table_t::table_LRU}, n_rec_locks = 0}, wsrep = 0 '\000', read_view = {<ReadViewBase> = {m_low_limit_id = 0, m_up_limit_id = 0, m_ids = std::vector of length 0, capacity 0, m_low_limit_no = 0}, m_open = {_M_base = {static _S_alignment = 1, _M_i = false}}, m_mutex = {m_mutex = pthread_mutex_t = {
            Type = Normal,
            Status = Not acquired,
            Robust = No,
            Shared = No,
            Protocol = None
          }, m_psi = 0x0}, m_creator_trx_id = 0}, is_recovered = false, op_info = 0x556cdf52d2cb "", isolation_level = 0, check_foreigns = true, bulk_insert = false, is_registered = true, active_commit_ordered = false, check_unique_secondary = true, flush_log_later = false, must_flush_log_later = false, duplicates = 0, dict_operation = false, dict_operation_lock_mode = 0, start_time = 1629489349, start_time_micro = 1629489349978600, commit_lsn = 0, mysql_thd = 0x14fcac000c58, mysql_log_file_name = 0x0, mysql_log_offset = 0, n_mysql_tables_in_use = 0, mysql_n_tables_locked = 0, error_state = DB_SUCCESS, error_info = 0x0, error_key_num = 18446744073709551615, graph = 0x0, trx_savepoints = {count = 0, start = 0x0, end = 0x0, node = &trx_named_savept_t::trx_savepoints}, undo_no = 0, last_sql_stat_start = {least_undo_no = 0}, rsegs = {m_redo = {rseg = 0x0, undo = 0x0}, m_noredo = {rseg = 0x0, undo = 0x0}}, roll_limit = 0, in_rollback = false, pages_undone = 0, n_autoinc_rows = 0, autoinc_locks = 0x14fcac0300a0, read_only = true, auto_commit = false, will_lock = true, fts_trx = 0x0, fts_next_doc_id = 0, flush_tables = 0, xid = {formatID = -1, gtrid_length = 0, bqual_length = 0, data = '\000' <repeats 127 times>}, mod_tables = std::map with 0 elements, detailed_error = 0x556ce2369e18 "", rw_trx_hash_element = 0x0, rw_trx_hash_pins = 0x0, magic_n = 91118598}) at /usr/include/c++/9/atomic:250
#1  thread_safe_trx_ilist_t::for_each<fetch_data_into_cache(trx_i_s_cache_t*)::<lambda(trx_t&)> > (this=<optimized out>, callback=<optimized out>) at /test/10.6_opt/storage/innobase/include/trx0sys.h:828
#2  fetch_data_into_cache (cache=0x556ce04ab760 <trx_i_s_cache_static>) at /test/10.6_opt/storage/innobase/trx/trx0i_s.cc:1214
#3  trx_i_s_possibly_fetch_data_into_cache (cache=cache@entry=0x556ce04ab760 <trx_i_s_cache_static>) at /test/10.6_opt/storage/innobase/trx/trx0i_s.cc:1243
#4  0x0000556cdf020123 in trx_i_s_common_fill_table (thd=0x14fcac000c58, tables=0x14fcac010f18) at /test/10.6_opt/storage/innobase/handler/i_s.cc:973
#5  0x0000556cdeb32d5e in get_schema_tables_result (join=join@entry=0x14fcac012170, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /test/10.6_opt/sql/sql_show.cc:8837
#6  0x0000556cdeb1609f in JOIN::exec_inner (this=0x14fcac012170) at /test/10.6_opt/sql/sql_select.cc:4693
#7  0x0000556cdeb168f8 in JOIN::exec (this=this@entry=0x14fcac012170) at /test/10.6_opt/sql/sql_select.cc:4515
#8  0x0000556cdeb1499a in mysql_select (thd=0x14fcac000c58, tables=0x14fcac010f18, fields=<optimized out>, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=<optimized out>, result=0x14fcac012148, unit=0x14fcac004e00, select_lex=0x14fcac010908) at /test/10.6_opt/sql/sql_select.cc:4993
#9  0x0000556cdeb15167 in handle_select (thd=thd@entry=0x14fcac000c58, lex=lex@entry=0x14fcac004d38, result=result@entry=0x14fcac012148, setup_tables_done_option=setup_tables_done_option@entry=0) at /test/10.6_opt/sql/sql_select.cc:545
#10 0x0000556cdeaa55c1 in execute_sqlcom_select (thd=0x14fcac000c58, all_tables=0x14fcac010f18) at /test/10.6_opt/sql/sql_parse.cc:6256
#11 0x0000556cdeab3448 in mysql_execute_command (thd=0x14fcac000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:3946
#12 0x0000556cdeaa05c6 in mysql_parse (thd=0x14fcac000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8030
#13 0x0000556cdeaac4c5 in dispatch_command (command=COM_QUERY, thd=0x14fcac000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1357
#14 0x0000556cdeaae3e7 in do_command (thd=0x14fcac000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#15 0x0000556cdebbcef7 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1418
#16 0x0000556cdebbd23d in handle_one_connection (arg=arg@entry=0x556ce2776398) at /test/10.6_opt/sql/sql_connect.cc:1312
#17 0x0000556cdef47778 in pfs_spawn_thread (arg=0x556ce2334708) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#18 0x000014fcda07b609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#19 0x000014fcd9c69293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.4.22 46c3e7e3537c31a94289033bfeccf3faf8d4069e (Debug)

Core was generated by `/test/MD160821-mariadb-10.4.22-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00005646ff805b01 in fetch_data_into_cache (
    cache=0x564700595920 <trx_i_s_cache_static>)
    at /test/10.4_dbg/storage/innobase/trx/trx0i_s.cc:1283
[Current thread is 1 (Thread 0x151068055700 (LWP 2709294))]
(gdb) bt
#0  0x00005646ff805b01 in fetch_data_into_cache (cache=0x564700595920 <trx_i_s_cache_static>) at /test/10.4_dbg/storage/innobase/trx/trx0i_s.cc:1283
#1  trx_i_s_possibly_fetch_data_into_cache (cache=cache@entry=0x564700595920 <trx_i_s_cache_static>) at /test/10.4_dbg/storage/innobase/trx/trx0i_s.cc:1315
#2  0x00005646ff62847b in trx_i_s_common_fill_table (thd=0x151030000d90, tables=0x151030013918) at /test/10.4_dbg/storage/innobase/handler/i_s.cc:1204
#3  0x00005646ff127d8d in get_schema_tables_result (join=join@entry=0x151030014e30, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /test/10.4_dbg/sql/sql_show.cc:9040
#4  0x00005646ff103cfa in JOIN::exec_inner (this=this@entry=0x151030014e30) at /test/10.4_dbg/sql/sql_select.cc:4511
#5  0x00005646ff104904 in JOIN::exec (this=this@entry=0x151030014e30) at /test/10.4_dbg/sql/sql_select.cc:4316
#6  0x00005646ff1029f9 in mysql_select (thd=thd@entry=0x151030000d90, tables=0x151030013918, wild_num=1, fields=@0x151030013448: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x151030013898, last = 0x151030015928, elements = 4}, <No data fields>}, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=<optimized out>, result=0x151030014e08, unit=0x151030004cc0, select_lex=0x151030013300) at /test/10.4_dbg/sql/sql_select.cc:4754
#7  0x00005646ff102d22 in handle_select (thd=thd@entry=0x151030000d90, lex=lex@entry=0x151030004c00, result=result@entry=0x151030014e08, setup_tables_done_option=setup_tables_done_option@entry=0) at /test/10.4_dbg/sql/sql_select.cc:436
#8  0x00005646ff070f20 in execute_sqlcom_select (thd=thd@entry=0x151030000d90, all_tables=0x151030013918) at /test/10.4_dbg/sql/sql_parse.cc:6449
#9  0x00005646ff07c3cc in mysql_execute_command (thd=thd@entry=0x151030000d90) at /test/10.4_dbg/sql/sql_parse.cc:3963
#10 0x00005646ff086f60 in mysql_parse (thd=thd@entry=0x151030000d90, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x151068054490, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.4_dbg/sql/sql_parse.cc:7995
#11 0x00005646ff0898d2 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x151030000d90, packet=packet@entry=0x15103001a361 "SELECT * FROM information_schema.innodb_lock_waits", packet_length=packet_length@entry=50, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /test/10.4_dbg/sql/sql_class.h:1201
#12 0x00005646ff08d168 in do_command (thd=0x151030000d90) at /test/10.4_dbg/sql/sql_parse.cc:1373
#13 0x00005646ff1cafc9 in do_handle_one_connection (connect=connect@entry=0x564702bd8780) at /test/10.4_dbg/sql/sql_connect.cc:1420
#14 0x00005646ff1cb0e8 in handle_one_connection (arg=0x564702bd8780) at /test/10.4_dbg/sql/sql_connect.cc:1316
#15 0x0000151069940609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#16 0x000015106952e293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.3.32 d9526ae60820d8b8d511f94edfff3ea2888766ca (Optimized)

Core was generated by `/test/MD160821-mariadb-10.3.32-linux-x86_64-opt/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055c408ca08ab in fetch_data_into_cache (cache=<optimized out>)
    at /test/10.3_opt/storage/innobase/trx/trx0i_s.cc:1287
[Current thread is 1 (Thread 0x14a0680d0700 (LWP 2713133))]
(gdb) bt
#0  0x000055c408ca08ab in fetch_data_into_cache (cache=<optimized out>) at /test/10.3_opt/storage/innobase/trx/trx0i_s.cc:1287
#1  trx_i_s_possibly_fetch_data_into_cache (cache=cache@entry=0x55c40979a520 <trx_i_s_cache_static>) at /test/10.3_opt/storage/innobase/trx/trx0i_s.cc:1319
#2  0x000055c408bcd353 in trx_i_s_common_fill_table (thd=0x14a030000c48, tables=0x14a03000f8d8) at /test/10.3_opt/storage/innobase/handler/i_s.cc:1204
#3  0x000055c4088709d3 in get_schema_tables_result (join=join@entry=0x14a030010560, executed_place=executed_place@entry=PROCESSED_BY_JOIN_EXEC) at /test/10.3_opt/sql/sql_show.cc:8993
#4  0x000055c408855f47 in JOIN::exec_inner (this=this@entry=0x14a030010560) at /test/10.3_opt/sql/sql_select.cc:4104
#5  0x000055c408856696 in JOIN::exec (this=this@entry=0x14a030010560) at /test/10.3_opt/sql/sql_select.cc:3936
#6  0x000055c408856818 in mysql_select (thd=0x14a030000c48, tables=0x14a03000f8d8, wild_num=1, fields=<optimized out>, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=<optimized out>, result=0x14a030010538, unit=0x14a030004938, select_lex=0x14a0300050d0) at /test/10.3_opt/sql/sql_select.cc:4344
#7  0x000055c40885719b in handle_select (thd=thd@entry=0x14a030000c48, lex=lex@entry=0x14a030004878, result=result@entry=0x14a030010538, setup_tables_done_option=setup_tables_done_option@entry=0) at /test/10.3_opt/sql/sql_select.cc:372
#8  0x000055c4087e7451 in execute_sqlcom_select (thd=0x14a030000c48, all_tables=0x14a03000f8d8) at /test/10.3_opt/sql/sql_parse.cc:6339
#9  0x000055c4087f5181 in mysql_execute_command (thd=<optimized out>) at /test/10.3_opt/sql/sql_parse.cc:3870
#10 0x000055c4087f7ce7 in mysql_parse (thd=0x14a030000c48, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /test/10.3_opt/sql/sql_parse.cc:7870
#11 0x000055c4087fa085 in dispatch_command (command=COM_QUERY, thd=0x14a030000c48, packet=<optimized out>, packet_length=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /test/10.3_opt/sql/sql_class.h:1152
#12 0x000055c4087fc07d in do_command (thd=0x14a030000c48) at /test/10.3_opt/sql/sql_parse.cc:1398
#13 0x000055c4088e4556 in do_handle_one_connection (connect=connect@entry=0x55c40b9bdd18) at /test/10.3_opt/sql/sql_connect.cc:1403
#14 0x000055c4088e472f in handle_one_connection (arg=0x55c40b9bdd18) at /test/10.3_opt/sql/sql_connect.cc:1308
#15 0x000014a06bfab609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#16 0x000014a06bed2293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Comment by Roel Van de Paar [ 2021-08-21 ]

# mysqld options required for replay:  --innodb-force-recovery=6
USE test;
SET GLOBAL innodb_log_checkpoint_now=TRUE;

Comment by Roel Van de Paar [ 2021-08-23 ]

That last testcase can also produce this stack (possibly sporadically or only when the C connector is used)

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Debug)

mysqld: /test/10.7_dbg/storage/innobase/fil/fil0fil.cc:1362: dberr_t fil_write_flushed_lsn(lsn_t): Assertion `!srv_read_only_mode' failed.

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Debug)

Core was generated by `/test/MD160821-mariadb-10.7.0-linux-x86_64-dbg/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x1539d4117700 (LWP 323874))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00001539e8c02859 in __GI_abort () at abort.c:79
#2  0x00001539e8c02729 in __assert_fail_base (fmt=0x1539e8d98588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55e267a583b6 "!srv_read_only_mode", file=0x55e267ad0aa0 "/test/10.7_dbg/storage/innobase/fil/fil0fil.cc", line=1362, function=<optimized out>) at assert.c:92
#3  0x00001539e8c13f36 in __GI___assert_fail (assertion=assertion@entry=0x55e267a583b6 "!srv_read_only_mode", file=file@entry=0x55e267ad0aa0 "/test/10.7_dbg/storage/innobase/fil/fil0fil.cc", line=line@entry=1362, function=function@entry=0x55e267ad1d58 "dberr_t fil_write_flushed_lsn(lsn_t)") at assert.c:101
#4  0x000055e26757dc71 in fil_write_flushed_lsn (lsn=8204) at /test/10.7_dbg/storage/innobase/fil/fil0fil.cc:1362
#5  0x000055e2672293bc in checkpoint_now_set (save=<optimized out>) at /test/10.7_dbg/storage/innobase/handler/ha_innodb.cc:17996
#6  0x000055e266bb07e9 in sys_var_pluginvar::global_update (this=0x55e26a31c9f0, thd=0x1539b0000db8, var=0x1539b0013ef8) at /test/10.7_dbg/sql/sql_plugin.cc:3622
#7  0x000055e266a8ebd4 in sys_var::update (this=0x55e26a31c9f0, thd=0x1539b0000db8, var=0x1539b0013ef8) at /test/10.7_dbg/sql/set_var.cc:207
#8  0x000055e266a8f101 in set_var::update (this=<optimized out>, thd=<optimized out>) at /test/10.7_dbg/sql/set_var.cc:863
#9  0x000055e266a90447 in sql_set_variables (thd=thd@entry=0x1539b0000db8, var_list=var_list@entry=0x1539b0006028, free=free@entry=true) at /test/10.7_dbg/sql/set_var.cc:745
#10 0x000055e266b94505 in mysql_execute_command (thd=thd@entry=0x1539b0000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.7_dbg/sql/sql_parse.cc:5036
#11 0x000055e266b7dac3 in mysql_parse (thd=thd@entry=0x1539b0000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x1539d4116400) at /test/10.7_dbg/sql/sql_parse.cc:8030
#12 0x000055e266b8c6c8 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x1539b0000db8, packet=packet@entry=0x1539b0370669 "SET GLOBAL innodb_log_checkpoint_now=TRUE;", packet_length=packet_length@entry=42, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_class.h:1357
#13 0x000055e266b8fae9 in do_command (thd=0x1539b0000db8, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_parse.cc:1404
#14 0x000055e266d05dd6 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55e26a7bd118, put_in_cache=put_in_cache@entry=true) at /test/10.7_dbg/sql/sql_connect.cc:1418
#15 0x000055e266d063db in handle_one_connection (arg=arg@entry=0x55e26a7bd118) at /test/10.7_dbg/sql/sql_connect.cc:1312
#16 0x000055e26716ece4 in pfs_spawn_thread (arg=0x55e26a6b0798) at /test/10.7_dbg/storage/perfschema/pfs.cc:2201
#17 0x00001539e9111609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#18 0x00001539e8cff293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Comment by Roel Van de Paar [ 2021-08-23 ]

Sometimes (sporadically) also seeing this stack

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Optimized)

Core was generated by `/test/MD160821-mariadb-10.7.0-linux-x86_64-opt/bin/mariadbd --no-defaults --lc-'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x153be909c800 (LWP 900636))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000153be9258859 in __GI_abort () at abort.c:79
#2  0x000056224a285358 in os_file_handle_error_cond_exit (name=<optimized out>, operation=0x56224ace9ecd "open", should_abort=<optimized out>, on_error_silent=<optimized out>) at /test/10.7_opt/storage/innobase/os/os0file.cc:3174
#3  0x000056224a97b826 in os_file_handle_error (operation=0x56224ace9ecd "open", name=0x56224cae9578 "./ibdata1") at /test/10.7_opt/storage/innobase/os/os0file.cc:1280
#4  os_file_create_func (name=name@entry=0x56224cae9578 "./ibdata1", create_mode=create_mode@entry=52, purpose=purpose@entry=62, type=type@entry=100, read_only=read_only@entry=true, success=success@entry=0x7ffd00edc7e7) at /test/10.7_opt/storage/innobase/os/os0file.cc:1280
#5  0x000056224aabf85b in pfs_os_file_create_func (purpose=62, type=100, src_file=0x56224aec15f8 "/test/10.7_opt/storage/innobase/fsp/fsp0file.cc", src_line=53, src_line=53, src_file=0x56224aec15f8 "/test/10.7_opt/storage/innobase/fsp/fsp0file.cc", success=0x7ffd00edc7e7, read_only=true, type=100, purpose=62, create_mode=52, name=0x56224cae9578 "./ibdata1", key=<optimized out>) at /test/10.7_opt/storage/innobase/include/os0file.ic:167
#6  Datafile::open_or_create (this=this@entry=0x56224cae9468, read_only_mode=true) at /test/10.7_opt/storage/innobase/fsp/fsp0file.cc:53
#7  0x000056224aac3183 in SysTablespace::create_file (this=0x56224bdd7060 <srv_sys_space>, file=@0x56224cae9468: {_vptr.Datafile = 0x56224b30a8b0 <vtable for Datafile+16>, m_filepath = 0x56224cae9578 "./ibdata1", m_filename = 0x56224cae957a "ibdata1", m_handle = {m_file = -1, m_psi = 0x0}, m_open_flags = OS_FILE_CREATE, m_size = 768, m_order = 0, m_type = SRV_NOT_RAW, m_space_id = 4294967295, m_flags = 21, m_exists = false, m_is_valid = false, m_first_page = 0x0, m_last_os_error = 0, m_defer = false, m_file_info = {st_dev = 0, st_ino = 0, st_nlink = 0, st_mode = 0, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 0, st_blocks = 0, st_atim = {tv_sec = 0, tv_nsec = 0}, st_mtim = {tv_sec = 0, tv_nsec = 0}, st_ctim = {tv_sec = 0, tv_nsec = 0}, __glibc_reserved = {0, 0, 0}}}) at /test/10.7_opt/storage/innobase/fsp/fsp0sysspace.cc:448
#8  0x000056224aac3872 in SysTablespace::open_or_create (this=0x56224bdd7060 <srv_sys_space>, is_temp=is_temp@entry=false, create_new_db=create_new_db@entry=true, sum_new_sizes=sum_new_sizes@entry=0x7ffd00edcac8, flush_lsn=flush_lsn@entry=0x7ffd00edcac0) at /test/10.7_opt/storage/innobase/fsp/fsp0sysspace.cc:863
#9  0x000056224a298a39 in srv_start (create_new_db=<optimized out>) at /test/10.7_opt/storage/innobase/srv/srv0start.cc:1260
#10 0x000056224a91b752 in innodb_init (p=<optimized out>) at /test/10.7_opt/storage/innobase/handler/ha_innodb.cc:4151
#11 0x000056224a63ce52 in ha_initialize_handlerton (plugin=0x56224ca2e7f0) at /test/10.7_opt/sql/handler.cc:659
#12 0x000056224a418528 in plugin_initialize (tmp_root=<optimized out>, plugin=0x56224ca2e7f0, argc=0x56224b5100c0 <remaining_argc>, argv=<optimized out>, options_only=<optimized out>) at /test/10.7_opt/sql/sql_plugin.cc:1463
#13 0x000056224a4196db in plugin_init (argc=argc@entry=0x56224b5100c0 <remaining_argc>, argv=<optimized out>, flags=1) at /test/10.7_opt/sql/sql_plugin.cc:1756
#14 0x000056224a2fb2fc in init_server_components () at /test/10.7_opt/sql/mysqld.cc:5048
#15 0x000056224a300ae3 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/10.7_opt/sql/mysqld.cc:5664
#16 0x0000153be925a0b3 in __libc_start_main (main=0x56224a2c5b40 <main(int, char**)>, argc=13, argv=0x7ffd00ee1fd8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd00ee1fc8) at ../csu/libc-start.c:308
#17 0x000056224a2f5b7e in _start () at /test/10.7_opt/sql/mysqld.cc:4480

Comment by Roel Van de Paar [ 2021-08-28 ]

Yet another stack which can be produced with:

# mysqld options required for replay:  --innodb-force-recovery=254
INSERT INTO mysql.innodb_table_stats SELECT database_name,''AS table_name,laST_UPDATE,0 AS n_rows,clustered_index_size,sum_of_other_index_sizes FROM mysql.innodb_table_stats WHERE table_name='';

However apparently only in given certain conditions, for example sporadically (with other non-hits resulting in the usual SIGABRT|ib::fatal::~fatal|log_t::file::write|log_write_buf|log_write instead), and possibly only when using C interface (not direct CLI) and/or with high server load. Testcase is confirmed to be able to produce the stack below.

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Debug)

mysqld: /test/10.7_dbg/storage/innobase/log/log0log.cc:841: void log_buffer_flush_to_disk(bool): Assertion `!srv_read_only_mode' failed.

10.7.0 52505bf20de0ce77a5c0b0a74af021051987bb0d (Debug)

Core was generated by `/test/MD160821-mariadb-10.7.0-linux-x86_64-dbg/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14b8f06bc700 (LWP 2384001))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000014b9065b1859 in __GI_abort () at abort.c:79
#2  0x000014b9065b1729 in __assert_fail_base (fmt=0x14b906747588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x558b878b93b6 "!srv_read_only_mode", file=0x558b878ded58 "/test/10.7_dbg/storage/innobase/log/log0log.cc", line=841, function=<optimized out>) at assert.c:92
#3  0x000014b9065c2f36 in __GI___assert_fail (assertion=assertion@entry=0x558b878b93b6 "!srv_read_only_mode", file=file@entry=0x558b878ded58 "/test/10.7_dbg/storage/innobase/log/log0log.cc", line=line@entry=841, function=function@entry=0x558b878df768 "void log_buffer_flush_to_disk(bool)") at assert.c:101
#4  0x0000558b87132a46 in log_buffer_flush_to_disk (sync=sync@entry=true) at /test/10.7_dbg/storage/innobase/log/log0log.cc:841
#5  0x0000558b8709d55b in ha_innobase::extra (this=0x14b8cc763aa0, operation=HA_EXTRA_END_ALTER_COPY) at /test/10.7_dbg/storage/innobase/handler/ha_innodb.cc:15409
#6  0x0000558b869a73eb in select_insert::prepare_eof (this=this@entry=0x14b8cc016650) at /test/10.7_dbg/sql/sql_insert.cc:4169
#7  0x0000558b869a7a56 in select_insert::send_eof (this=0x14b8cc016650) at /test/10.7_dbg/sql/sql_insert.cc:4268
#8  0x0000558b86a853cc in do_select (procedure=<optimized out>, join=0x14b8cc016710) at /test/10.7_dbg/sql/sql_select.cc:20657
#9  JOIN::exec_inner (this=this@entry=0x14b8cc016710) at /test/10.7_dbg/sql/sql_select.cc:4735
#10 0x0000558b86a85718 in JOIN::exec (this=this@entry=0x14b8cc016710) at /test/10.7_dbg/sql/sql_select.cc:4513
#11 0x0000558b86a836f6 in mysql_select (thd=thd@entry=0x14b8cc000db8, tables=0x14b8cc015260, fields=@0x14b8cc0149b8: {<base_list> = {<Sql_alloc> = {<No data fields>}, first = 0x14b8cc014cb8, last = 0x14b8cc0151f0, elements = 6}, <No data fields>}, conds=0x14b8cc015b20, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2202312376832, result=0x14b8cc016650, unit=0x14b8cc005120, select_lex=0x14b8cc014718) at /test/10.7_dbg/sql/sql_select.cc:4991
#12 0x0000558b86a839ab in handle_select (thd=thd@entry=0x14b8cc000db8, lex=lex@entry=0x14b8cc005058, result=result@entry=0x14b8cc016650, setup_tables_done_option=setup_tables_done_option@entry=1073741824) at /test/10.7_dbg/sql/sql_select.cc:545
#13 0x0000558b869f46ae in mysql_execute_command (thd=thd@entry=0x14b8cc000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.7_dbg/sql/sql_parse.cc:4711
#14 0x0000558b869deac3 in mysql_parse (thd=thd@entry=0x14b8cc000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14b8f06bb400) at /test/10.7_dbg/sql/sql_parse.cc:8030
#15 0x0000558b869ed6c8 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14b8cc000db8, packet=packet@entry=0x14b8cc00b859 "INSERT INTO mysql.innodb_table_stats SELECT database_name, 'stats_rename2' AS table_name, last_update, 123 AS n_rows, clustered_index_size, sum_of_other_index_sizes FROM mysql.innodb_table_stats WHERE"..., packet_length=packet_length@entry=230, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_class.h:1357
#16 0x0000558b869f0ae9 in do_command (thd=0x14b8cc000db8, blocking=blocking@entry=true) at /test/10.7_dbg/sql/sql_parse.cc:1404
#17 0x0000558b86b66dd6 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x558b8ac4d118, put_in_cache=put_in_cache@entry=true) at /test/10.7_dbg/sql/sql_connect.cc:1418
#18 0x0000558b86b673db in handle_one_connection (arg=arg@entry=0x558b8ac4d118) at /test/10.7_dbg/sql/sql_connect.cc:1312
#19 0x0000558b86fcfce4 in pfs_spawn_thread (arg=0x558b8ab40798) at /test/10.7_dbg/storage/perfschema/pfs.cc:2201
#20 0x000014b906ac0609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#21 0x000014b9066ae293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Comment by Marko Mäkelä [ 2021-09-17 ]

The innodb_force_recovery option is not supposed to be set while installing a database. Any value larger than 3 should block any writes inside InnoDB, but apparently it is not being blocked at the high level. The innodb_read_only parameter is similar.

What is the suggested fix? Ignore these parameters in bootstrap mode?

Comment by Roel Van de Paar [ 2021-09-17 ]

marko I think that's a great idea. Do you see any caveats/unforeseen issues?

Comment by Marko Mäkelä [ 2021-10-20 ]

serg, I think that this may need your resolution. I am inclined to think that we should return an error on bootstrap if either innodb_force_recovery or innodb_read_only are set.

Comment by Marko Mäkelä [ 2021-10-25 ]

It turns out that we already refuse bootstrap (and exit with status 1) if innodb_read_only is set.

../sql/mysqld --no-defaults --datadir=/dev/shm/10.2/mysql-test/var/install.db --innodb --innodb-read-only --bootstrap

10.2 a441a569157bf75303e3f9f485211c4b5d4f6129

2021-10-25 14:50:26 139916308961216 [Warning] InnoDB: Unable to open "./ib_logfile0" to check native AIO read support.
2021-10-25 14:50:26 139916308961216 [Warning] InnoDB: Linux Native AIO disabled.
2021-10-25 14:50:26 139916308961216 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-10-25 14:50:26 139916308961216 [Note] InnoDB: Completed initialization of buffer pool
2021-10-25 14:50:26 139916308961216 [ERROR] InnoDB: Can't create file './ibdata1' when --innodb-read-only is set
2021-10-25 14:50:26 139916308961216 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[1873] with error Generic error
2021-10-25 14:50:26 139916308961216 [Note] InnoDB: Starting shutdown...
2021-10-25 14:50:26 139916308961216 [ERROR] Plugin 'InnoDB' init function returned error.
2021-10-25 14:50:26 139916308961216 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2021-10-25 14:50:26 139916308961216 [Note] Plugin 'FEEDBACK' is disabled.
2021-10-25 14:50:26 139916308961216 [ERROR] Unknown/unsupported storage engine: InnoDB
2021-10-25 14:50:26 139916308961216 [ERROR] Aborting

We only have to do the same if innodb_force_recovery is set to a nonzero value.

Comment by Roel Van de Paar [ 2021-10-26 ]

Thank you marko!

Generated at Thu Feb 08 09:21:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.