[MDEV-26381] InnoDB: Failing assertion: fsize != os_offset_t(-1) on CREATE TABLE with many partitions / ENOMEM (out of kernel memory) from fstat handling - Jira

Roel Van de Paar created issue - 2021-08-17 06:13

Roel Van de Paar made changes - 2021-08-17 06:14

Field	Original Value	New Value
Attachment		MDEV-26381_1.sql [ 58569 ]

Roel Van de Paar made changes - 2021-08-17 06:14

Description

The attached (3 line) testcase is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). There is a core file and Marko will analyze that one.

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

_The attached (3 line) testcase (~~MDEV-26381~~_1.sql) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

Roel Van de Paar made changes - 2021-08-17 06:15

Attachment

MDEV-26381_2.sql [ 58570 ]

Roel Van de Paar made changes - 2021-08-17 06:33

Description

_The attached (3 line) testcase (~~MDEV-26381~~_1.sql) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

_The attached (3 line) testcase (~~MDEV-26381~~_1.sql) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

Roel Van de Paar made changes - 2021-08-17 06:36

Summary

InnoDB: Failing assertion: fsize != os_offset_t(-1)

InnoDB: Failing assertion: fsize != os_offset_t(-1) on CREATE TABLE with many partitions

Roel Van de Paar made changes - 2021-08-17 06:37

Description

_The attached (3 line) testcase (~~MDEV-26381~~_1.sql) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

_The attached (3 line) testcase (~~MDEV-26381~~_1.sql) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). As discussed elsewhere, the issue is likely not confined to partitioning but is rather related to the accessing of many .ibd files whilst running into a [bad] file descriptor issue,. There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

Marko Mäkelä made changes - 2021-08-17 15:15

Labels

need_feedback need_rr

Roel Van de Paar made changes - 2021-08-19 10:30

Assignee

Marko Mäkelä [ marko ]

Roel Van de Paar [ roel ]

Roel Van de Paar made changes - 2021-09-25 01:51

Labels

need_feedback need_rr

Roel Van de Paar made changes - 2021-09-25 01:51

Assignee

Roel Van de Paar [ roel ]

Marko Mäkelä [ marko ]

Roel Van de Paar made changes - 2021-09-25 01:51

Labels

need_rr

Roel Van de Paar made changes - 2021-10-13 23:10

Attachment

MDEV-26381_3.sql [ 59748 ]

Roel Van de Paar made changes - 2021-10-13 23:32

Attachment

MDEV-26381_4.sql [ 59749 ]

Roel Van de Paar made changes - 2021-10-13 23:39

Description

_The attached (3 line) testcase (~~MDEV-26381~~_1.sql) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). As discussed elsewhere, the issue is likely not confined to partitioning but is rather related to the accessing of many .ibd files whilst running into a [bad] file descriptor issue,. There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

_The attached (3 line) testcase ( [^~~MDEV-26381~~_1.sql] ) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). As discussed elsewhere, the issue is likely not confined to partitioning but is rather related to the accessing of many .ibd files whilst running into a [bad] file descriptor issue,. There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

Roel Van de Paar made changes - 2021-10-13 23:49

Attachment

MDEV-26381_OTHER_1.sql [ 59750 ]

Roel Van de Paar made changes - 2021-10-13 23:55

Attachment

~~MDEV-26381~~_OTHER_1.sql [ 59750 ]

Roel Van de Paar made changes - 2021-10-13 23:55

Attachment

MDEV-26381_OTHER_1.sql [ 59751 ]

Roel Van de Paar made changes - 2021-10-13 23:59

Link

This issue relates to ~~MDEV-25215~~ [ ~~MDEV-25215~~ ]

Roel Van de Paar made changes - 2021-10-14 00:16

Attachment

MDEV-26381_OTHER_2.sql [ 59752 ]

Roel Van de Paar made changes - 2021-10-14 00:22

Link

This issue relates to ~~MDEV-26293~~ [ ~~MDEV-26293~~ ]

Roel Van de Paar made changes - 2021-10-14 00:36

Labels

need_rr

hang need_rr

Roel Van de Paar made changes - 2021-10-14 00:42

Attachment		MDEV-26381_OTHER_2.threads.opt.log [ 59753 ]
Attachment		MDEV-26381_OTHER_1.threads.dbg.log [ 59754 ]

Roel Van de Paar made changes - 2021-10-14 00:45

Attachment

MDEV-26381_OTHER_1.threads_partial_unlock.dbg.log [ 59755 ]

Marko Mäkelä made changes - 2021-10-15 09:24

Labels

hang need_rr

Roel Van de Paar made changes - 2021-10-15 09:27

Assignee

Marko Mäkelä [ marko ]

Roel Van de Paar [ roel ]

Roel Van de Paar made changes - 2021-10-15 09:29

Assignee

Roel Van de Paar [ roel ]

Marko Mäkelä [ marko ]

Roel Van de Paar made changes - 2021-10-15 09:31

Summary

InnoDB: Failing assertion: fsize != os_offset_t(-1) on CREATE TABLE with many partitions

InnoDB: Failing assertion: fsize != os_offset_t(-1) on CREATE TABLE with many partitions / ENOMEM (out of kernel memory) from fstat handling

Roel Van de Paar made changes - 2021-10-15 09:31

Priority

Critical [ 2 ]

Major [ 3 ]

Roel Van de Paar made changes - 2021-10-15 09:32

Fix Version/s

10.6 [ 24028 ]

Roel Van de Paar made changes - 2021-10-15 09:32

Description

_The attached (3 line) testcase ( [^~~MDEV-26381~~_1.sql] ) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). As discussed elsewhere, the issue is likely not confined to partitioning but is rather related to the accessing of many .ibd files whilst running into a [bad] file descriptor issue,. There is a core file and Marko will analyze that one._

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

_The attached (3 line) testcase ( [^~~MDEV-26381~~_1.sql] ) is not finalized. Whilst I am confident that the said testcase can produce the assertion as-is (it was reduced to this based on seeing the assert or not), reproducing the crash itself is not straightforward. It may be highly sporadic, or more likely dependent on some system condition (like running out of file descriptors, some OS shortcoming, or similar). As discussed elsewhere, the issue is likely not confined to partitioning but is rather related to the accessing of many .ibd files whilst running into a [bad] file descriptor issue,. -There is a core file and Marko will analyze that one.- [DONE]_

Possibly related: ~~MDEV-20648~~ (?), ~~MDEV-26293~~ (likely)

mysqld options from original run. Not all necessary.
{noformat}
--max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode=ONLY_FULL_GROUP_BY --log_bin --innodb_file_per_table=1 --innodb_flush_method=O_DIRECT --table-cache=2 --symbolic-links=0 --innodb-log-checksums=0
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
InnoDB: Failing assertion: fsize != os_offset_t(-1)
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
Core was generated by `/test/MD260721-mariadb-10.6.4-linux-x86_64-opt/bin/mysqld --no-defaults --max_a'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14e10c449700 (LWP 3154497))]
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x000014e10d840859 in __GI_abort () at abort.c:79
#2 0x000055b95dd66b11 in ut_dbg_assertion_failed (expr=expr@entry=0x55b95e9746b6 "fsize != os_offset_t(-1)", file=file@entry=0x55b95e973e70 "/test/10.6_opt/storage/innobase/fil/fil0fil.cc", line=line@entry=591) at /test/10.6_opt/storage/innobase/ut/ut0dbg.cc:60
#3 0x000055b95dd77806 in fil_space_extend_must_retry (space=space@entry=0x14e070cdb488, node=0x14e070cdb5e8, size=size@entry=6, success=success@entry=0x14e10c444bdf) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:591
#4 0x000055b95dd77b45 in fil_space_extend (space=space@entry=0x14e070cdb488, size=6) at /test/10.6_opt/storage/innobase/fil/fil0fil.cc:705
#5 0x000055b95dd7ab4d in fsp_try_extend_data_file_with_pages (space=0x14e070cdb488, page_no=<optimized out>, header=0x14e0ec018480, mtr=0x14e10c445000) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:625
#6 0x000055b95e58544d in fseg_create (space=space@entry=0x14e070cdb488, byte_offset=byte_offset@entry=74, mtr=mtr@entry=0x14e10c445000, has_done_reservation=has_done_reservation@entry=false, block=block@entry=0x14e0ec0bcb80) at /test/10.6_opt/storage/innobase/fsp/fsp0fsp.cc:1698
#7 0x000055b95e4f7474 in btr_create (type=1, space=0x14e070cdb488, index_id=4850, index=index@entry=0x14e070a11db0, mtr=mtr@entry=0x14e10c445000) at /test/10.6_opt/storage/innobase/btr/btr0btr.cc:1031
#8 0x000055b95e5477ad in dict_create_index_tree_step (node=0x14e07813a200) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:768
#9 0x000055b95e548177 in dict_create_index_step (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/dict/dict0crea.cc:1249
#10 0x000055b95e467488 in que_thr_step (thr=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:663
#11 que_run_threads_low (thr=<optimized out>) at /test/10.6_opt/storage/innobase/que/que0que.cc:709
#12 que_run_threads (thr=thr@entry=0x14e07105f5c0) at /test/10.6_opt/storage/innobase/que/que0que.cc:729
#13 0x000055b95e494d10 in row_create_index_for_mysql (index=<optimized out>, trx=0x14e10c750330, field_lengths=field_lengths@entry=0x0, mode=FIL_ENCRYPTION_DEFAULT, key_id=<optimized out>) at /test/10.6_opt/storage/innobase/row/row0mysql.cc:2394
#14 0x000055b95e3eb653 in create_table_info_t::create_table (this=0x14e10c445920, create_fk=<optimized out>) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:12603
#15 0x000055b95e3ef791 in ha_innobase::create (this=<optimized out>, name=0x14e10c4460e0 "./test/t2#P#p4560", form=<optimized out>, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14e10c750330) at /test/10.6_opt/storage/innobase/handler/ha_innodb.cc:13135
#16 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=name@entry=0x14e10c4460e0 "./test/t2#P#p4560", form=form@entry=0x14e10c446420, info_arg=info_arg@entry=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#17 0x000055b95e32ab40 in ha_partition::create (this=0x14e079158bd0, name=0x14e10c447780 "./test/t2", table_arg=0x14e10c446420, create_info=0x14e10c447c70) at /test/10.6_opt/sql/ha_partition.cc:786
#18 0x000055b95e0d49c4 in handler::ha_create (this=<optimized out>, name=<optimized out>, form=0x14e10c446420, info_arg=0x14e10c447c70) at /test/10.6_opt/sql/handler.cc:5405
#19 0x000055b95e0d52ca in ha_create_table (thd=thd@entry=0x14e078000c58, path=<optimized out>, db=0x14e078011030 "test", table_name=0x14e078010918 "t2", create_info=create_info@entry=0x14e10c447c70, frm=frm@entry=0x14e10c447770, skip_frm_file=false) at /test/10.6_opt/sql/handler.cc:5870
#20 0x000055b95df3f655 in create_table_impl (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=<optimized out>, orig_db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, orig_table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, db=@0x14e078010968: {str = 0x14e078011030 "test", length = 4}, table_name=@0x14e078010978: {str = 0x14e078010918 "t2", length = 2}, path=@0x14e10c447760: {str = 0x14e10c447780 "./test/t2", length = 9}, options={m_options = DDL_options_st::OPT_NONE}, create_info=0x14e10c447c70, alter_info=0x14e10c447b80, create_table_mode=0, is_trans=0x14e10c447a1f, key_info=0x14e10c447758, key_count=0x14e10c447754, frm=0x14e10c447770) at /test/10.6_opt/sql/sql_table.cc:4447
#21 0x000055b95df3ffd8 in mysql_create_table_no_lock (thd=thd@entry=0x14e078000c58, ddl_log_state_create=ddl_log_state_create@entry=0x14e10c447a20, ddl_log_state_rm=ddl_log_state_rm@entry=0x14e10c447a40, db=db@entry=0x14e078010968, table_name=table_name@entry=0x14e078010978, create_info=create_info@entry=0x14e10c447c70, alter_info=0x14e10c447b80, is_trans=0x14e10c447a1f, create_table_mode=0, table_list=0x14e078010950) at /test/10.6_opt/sql/sql_table.cc:4546
#22 0x000055b95df403c4 in mysql_create_table (thd=thd@entry=0x14e078000c58, create_table=create_table@entry=0x14e078010950, create_info=create_info@entry=0x14e10c447c70, alter_info=alter_info@entry=0x14e10c447b80) at /test/10.6_opt/sql/sql_table.cc:4658
#23 0x000055b95df41c59 in Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14e078000c58) at /test/10.6_opt/sql/sql_table.cc:11778
#24 0x000055b95de9d63e in mysql_execute_command (thd=0x14e078000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:5993
#25 0x000055b95de8da46 in mysql_parse (thd=0x14e078000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.6_opt/sql/sql_parse.cc:8026
#26 0x000055b95de99945 in dispatch_command (command=COM_QUERY, thd=0x14e078000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.6_opt/sql/sql_class.h:1340
#27 0x000055b95de9b867 in do_command (thd=0x14e078000c58, blocking=blocking@entry=true) at /test/10.6_opt/sql/sql_parse.cc:1404
#28 0x000055b95dfaa017 in do_handle_one_connection (connect=<optimized out>, put_in_cache=true) at /test/10.6_opt/sql/sql_connect.cc:1410
#29 0x000055b95dfaa37d in handle_one_connection (arg=arg@entry=0x55b95fbcad18) at /test/10.6_opt/sql/sql_connect.cc:1312
#30 0x000055b95e334948 in pfs_spawn_thread (arg=0x55b960067bf8) at /test/10.6_opt/storage/perfschema/pfs.cc:2201
#31 0x000014e10dd4f609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014e10d93d293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
{noformat}

{noformat:title=10.6.4 42b9daaea7ece6155558f7c3bf638e8e175a7ff7 (Optimized)}
...
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2604.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2605.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [Note] InnoDB: Cannot close file ./test/t2#P#p2606.ibd because of 0 pending operations and pending fsync
2021-07-29 0:00:17 8 [ERROR] InnoDB: preallocating 98304 bytes for file ./test/t2#P#p4560.ibd failed with error 9
2021-07-29 00:00:17 0x14e10c449700 InnoDB: Assertion failure in file /test/10.6_opt/storage/innobase/fil/fil0fil.cc line 591
InnoDB: Failing assertion: fsize != os_offset_t(-1)
InnoDB: We intentionally generate a memory trap.
{noformat}

Marko Mäkelä made changes - 2021-10-15 10:11

Fix Version/s		10.2 [ 14601 ]
Fix Version/s		10.3 [ 22126 ]
Fix Version/s		10.4 [ 22408 ]
Fix Version/s		10.5 [ 23123 ]
Fix Version/s		10.7 [ 24805 ]
Affects Version/s		10.2 [ 14601 ]
Affects Version/s		10.3 [ 22126 ]
Affects Version/s		10.4 [ 22408 ]
Affects Version/s		10.5 [ 23123 ]
Affects Version/s		10.7 [ 24805 ]
Assignee	Marko Mäkelä [ marko ]	Eugene Kosov [ kevg ]

Roel Van de Paar made changes - 2021-10-21 04:55

Link

This issue split to ~~MDEV-26873~~ [ ~~MDEV-26873~~ ]

Sergei Golubchik made changes - 2021-12-06 21:35

Workflow

MariaDB v3 [ 124331 ]

MariaDB v4 [ 143091 ]

Marko Mäkelä made changes - 2022-02-03 15:34

Assignee	Eugene Kosov [ kevg ]	Roel Van de Paar [ roel ]
Status	Open [ 1 ]	Needs Feedback [ 10501 ]

Sergei Golubchik made changes - 2022-03-07 11:43

Assignee	Roel Van de Paar [ roel ]	Marko Mäkelä [ marko ]
Status	Needs Feedback [ 10501 ]	Open [ 1 ]

Marko Mäkelä made changes - 2022-03-07 12:05

Assignee	Marko Mäkelä [ marko ]	Roel Van de Paar [ roel ]
Status	Open [ 1 ]	Needs Feedback [ 10501 ]

Roel Van de Paar made changes - 2022-03-08 05:10

Status

Needs Feedback [ 10501 ]

Open [ 1 ]

Roel Van de Paar made changes - 2022-03-08 05:10

Assignee

Roel Van de Paar [ roel ]

Marko Mäkelä [ marko ]

Marko Mäkelä made changes - 2022-05-20 06:38

Assignee	Marko Mäkelä [ marko ]	Roel Van de Paar [ roel ]
Status	Open [ 1 ]	Needs Feedback [ 10501 ]

Sergei Golubchik made changes - 2022-06-20 09:10

Fix Version/s		N/A [ 14700 ]
Fix Version/s	10.2 [ 14601 ]
Fix Version/s	10.3 [ 22126 ]
Fix Version/s	10.4 [ 22408 ]
Fix Version/s	10.5 [ 23123 ]
Fix Version/s	10.6 [ 24028 ]
Fix Version/s	10.7 [ 24805 ]
Resolution		Incomplete [ 4 ]
Status	Needs Feedback [ 10501 ]	Closed [ 6 ]

Alice Sherepa made changes - 2023-06-16 12:05

Link

This issue relates to ~~MDEV-31486~~ [ ~~MDEV-31486~~ ]

Daniel Black made changes - 2023-06-21 03:50

Link

This issue relates to ~~MDEV-31486~~ [ ~~MDEV-31486~~ ]

Valerii Kravchuk made changes - 2023-09-07 10:02

Resolution	Incomplete [ 4 ]
Status	Closed [ 6 ]	Stalled [ 10000 ]

Roel Van de Paar made changes - 2023-09-08 03:51

Affects Version/s		11.0 [ 28320 ]
Affects Version/s		11.2 [ 28603 ]

Roel Van de Paar made changes - 2023-09-08 03:52

Comment

[ Interestingly, the first frame is {{fil_space_extend_must_retry}} which is also the first frame in ~~MDEV-26068~~ (Assertion {{size >= 4}} failed).
Can the test from ~~MDEV-26068~~ clarify anything here? ]

Roel Van de Paar made changes - 2023-09-08 03:52

Link

This issue relates to ~~MDEV-26068~~ [ ~~MDEV-26068~~ ]

Roel Van de Paar made changes - 2023-09-08 07:05

Comment

[ Continued output (JIRA limitation)
{noformat:title=11.3.0 7ba9c7fb84b5f28e4736656b57d9508b70ca6369 (Debug)}
Thread 54 (LWP 1948928):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 53 (Thread 0x14a3677fb640 (LWP 2142666)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3677faa90, op=137, expected=0, futex_word=0x55b9bd8ac638) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3677faa90, clockid=0, expected=0, futex_word=0x55b9bd8ac638) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac638, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3677faa90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3677faa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac610) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3677faa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac610) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac610, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3677faa90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac610) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac610) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac610) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac610) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac610) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac610, t=t@entry=0x14a3677fab38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac610) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 52 (Thread 0x14a3643e1640 (LWP 2238759)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3643e0a90, op=137, expected=0, futex_word=0x55b9bd8ac53c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3643e0a90, clockid=0, expected=0, futex_word=0x55b9bd8ac53c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac53c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3643e0a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3643e0a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac510) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3643e0a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac510) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac510, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3643e0a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac510) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac510) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac510) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac510) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac510) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac510, t=t@entry=0x14a3643e0b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac510) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 51 (Thread 0x14a3639dc640 (LWP 2142650)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3639dba90, op=137, expected=0, futex_word=0x55b9bd8acc3c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3639dba90, clockid=0, expected=0, futex_word=0x55b9bd8acc3c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8acc3c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3639dba90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3639dba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acc10) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3639dba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acc10) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8acc10, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3639dba90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8acc10) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acc10) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acc10) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8acc10) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8acc10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8acc10, t=t@entry=0x14a3639dbb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8acc10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 50 (Thread 0x14a3657eb640 (LWP 2238758)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3657eaa90, op=137, expected=0, futex_word=0x55b9bd8ac13c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3657eaa90, clockid=0, expected=0, futex_word=0x55b9bd8ac13c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac13c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3657eaa90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3657eaa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac110) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3657eaa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac110) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac110, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3657eaa90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac110) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac110) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac110) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac110) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac110) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac110, t=t@entry=0x14a3657eab38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac110) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 49 (Thread 0x14a365fef640 (LWP 1897397)):
#0 __futex_abstimed_wait_common64 (private=-1154830387, cancel=true, abstime=0x14a365feea90, op=137, expected=0, futex_word=0x55b9bd8ac8b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1154830387, abstime=0x14a365feea90, clockid=0, expected=0, futex_word=0x55b9bd8ac8b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac8b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a365feea90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a365feea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac890) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a365feea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac890) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac890, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a365feea90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac890) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac890) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac890) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac890) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac890) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac890, t=t@entry=0x14a365feeb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac890) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 48 (LWP 2046881):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 47 (Thread 0x14a3b42eb640 (LWP 1897392)):
#0 __futex_abstimed_wait_common64 (private=-1102119387, cancel=true, abstime=0x14a3b42eaa90, op=137, expected=0, futex_word=0x55b9bd8ab938) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1102119387, abstime=0x14a3b42eaa90, clockid=0, expected=0, futex_word=0x55b9bd8ab938) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab938, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b42eaa90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b42eaa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab910) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b42eaa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab910) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab910, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b42eaa90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab910) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab910) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab910) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab910) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab910) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab910, t=t@entry=0x14a3b42eab38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab910) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 46 (Thread 0x14a3b4ef1640 (LWP 2238757)):
#0 __futex_abstimed_wait_common64 (private=-1102119387, cancel=true, abstime=0x14a3b4ef0a90, op=137, expected=0, futex_word=0x55b9bd8acf3c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1102119387, abstime=0x14a3b4ef0a90, clockid=0, expected=0, futex_word=0x55b9bd8acf3c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8acf3c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b4ef0a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b4ef0a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acf10) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b4ef0a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acf10) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8acf10, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b4ef0a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8acf10) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acf10) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acf10) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8acf10) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8acf10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8acf10, t=t@entry=0x14a3b4ef0b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8acf10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 45 (LWP 2269997):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 44 (LWP 2301906):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 43 (Thread 0x14a3663f1640 (LWP 1897366)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3663f0a90, op=137, expected=0, futex_word=0x55b9bd8ac73c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3663f0a90, clockid=0, expected=0, futex_word=0x55b9bd8ac73c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac73c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3663f0a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3663f0a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac710) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3663f0a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac710) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac710, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3663f0a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac710) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac710) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac710) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac710) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac710) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac710, t=t@entry=0x14a3663f0b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac710) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 42 (Thread 0x14a3b54f4640 (LWP 2238752)):
#0 __futex_abstimed_wait_common64 (private=-1102119387, cancel=true, abstime=0x14a3b54f3a90, op=137, expected=0, futex_word=0x55b9bd8ab038) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1102119387, abstime=0x14a3b54f3a90, clockid=0, expected=0, futex_word=0x55b9bd8ab038) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab038, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b54f3a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b54f3a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab010) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b54f3a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab010) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab010, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b54f3a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab010) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab010) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab010) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab010) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab010) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab010, t=t@entry=0x14a3b54f3b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab010) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 41 (Thread 0x14a3b7bff640 (LWP 2031968)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b7bfea90, op=137, expected=0, futex_word=0x55b9bd8ac23c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b7bfea90, clockid=0, expected=0, futex_word=0x55b9bd8ac23c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac23c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b7bfea90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b7bfea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac210) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b7bfea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac210) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac210, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b7bfea90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac210) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac210) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac210) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac210) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac210) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac210, t=t@entry=0x14a3b7bfeb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac210) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 40 (LWP 2299033):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 39 (Thread 0x14a366bf5640 (LWP 2238755)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a366bf4a90, op=137, expected=0, futex_word=0x55b9bd8ab638) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a366bf4a90, clockid=0, expected=0, futex_word=0x55b9bd8ab638) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab638, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a366bf4a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a366bf4a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab610) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a366bf4a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab610) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab610, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a366bf4a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab610) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab610) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab610) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab610) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab610) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab610, t=t@entry=0x14a366bf4b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab610) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 38 (Thread 0x14a3b5d8e640 (LWP 1389566)):
#0 __futex_abstimed_wait_common64 (private=-1109302528, cancel=true, abstime=0x14a3b5d8db00, op=393, expected=0, futex_word=0x55b9bc780c08 <COND_checkpoint+40>) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1109302528, abstime=0x14a3b5d8db00, clockid=2072, expected=0, futex_word=0x55b9bc780c08 <COND_checkpoint+40>) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bc780c08 <COND_checkpoint+40>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x14a3b5d8db00, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc93f1b in __pthread_cond_wait_common (abstime=0x14a3b5d8db00, clockid=0, mutex=0x55b9bc780c48 <LOCK_checkpoint+40>, cond=0x55b9bc780be0 <COND_checkpoint>) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_timedwait64 (cond=cond@entry=0x55b9bc780be0 <COND_checkpoint>, mutex=mutex@entry=0x55b9bc780c48 <LOCK_checkpoint+40>, abstime=abstime@entry=0x14a3b5d8db00) at ./nptl/pthread_cond_wait.c:652
#5 0x000055b9bb38ce77 in safe_cond_timedwait (cond=0x55b9bc780be0 <COND_checkpoint>, mp=0x55b9bc780c20 <LOCK_checkpoint>, abstime=abstime@entry=0x14a3b5d8db00, file=file@entry=0x55b9bb5231d8 "/test/git-bisect/11.3_dbg/include/mysql/psi/mysql_thread.h", line=line@entry=1088) at /test/git-bisect/11.3_dbg/mysys/thr_mutex.c:548
#6 0x000055b9baf1ee7f in inline_mysql_cond_timedwait (src_line=115, src_file=0x55b9bb73c8a8 "/test/git-bisect/11.3_dbg/storage/maria/ma_servicethread.c", abstime=0x14a3b5d8db00, mutex=<optimized out>, that=<optimized out>) at /test/git-bisect/11.3_dbg/include/mysql/psi/mysql_thread.h:1088
#7 my_service_thread_sleep (control=control@entry=0x55b9bbe83580 <checkpoint_control>, sleep_time=sleep_time@entry=30000000000) at /test/git-bisect/11.3_dbg/storage/maria/ma_servicethread.c:115
#8 0x000055b9baf17401 in ma_checkpoint_background (arg=0x1e) at /test/git-bisect/11.3_dbg/storage/maria/ma_checkpoint.c:725
#9 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#10 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 37 (LWP 2333039):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 36 (LWP 2209359):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 35 (Thread 0x14a3667f3640 (LWP 2031966)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3667f2a90, op=137, expected=0, futex_word=0x55b9bd8acb38) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3667f2a90, clockid=0, expected=0, futex_word=0x55b9bd8acb38) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8acb38, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3667f2a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3667f2a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acb10) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3667f2a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acb10) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8acb10, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3667f2a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8acb10) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acb10) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acb10) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8acb10) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8acb10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8acb10, t=t@entry=0x14a3667f2b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8acb10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 34 (LWP 2295471):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 33 (LWP 2315260):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 32 (LWP 2225463):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 31 (Thread 0x14a3645e2640 (LWP 2142659)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3645e1a90, op=137, expected=0, futex_word=0x55b9bd8ac33c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3645e1a90, clockid=0, expected=0, futex_word=0x55b9bd8ac33c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac33c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3645e1a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3645e1a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac310) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3645e1a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac310) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac310, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3645e1a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac310) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac310) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac310) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac310) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac310) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac310, t=t@entry=0x14a3645e1b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac310) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 30 (LWP 2047913):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 29 (LWP 2259009):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 28 (Thread 0x14a3b6e17640 (LWP 2031961)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b6e16a90, op=137, expected=0, futex_word=0x55b9bd8acebc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b6e16a90, clockid=0, expected=0, futex_word=0x55b9bd8acebc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8acebc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b6e16a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b6e16a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ace90) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b6e16a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ace90) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ace90, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b6e16a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ace90) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ace90) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ace90) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ace90) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ace90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ace90, t=t@entry=0x14a3b6e16b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ace90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 27 (LWP 2293639):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 26 (Thread 0x14a3bd032640 (LWP 1389332)):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x000014a3be4efd43 in ____sys_io_uring_enter2 (sz=<optimized out>, sig=<optimized out>, flags=<optimized out>, min_complete=<optimized out>, to_submit=<optimized out>, fd=<optimized out>) at syscall.h:127
#2 _io_uring_get_cqe (ring=0x55b9bd8f0028, cqe_ptr=0x14a3bd031b28, data=data@entry=0x14a3bd031af0) at queue.c:111
#3 0x000014a3be4efebb in __io_uring_get_cqe (ring=<optimized out>, cqe_ptr=<optimized out>, submit=<optimized out>, wait_nr=<optimized out>, sigmask=<optimized out>) at queue.c:139
#4 0x000055b9bb32b1f3 in io_uring_wait_cqe_nr (wait_nr=1, cqe_ptr=0x14a3bd031b28, ring=0x55b9bd8f0028) at /usr/include/liburing.h:772
#5 io_uring_wait_cqe (cqe_ptr=0x14a3bd031b28, ring=0x55b9bd8f0028) at /usr/include/liburing.h:792
#6 (anonymous namespace)::aio_uring::thread_routine (aio=0x55b9bd8f0020) at /test/git-bisect/11.3_dbg/tpool/aio_liburing.cc:145
#7 0x000055b9bb32aed6 in std::__invoke_impl<void, void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> (__f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:61
#8 std::__invoke<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#9 std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::_M_invoke<0, 1> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#10 std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#11 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#12 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#14 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 25 (Thread 0x14a365dee640 (LWP 2142656)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a365deda90, op=137, expected=0, futex_word=0x55b9bd8abcbc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a365deda90, clockid=0, expected=0, futex_word=0x55b9bd8abcbc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8abcbc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a365deda90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a365deda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8abc90) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a365deda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8abc90) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8abc90, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a365deda90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8abc90) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8abc90) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8abc90) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8abc90) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8abc90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8abc90, t=t@entry=0x14a365dedb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8abc90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 24 (LWP 2302721):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 23 (LWP 1955249):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 22 (Thread 0x14a3b6412640 (LWP 2142653)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b6411a90, op=137, expected=0, futex_word=0x55b9bd8aca38) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b6411a90, clockid=0, expected=0, futex_word=0x55b9bd8aca38) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8aca38, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b6411a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b6411a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8aca10) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b6411a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8aca10) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8aca10, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b6411a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8aca10) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8aca10) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8aca10) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8aca10) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8aca10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8aca10, t=t@entry=0x14a3b6411b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8aca10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 21 (Thread 0x14a3647e3640 (LWP 2142663)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3647e2a90, op=137, expected=0, futex_word=0x55b9bd8accbc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3647e2a90, clockid=0, expected=0, futex_word=0x55b9bd8accbc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8accbc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3647e2a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3647e2a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acc90) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3647e2a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acc90) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8acc90, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3647e2a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8acc90) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acc90) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acc90) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8acc90) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8acc90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8acc90, t=t@entry=0x14a3647e2b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8acc90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 20 (LWP 2168872):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 19 (LWP 2246080):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 18 (LWP 2260347):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 17 (LWP 2148171):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 16 (LWP 2183755):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 15 (Thread 0x14a363dde640 (LWP 2238761)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a363ddda90, op=137, expected=0, futex_word=0x55b9bd8abf38) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a363ddda90, clockid=0, expected=0, futex_word=0x55b9bd8abf38) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8abf38, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a363ddda90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a363ddda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8abf10) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a363ddda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8abf10) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8abf10, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a363ddda90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8abf10) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8abf10) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8abf10) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8abf10) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8abf10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8abf10, t=t@entry=0x14a363dddb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8abf10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 14 (LWP 2166858):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 13 (Thread 0x14a3b5b8d640 (LWP 1389646)):
#0 0x000014a3bdc4321a in __GI___sigtimedwait (set=0x14a3b5b8ca50, info=0x14a3b5b8cad0, timeout=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:61
#1 0x000055b9ba8e4c05 in my_sigwait (code=<synthetic pointer>, sig=0x14a3b5b8ca4c, set=0x14a3b5b8ca50) at /test/git-bisect/11.3_dbg/include/my_pthread.h:195
#2 signal_hand (arg=<optimized out>) at /test/git-bisect/11.3_dbg/sql/mysqld.cc:3277
#3 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#4 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 12 (Thread 0x14a363bdd640 (LWP 1897361)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a363bdca90, op=137, expected=0, futex_word=0x55b9bd8ac0b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a363bdca90, clockid=0, expected=0, futex_word=0x55b9bd8ac0b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac0b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a363bdca90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a363bdca90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac090) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a363bdca90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac090) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac090, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a363bdca90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac090) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac090) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac090) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac090) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac090) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac090, t=t@entry=0x14a363bdcb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac090) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 11 (Thread 0x14a3669f4640 (LWP 1653089)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3669f3a90, op=137, expected=0, futex_word=0x55b9bd8ab73c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3669f3a90, clockid=0, expected=0, futex_word=0x55b9bd8ab73c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab73c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3669f3a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3669f3a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab710) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3669f3a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab710) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab710, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3669f3a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab710) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab710) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab710) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab710) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab710) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab710, t=t@entry=0x14a3669f3b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab710) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 10 (LWP 2227923):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 9 (LWP 2157174):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 8 (LWP 2133300):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 7 (Thread 0x14a3bdec4640 (LWP 1389301)):
#0 __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x14a3bdec3b50, op=393, expected=0, futex_word=0x55b9bc7c6e88 <COND_timer+40>) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x14a3bdec3b50, clockid=0, expected=0, futex_word=0x55b9bc7c6e88 <COND_timer+40>) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bc7c6e88 <COND_timer+40>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x14a3bdec3b50, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc93f1b in __pthread_cond_wait_common (abstime=0x14a3bdec3b50, clockid=0, mutex=0x55b9bc7c6ec8 <LOCK_timer+40>, cond=0x55b9bc7c6e60 <COND_timer>) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_timedwait64 (cond=cond@entry=0x55b9bc7c6e60 <COND_timer>, mutex=mutex@entry=0x55b9bc7c6ec8 <LOCK_timer+40>, abstime=abstime@entry=0x14a3bdec3b50) at ./nptl/pthread_cond_wait.c:652
#5 0x000055b9bb38ce77 in safe_cond_timedwait (cond=cond@entry=0x55b9bc7c6e60 <COND_timer>, mp=mp@entry=0x55b9bc7c6ea0 <LOCK_timer>, abstime=abstime@entry=0x14a3bdec3b50, file=file@entry=0x55b9bb5231d8 "/test/git-bisect/11.3_dbg/include/mysql/psi/mysql_thread.h", line=line@entry=1088) at /test/git-bisect/11.3_dbg/mysys/thr_mutex.c:548
#6 0x000055b9bb38d429 in inline_mysql_cond_timedwait (src_line=321, src_file=0x55b9bb7e3940 "/test/git-bisect/11.3_dbg/mysys/thr_timer.c", abstime=0x14a3bdec3b50, mutex=0x55b9bc7c6ea0 <LOCK_timer>, that=0x55b9bc7c6e60 <COND_timer>) at /test/git-bisect/11.3_dbg/include/mysql/psi/mysql_thread.h:1088
#7 timer_handler (arg=<optimized out>) at /test/git-bisect/11.3_dbg/mysys/thr_timer.c:321
#8 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#9 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 6 (LWP 2058143):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 5 (LWP 1946611):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 4 (Thread 0x14a3b52f3640 (LWP 2142652)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b52f2a90, op=137, expected=0, futex_word=0x55b9bd8ac3bc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b52f2a90, clockid=0, expected=0, futex_word=0x55b9bd8ac3bc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac3bc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b52f2a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b52f2a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac390) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b52f2a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac390) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac390, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b52f2a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac390) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac390) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac390) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac390) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac390) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac390, t=t@entry=0x14a3b52f2b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac390) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 3 (Thread 0x14a3b4aef640 (LWP 1672295)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b4aeea90, op=137, expected=0, futex_word=0x55b9bd8ab8bc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b4aeea90, clockid=0, expected=0, futex_word=0x55b9bd8ab8bc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab8bc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b4aeea90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b4aeea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab890) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b4aeea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab890) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab890, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b4aeea90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab890) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab890) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab890) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab890) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab890) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab890, t=t@entry=0x14a3b4aeeb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab890) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 2 (Thread 0x14a3bde72640 (LWP 1389633)):
#0 __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x55b9bbf4cc2c <COND_manager+44>) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55b9bbf4cc2c <COND_manager+44>) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bbf4cc2c <COND_manager+44>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc93ac1 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55b9bbf4cc68 <LOCK_manager+40>, cond=0x55b9bbf4cc00 <COND_manager>) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_wait (cond=cond@entry=0x55b9bbf4cc00 <COND_manager>, mutex=mutex@entry=0x55b9bbf4cc68 <LOCK_manager+40>) at ./nptl/pthread_cond_wait.c:627
#5 0x000055b9bb38cbdb in safe_cond_wait (cond=0x55b9bbf4cc00 <COND_manager>, mp=0x55b9bbf4cc40 <LOCK_manager>, file=0x55b9bb5231d8 "/test/git-bisect/11.3_dbg/include/mysql/psi/mysql_thread.h", line=1072) at /test/git-bisect/11.3_dbg/mysys/thr_mutex.c:494
#6 0x000055b9ba9f5a5a in inline_mysql_cond_wait (src_line=103, src_file=0x55b9bb5497b0 "/test/git-bisect/11.3_dbg/sql/sql_manager.cc", mutex=0x55b9bbf4cc40 <LOCK_manager>, that=0x55b9bbf4cc00 <COND_manager>) at /test/git-bisect/11.3_dbg/include/mysql/psi/mysql_thread.h:1072
#7 handle_manager (arg=<optimized out>) at /test/git-bisect/11.3_dbg/sql/sql_manager.cc:103
#8 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#9 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 1 (Thread 0x14a3be22f3c0 (LWP 1389245)):
#0 0x000014a3bdd18d7f in __GI___poll (fds=0x55b9bdca1d28, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
#1 0x000055b9ba8ec546 in poll (__timeout=-1, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/poll2.h:39
#2 handle_connections_sockets () at /test/git-bisect/11.3_dbg/sql/mysqld.cc:6362
#3 0x000055b9ba8effb1 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /test/git-bisect/11.3_dbg/sql/mysqld.cc:6064
#4 0x000055b9ba8e3c76 in main (argc=<optimized out>, argv=<optimized out>) at /test/git-bisect/11.3_dbg/sql/main.cc:34
{noformat} ]

Roel Van de Paar made changes - 2023-09-08 07:05

Comment

[ The same testcase timed out after 900 seconds on an 11.3 build of 2 Sep 23. It is not clear yet if this was due to system load or an actual hang/bug.
{noformat:title=11.3.0 7ba9c7fb84b5f28e4736656b57d9508b70ca6369 (Debug)}
...
main.test 'innodb' w15 [ fail ] timeout after 900 seconds
Test ended at 2023-09-08 14:43:22

Test case timeout after 900 seconds
...
warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing
...
[New LWP 1389245]
...
[New LWP 1389336]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/test/MD020923-mariadb-11.3.0-linux-x86_64-dbg/bin/mariadbd --defaults-group-su'.
Program terminated with signal SIGABRT, Aborted.
#0 0x000014a3bdd18d7f in __GI___poll (fds=0x55b9bdca1d28, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
29 ../sysdeps/unix/sysv/linux/poll.c: No such file or directory.
[Current thread is 1 (Thread 0x14a3be22f3c0 (LWP 1389245))]

Thread 94 (Thread 0x14a3bc631640 (LWP 1389336)):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x000014a3be4efe25 in ____sys_io_uring_enter2 (sz=8, sig=0x0, flags=<optimized out>, min_complete=<optimized out>, to_submit=<optimized out>, fd=<optimized out>) at syscall.h:127
#2 ____sys_io_uring_enter (sig=0x0, flags=<optimized out>, min_complete=<optimized out>, to_submit=<optimized out>, fd=<optimized out>) at syscall.h:137
#3 __io_uring_submit (ring=<optimized out>, submitted=<optimized out>, wait_nr=<optimized out>) at queue.c:369
#4 0x000055b9bb32af8f in (anonymous namespace)::aio_uring::submit_io (this=0x55b9bd8f0020, cb=0x55b9bd909f28) at /test/git-bisect/11.3_dbg/tpool/aio_liburing.cc:116
#5 0x000055b9bb0975bb in tpool::thread_pool::submit_io (cb=0x55b9bd909f28, this=<optimized out>) at /test/git-bisect/11.3_dbg/tpool/tpool.h:256
#6 os_aio (type=..., buf=buf@entry=0x14a3bcb54000, offset=offset@entry=16384, n=n@entry=16384) at /test/git-bisect/11.3_dbg/storage/innobase/os/os0file.cc:3840
#7 0x000055b9bb2aa814 in fil_space_t::io (this=this@entry=0x14a37208b640, type=..., offset=16384, len=16384, buf=buf@entry=0x14a3bcb54000, bpage=bpage@entry=0x14a3bcb325b0) at /test/git-bisect/11.3_dbg/storage/innobase/fil/fil0fil.cc:2897
#8 0x000055b9bb246a22 in buf_page_t::flush (this=this@entry=0x14a3bcb325b0, evict=evict@entry=false, space=space@entry=0x14a37208b640) at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0flu.cc:856
#9 0x000055b9bb248c83 in buf_flush_LRU_list_batch (n=<synthetic pointer>, evict=false, max=2000) at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0flu.cc:1308
#10 buf_do_LRU_batch (n=<synthetic pointer>, evict=false, max=2000) at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0flu.cc:1342
#11 buf_flush_LRU (max_n=max_n@entry=2000, evict=evict@entry=false) at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0flu.cc:1688
#12 0x000055b9bb24b509 in buf_flush_page_cleaner () at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0flu.cc:2432
#13 0x000055b9bb24c4ff in std::__invoke_impl<void, void (*)()> (__f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:60
#14 std::__invoke<void (*)()> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (*)()> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 93 (Thread 0x14a3b5b42640 (LWP 1389739)):
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1 0x000055b9bb1991ae in srw_mutex_impl<true>::wait (lk=2147483650, this=0x14a3bc9350d8) at /test/git-bisect/11.3_dbg/storage/innobase/sync/srw_lock.cc:238
#2 srw_mutex_impl<true>::wait_and_lock (this=this@entry=0x14a3bc9350d8) at /test/git-bisect/11.3_dbg/storage/innobase/sync/srw_lock.cc:326
#3 0x000055b9bb2349f1 in srw_mutex_impl<true>::wr_lock (this=0x14a3bc9350d8) at /test/git-bisect/11.3_dbg/storage/innobase/include/srw_lock.h:134
#4 ssux_lock_impl<true>::u_lock (this=<optimized out>) at /test/git-bisect/11.3_dbg/storage/innobase/include/srw_lock.h:250
#5 sux_lock<ssux_lock_impl<true> >::u_lock (this=0x14a3bc9350d8) at /test/git-bisect/11.3_dbg/storage/innobase/include/sux_lock.h:378
#6 buf_page_get_low (page_id=..., page_id@entry=..., zip_size=zip_size@entry=0, rw_latch=rw_latch@entry=4, guess=<optimized out>, guess@entry=0x0, mode=mode@entry=16, mtr=mtr@entry=0x14a3b5b3a1c0, err=<optimized out>) at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0buf.cc:2593
#7 0x000055b9bb235269 in buf_page_get_gen (page_id=page_id@entry=..., zip_size=0, rw_latch=rw_latch@entry=4, guess=guess@entry=0x0, mode=mode@entry=16, mtr=mtr@entry=0x14a3b5b3a1c0, err=0x14a3b5b39ad8) at /test/git-bisect/11.3_dbg/storage/innobase/buf/buf0buf.cc:2656
#8 0x000055b9bb2b955a in fsp_get_header (space=space@entry=0x55b9bd96b120, mtr=mtr@entry=0x14a3b5b3a1c0, err=err@entry=0x14a3b5b39ad8) at /test/git-bisect/11.3_dbg/storage/innobase/fsp/fsp0fsp.cc:130
#9 0x000055b9bb2bd01a in fsp_reserve_free_extents (n_reserved=n_reserved@entry=0x14a3b5b39b58, space=0x55b9bd96b120, n_ext=3, alloc_type=alloc_type@entry=FSP_NORMAL, mtr=mtr@entry=0x14a3b5b3a1c0, n_pages=n_pages@entry=2) at /test/git-bisect/11.3_dbg/storage/innobase/fsp/fsp0fsp.cc:2401
#10 0x000055b9bb1ffedb in btr_cur_pessimistic_insert (flags=flags@entry=0, cursor=cursor@entry=0x14a3b5b39c30, offsets=offsets@entry=0x14a3b5b39c18, heap=heap@entry=0x14a3b5b39c10, entry=entry@entry=0x14a370199e80, rec=rec@entry=0x14a3b5b39c28, big_rec=0x14a3b5b39c08, n_ext=<optimized out>, thr=0x14a3720ba390, mtr=0x14a3b5b3a1c0) at /test/git-bisect/11.3_dbg/storage/innobase/btr/btr0cur.cc:2490
#11 0x000055b9bb10c9dc in row_ins_clust_index_entry_low (flags=flags@entry=0, mode=<optimized out>, mode@entry=BTR_MODIFY_TREE, index=index@entry=0x14a371fc1fe0, n_uniq=n_uniq@entry=4, entry=entry@entry=0x14a370199e80, n_ext=n_ext@entry=0, thr=<optimized out>) at /test/git-bisect/11.3_dbg/storage/innobase/row/row0ins.cc:2887
#12 0x000055b9bb10cfa1 in row_ins_clust_index_entry (index=index@entry=0x14a371fc1fe0, entry=entry@entry=0x14a370199e80, thr=thr@entry=0x14a3720ba390, n_ext=n_ext@entry=0) at /test/git-bisect/11.3_dbg/storage/innobase/row/row0ins.cc:3260
#13 0x000055b9bb10e8fb in row_ins_index_entry (thr=0x14a3720ba390, entry=0x14a370199e80, index=0x14a371fc1fe0) at /test/git-bisect/11.3_dbg/storage/innobase/row/row0ins.cc:3375
#14 row_ins_index_entry_step (thr=0x14a3720ba390, node=0x14a3720ba0d8) at /test/git-bisect/11.3_dbg/storage/innobase/row/row0ins.cc:3543
#15 row_ins (thr=0x14a3720ba390, node=0x14a3720ba0d8) at /test/git-bisect/11.3_dbg/storage/innobase/row/row0ins.cc:3660
#16 row_ins_step (thr=thr@entry=0x14a3720ba390) at /test/git-bisect/11.3_dbg/storage/innobase/row/row0ins.cc:3789
#17 0x000055b9bb0d1016 in que_thr_step (thr=0x14a3720ba390) at /test/git-bisect/11.3_dbg/storage/innobase/que/que0que.cc:567
#18 que_run_threads_low (thr=0x14a3720ba390) at /test/git-bisect/11.3_dbg/storage/innobase/que/que0que.cc:644
#19 que_run_threads (thr=0x14a3720ba390) at /test/git-bisect/11.3_dbg/storage/innobase/que/que0que.cc:664
#20 0x000055b9bb0d15cd in que_eval_sql (info=info@entry=0x14a3735d7920, sql=sql@entry=0x55b9bb7c3080 "PROCEDURE INDEX_STATS_SAVE () IS\nBEGIN\nDELETE FROM \"mysql/innodb_index_stats\"\nWHERE\ndatabase_name = :database_name AND\ntable_name = :table_name AND\nindex_name = :index_name AND\nstat_name = :stat_name;"..., trx=trx@entry=0x14a3bd035180) at /test/git-bisect/11.3_dbg/storage/innobase/que/que0que.cc:703
#21 0x000055b9bb294d17 in dict_stats_exec_sql (pinfo=pinfo@entry=0x14a3735d7920, sql=sql@entry=0x55b9bb7c3080 "PROCEDURE INDEX_STATS_SAVE () IS\nBEGIN\nDELETE FROM \"mysql/innodb_index_stats\"\nWHERE\ndatabase_name = :database_name AND\ntable_name = :table_name AND\nindex_name = :index_name AND\nstat_name = :stat_name;"..., trx=trx@entry=0x14a3bd035180) at /test/git-bisect/11.3_dbg/storage/innobase/dict/dict0stats.cc:548
#22 0x000055b9bb295440 in dict_stats_save_index_stat (index=index@entry=0x14a371ea5ad8, last_update=last_update@entry=1694148150, stat_name=<optimized out>, stat_name@entry=0x55b9bb7b8273 "size", stat_value=1, sample_size=sample_size@entry=0x0, stat_description=stat_description@entry=0x55b9bb7c27b1 "Number of pages in the index", trx=0x14a3bd035180) at /test/git-bisect/11.3_dbg/storage/innobase/dict/dict0stats.cc:3098
#23 0x000055b9bb299a5a in dict_stats_save (table_orig=table_orig@entry=0x14a371f0d890, only_for_index=only_for_index@entry=0x0) at /test/git-bisect/11.3_dbg/storage/innobase/dict/dict0stats.cc:3381
#24 0x000055b9bb29c5a1 in dict_stats_update (table=0x14a371f0d890, stats_upd_option=stats_upd_option@entry=DICT_STATS_EMPTY_TABLE) at /test/git-bisect/11.3_dbg/storage/innobase/dict/dict0stats.cc:4044
#25 0x000055b9bafe823f in ha_innobase::create (this=<optimized out>, name=<optimized out>, form=0x14a3b5b3c420, create_info=<optimized out>, file_per_table=<optimized out>, trx=0x14a3bd034680, trx@entry=0x0) at /test/git-bisect/11.3_dbg/storage/innobase/handler/ha_innodb.h:714
#26 0x000055b9bafe9ad1 in ha_innobase::create (this=<optimized out>, name=<optimized out>, form=<optimized out>, create_info=<optimized out>) at /test/git-bisect/11.3_dbg/storage/innobase/handler/ha_innodb.cc:13271
#27 0x000055b9bacfa3c5 in handler::ha_create (this=0x14a371bd14f0, name=name@entry=0x14a3b5b3c0f0 "./test/t2#P#p7035", form=form@entry=0x14a3b5b3c420, info_arg=info_arg@entry=0x14a3b5b3de70) at /test/git-bisect/11.3_dbg/sql/handler.cc:5667
#28 0x000055b9baf7b87a in ha_partition::create (this=0x14a3735ea780, name=0x14a3b5b3d8d0 "./test/t2", table_arg=0x14a3b5b3c420, create_info=0x14a3b5b3de70) at /test/git-bisect/11.3_dbg/sql/ha_partition.cc:855
#29 0x000055b9bacfa3c5 in handler::ha_create (this=0x14a3735ea780, name=0x14a3b5b3d8d0 "./test/t2", form=form@entry=0x14a3b5b3c420, info_arg=info_arg@entry=0x14a3b5b3de70) at /test/git-bisect/11.3_dbg/sql/handler.cc:5667
#30 0x000055b9bacfafe4 in ha_create_table (thd=thd@entry=0x14a370000d58, path=<optimized out>, db=<optimized out>, table_name=<optimized out>, create_info=create_info@entry=0x14a3b5b3de70, frm=frm@entry=0x14a3b5b3d8c0, skip_frm_file=<optimized out>) at /test/git-bisect/11.3_dbg/sql/handler.cc:6136
#31 0x000055b9baae2c66 in create_table_impl (thd=thd@entry=0x14a370000d58, ddl_log_state_create=ddl_log_state_create@entry=0x14a3b5b3dbf0, ddl_log_state_rm=<optimized out>, ddl_log_state_rm@entry=0x14a3b5b3dd80, orig_db=..., orig_table_name=..., db=..., table_name=..., path=..., options=..., create_info=0x14a3b5b3de70, alter_info=0x14a3b5b3dc90, create_table_mode=0, is_trans=0x14a3b5b3dbd0, key_info=0x14a3b5b3d8a8, key_count=0x14a3b5b3d8a4, frm=0x14a3b5b3d8c0) at /test/git-bisect/11.3_dbg/sql/sql_table.cc:4667
#32 0x000055b9baae3013 in mysql_create_table_no_lock (thd=thd@entry=0x14a370000d58, ddl_log_state_create=ddl_log_state_create@entry=0x14a3b5b3dbf0, ddl_log_state_rm=ddl_log_state_rm@entry=0x14a3b5b3dd80, db=db@entry=0x14a370187ad0, table_name=table_name@entry=0x14a370187ae0, create_info=create_info@entry=0x14a3b5b3de70, alter_info=0x14a3b5b3dc90, is_trans=0x14a3b5b3dbd0, create_table_mode=0, table_list=0x14a370187ab8) at /test/git-bisect/11.3_dbg/sql/sql_table.cc:4768
#33 0x000055b9baae4f22 in mysql_create_table (alter_info=0x14a3b5b3dc90, create_info=0x14a3b5b3de70, create_table=0x14a370187ab8, thd=0x14a370000d58) at /test/git-bisect/11.3_dbg/sql/sql_table.cc:4884
#34 Sql_cmd_create_table_like::execute (this=<optimized out>, thd=0x14a370000d58) at /test/git-bisect/11.3_dbg/sql/sql_table.cc:12821
#35 0x000055b9baa07b19 in mysql_execute_command (thd=0x14a370000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/git-bisect/11.3_dbg/sql/sql_parse.cc:5733
#36 0x000055b9bac02148 in sp_instr_stmt::exec_core (this=0x14a37000cf28, thd=<optimized out>, nextp=0x14a3b5b403e4) at /test/git-bisect/11.3_dbg/sql/sp_instr.cc:956
#37 0x000055b9bac063f2 in sp_lex_keeper::reset_lex_and_exec_core (this=this@entry=0x14a37000cf60, thd=thd@entry=0x14a370000d58, nextp=nextp@entry=0x14a3b5b403e4, open_tables=open_tables@entry=false, instr=instr@entry=0x14a37000cf28, rerun_the_same_instr=rerun_the_same_instr@entry=false) at /test/git-bisect/11.3_dbg/sql/sp_instr.cc:296
#38 0x000055b9bac06e25 in sp_lex_keeper::validate_lex_and_exec_core (this=this@entry=0x14a37000cf60, thd=thd@entry=0x14a370000d58, nextp=nextp@entry=0x14a3b5b403e4, open_tables=open_tables@entry=false, instr=instr@entry=0x14a37000cf28) at /test/git-bisect/11.3_dbg/sql/sp_instr.cc:475
#39 0x000055b9bac07759 in sp_instr_stmt::execute (this=0x14a37000cf28, thd=0x14a370000d58, nextp=0x14a3b5b403e4) at /test/git-bisect/11.3_dbg/sql/sp_instr.cc:862
#40 0x000055b9ba949382 in sp_head::execute (this=this@entry=0x14a37019b800, thd=thd@entry=0x14a370000d58, merge_da_on_success=merge_da_on_success@entry=true) at /test/git-bisect/11.3_dbg/sql/sp_head.cc:1280
#41 0x000055b9ba94b726 in sp_head::execute_procedure (this=0x14a37019b800, thd=thd@entry=0x14a370000d58, args=0x14a370005fb8) at /test/git-bisect/11.3_dbg/sql/sp_head.cc:2267
#42 0x000055b9ba9f6976 in do_execute_sp (thd=thd@entry=0x14a370000d58, sp=<optimized out>) at /test/git-bisect/11.3_dbg/sql/sql_parse.cc:3025
#43 0x000055b9baa06f94 in mysql_execute_command (thd=thd@entry=0x14a370000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/git-bisect/11.3_dbg/sql/sql_parse.cc:5464
#44 0x000055b9baa093a8 in mysql_parse (thd=thd@entry=0x14a370000d58, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14a3b5b41200) at /test/git-bisect/11.3_dbg/sql/sql_parse.cc:7760
#45 0x000055b9baa0b53c in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14a370000d58, packet=packet@entry=0x14a3700917a9 "", packet_length=packet_length@entry=328, blocking=blocking@entry=true) at /test/git-bisect/11.3_dbg/sql/sql_class.h:247
#46 0x000055b9baa0d417 in do_command (thd=0x14a370000d58, blocking=blocking@entry=true) at /test/git-bisect/11.3_dbg/sql/sql_parse.cc:1406
#47 0x000055b9bab642ae in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55b9bdbb4eb8, put_in_cache=put_in_cache@entry=true) at /test/git-bisect/11.3_dbg/sql/sql_connect.cc:1445
#48 0x000055b9bab6450d in handle_one_connection (arg=0x55b9bdbb4eb8) at /test/git-bisect/11.3_dbg/sql/sql_connect.cc:1347
#49 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#50 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 92 (LWP 2340545):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 91 (LWP 2339303):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 90 (LWP 2324533):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 89 (LWP 2320839):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 88 (LWP 2309931):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 87 (LWP 2296157):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 86 (LWP 2285197):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 85 (LWP 2283372):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 84 (LWP 2282295):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 83 (LWP 2279225):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 82 (LWP 2258990):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 81 (LWP 2240931):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 80 (Thread 0x14a3b56f5640 (LWP 2238763)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b56f4a90, op=137, expected=0, futex_word=0x55b9bd8ab33c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b56f4a90, clockid=0, expected=0, futex_word=0x55b9bd8ab33c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab33c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b56f4a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b56f4a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab310) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b56f4a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab310) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab310, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b56f4a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab310) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab310) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab310) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab310) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab310) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab310, t=t@entry=0x14a3b56f4b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab310) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 79 (Thread 0x14a3633d9640 (LWP 2238760)):
#0 __futex_abstimed_wait_common64 (private=-1114132144, cancel=true, abstime=0x14a3633d8a90, op=137, expected=0, futex_word=0x55b9bd8ac43c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1114132144, abstime=0x14a3633d8a90, clockid=0, expected=0, futex_word=0x55b9bd8ac43c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac43c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3633d8a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3633d8a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac410) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3633d8a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac410) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac410, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3633d8a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac410) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac410) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac410) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac410) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac410) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac410, t=t@entry=0x14a3633d8b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac410) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 78 (LWP 2232648):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 77 (LWP 2202381):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 76 (LWP 2179659):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 75 (LWP 2143496):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 74 (Thread 0x14a3649e4640 (LWP 2142672)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3649e3a90, op=137, expected=0, futex_word=0x55b9bd8ab138) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3649e3a90, clockid=0, expected=0, futex_word=0x55b9bd8ab138) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab138, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3649e3a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3649e3a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab110) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3649e3a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab110) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab110, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3649e3a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab110) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab110) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab110) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab110) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab110) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab110, t=t@entry=0x14a3649e3b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab110) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 73 (LWP 2260373):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 72 (Thread 0x14a3661f0640 (LWP 2142669)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3661efa90, op=137, expected=0, futex_word=0x55b9bd8ac93c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3661efa90, clockid=0, expected=0, futex_word=0x55b9bd8ac93c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac93c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3661efa90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3661efa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac910) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3661efa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac910) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac910, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3661efa90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac910) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac910) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac910) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac910) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac910) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac910, t=t@entry=0x14a3661efb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac910) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 71 (LWP 2239500):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 70 (Thread 0x14a3651e8640 (LWP 2142662)):
#0 __futex_abstimed_wait_common64 (private=1006636928, cancel=true, abstime=0x14a3651e7a90, op=137, expected=0, futex_word=0x55b9bd8ab3bc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=1006636928, abstime=0x14a3651e7a90, clockid=0, expected=0, futex_word=0x55b9bd8ab3bc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab3bc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3651e7a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3651e7a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab390) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3651e7a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab390) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab390, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3651e7a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab390) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab390) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab390) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab390) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab390) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab390, t=t@entry=0x14a3651e7b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab390) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 69 (Thread 0x14a3679fc640 (LWP 2142661)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3679fba90, op=137, expected=0, futex_word=0x55b9bd8ab4b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3679fba90, clockid=0, expected=0, futex_word=0x55b9bd8ab4b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab4b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3679fba90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3679fba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab490) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3679fba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab490) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab490, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3679fba90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab490) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab490) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab490) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab490) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab490) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab490, t=t@entry=0x14a3679fbb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab490) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 68 (Thread 0x14a366df6640 (LWP 2238756)):
#0 __futex_abstimed_wait_common64 (private=-1154830387, cancel=true, abstime=0x14a366df5a90, op=137, expected=0, futex_word=0x55b9bd8ac03c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1154830387, abstime=0x14a366df5a90, clockid=0, expected=0, futex_word=0x55b9bd8ac03c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac03c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a366df5a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a366df5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac010) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a366df5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac010) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac010, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a366df5a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac010) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac010) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac010) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac010) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac010) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac010, t=t@entry=0x14a366df5b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac010) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 67 (Thread 0x14a3b58f6640 (LWP 2142654)):
#0 __futex_abstimed_wait_common64 (private=-1154830387, cancel=true, abstime=0x14a3b58f5a90, op=137, expected=0, futex_word=0x55b9bd8ac2b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1154830387, abstime=0x14a3b58f5a90, clockid=0, expected=0, futex_word=0x55b9bd8ac2b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac2b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b58f5a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b58f5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac290) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b58f5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac290) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac290, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b58f5a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac290) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac290) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac290) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac290) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac290) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac290, t=t@entry=0x14a3b58f5b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac290) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 66 (Thread 0x14a3b5f8f640 (LWP 2238754)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b5f8ea90, op=137, expected=0, futex_word=0x55b9bd8ac9b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b5f8ea90, clockid=0, expected=0, futex_word=0x55b9bd8ac9b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac9b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b5f8ea90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b5f8ea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac990) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b5f8ea90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac990) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac990, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b5f8ea90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac990) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac990) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac990) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac990) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac990) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac990, t=t@entry=0x14a3b5f8eb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac990) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 65 (Thread 0x14a3b79fe640 (LWP 2142651)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b79fda90, op=137, expected=0, futex_word=0x55b9bd8ab538) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b79fda90, clockid=0, expected=0, futex_word=0x55b9bd8ab538) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab538, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b79fda90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b79fda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab510) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b79fda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab510) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab510, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b79fda90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab510) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab510) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab510) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab510) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab510) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab510, t=t@entry=0x14a3b79fdb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab510) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 64 (Thread 0x14a3b44ec640 (LWP 2238753)):
#0 __futex_abstimed_wait_common64 (private=-1154830387, cancel=true, abstime=0x14a3b44eba90, op=137, expected=0, futex_word=0x55b9bd8ac5bc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1154830387, abstime=0x14a3b44eba90, clockid=0, expected=0, futex_word=0x55b9bd8ac5bc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ac5bc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b44eba90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b44eba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac590) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b44eba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ac590) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ac590, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b44eba90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ac590) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac590) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ac590) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ac590) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ac590) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ac590, t=t@entry=0x14a3b44ebb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ac590) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 63 (Thread 0x14a362dd6640 (LWP 2142648)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a362dd5a90, op=137, expected=0, futex_word=0x55b9bd8ab0b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a362dd5a90, clockid=0, expected=0, futex_word=0x55b9bd8ab0b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab0b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a362dd5a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a362dd5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab090) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a362dd5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab090) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab090, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a362dd5a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab090) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab090) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab090) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab090) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab090) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab090, t=t@entry=0x14a362dd5b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab090) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 62 (Thread 0x14a3b5af7640 (LWP 2238751)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3b5af6a90, op=137, expected=0, futex_word=0x55b9bd8ab238) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3b5af6a90, clockid=0, expected=0, futex_word=0x55b9bd8ab238) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab238, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3b5af6a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3b5af6a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab210) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3b5af6a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab210) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab210, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3b5af6a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab210) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab210) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab210) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab210) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab210) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab210, t=t@entry=0x14a3b5af6b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab210) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 61 (LWP 2278029):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 60 (LWP 2147637):
#0 0x0000000000000000 in ?? ()
Backtrace stopped: Cannot access memory at address 0x0

Thread 59 (Thread 0x14a366ff7640 (LWP 2031964)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a366ff6a90, op=137, expected=0, futex_word=0x55b9bd8ab2b8) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a366ff6a90, clockid=0, expected=0, futex_word=0x55b9bd8ab2b8) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab2b8, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a366ff6a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a366ff6a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab290) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a366ff6a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab290) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab290, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a366ff6a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab290) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab290) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab290) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab290) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab290) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab290, t=t@entry=0x14a366ff6b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab290) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 58 (Thread 0x14a3641e0640 (LWP 2238764)):
#0 __futex_abstimed_wait_common64 (private=-1102119387, cancel=true, abstime=0x14a3641dfa90, op=137, expected=0, futex_word=0x55b9bd8acbbc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1102119387, abstime=0x14a3641dfa90, clockid=0, expected=0, futex_word=0x55b9bd8acbbc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8acbbc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3641dfa90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3641dfa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acb90) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3641dfa90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8acb90) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8acb90, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3641dfa90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8acb90) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acb90) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8acb90) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8acb90) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8acb90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8acb90, t=t@entry=0x14a3641dfb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8acb90) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 57 (Thread 0x14a367dfe640 (LWP 2031958)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a367dfda90, op=137, expected=0, futex_word=0x55b9bd8ab83c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a367dfda90, clockid=0, expected=0, futex_word=0x55b9bd8ab83c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab83c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a367dfda90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a367dfda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab810) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a367dfda90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab810) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab810, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a367dfda90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab810) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab810) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab810) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab810) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab810) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab810, t=t@entry=0x14a367dfdb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab810) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 56 (Thread 0x14a3659ec640 (LWP 2142670)):
#0 __futex_abstimed_wait_common64 (private=-744, cancel=true, abstime=0x14a3659eba90, op=137, expected=0, futex_word=0x55b9bd8abe3c) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-744, abstime=0x14a3659eba90, clockid=0, expected=0, futex_word=0x55b9bd8abe3c) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8abe3c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a3659eba90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a3659eba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8abe10) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a3659eba90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8abe10) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8abe10, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a3659eba90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8abe10) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8abe10) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8abe10) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8abe10) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8abe10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8abe10, t=t@entry=0x14a3659ebb38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8abe10) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Thread 55 (Thread 0x14a364de6640 (LWP 2238762)):
#0 __futex_abstimed_wait_common64 (private=-1154830387, cancel=true, abstime=0x14a364de5a90, op=137, expected=0, futex_word=0x55b9bd8ab7bc) at ./nptl/futex-internal.c:57
#1 __futex_abstimed_wait_common (cancel=true, private=-1154830387, abstime=0x14a364de5a90, clockid=0, expected=0, futex_word=0x55b9bd8ab7bc) at ./nptl/futex-internal.c:87
#2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55b9bd8ab7bc, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x14a364de5a90, private=private@entry=0) at ./nptl/futex-internal.c:139
#3 0x000014a3bdc9435d in __pthread_cond_wait_common (abstime=0x14a364de5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab790) at ./nptl/pthread_cond_wait.c:503
#4 ___pthread_cond_clockwait64 (abstime=0x14a364de5a90, clockid=1, mutex=0x55b9bd8aad80, cond=0x55b9bd8ab790) at ./nptl/pthread_cond_wait.c:691
#5 ___pthread_cond_clockwait64 (cond=0x55b9bd8ab790, mutex=0x55b9bd8aad80, clockid=1, abstime=0x14a364de5a90) at ./nptl/pthread_cond_wait.c:679
#6 0x000055b9bb327c6e in std::__condvar::wait_until (__abs_time=..., __clock=1, __m=..., this=0x55b9bd8ab790) at /usr/include/c++/11/bits/std_mutex.h:169
#7 std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab790) at /usr/include/c++/11/condition_variable:201
#8 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (__atime=..., __lock=..., this=0x55b9bd8ab790) at /usr/include/c++/11/condition_variable:111
#9 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (__rtime=..., __lock=..., this=0x55b9bd8ab790) at /usr/include/c++/11/condition_variable:163
#10 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x55b9bd8aabe0, lk=..., thread_data=thread_data@entry=0x55b9bd8ab790) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:480
#11 0x000055b9bb328c9b in tpool::thread_pool_generic::get_task (this=this@entry=0x55b9bd8aabe0, thread_var=thread_var@entry=0x55b9bd8ab790, t=t@entry=0x14a364de5b38) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:533
#12 0x000055b9bb328db8 in tpool::thread_pool_generic::worker_main (this=0x55b9bd8aabe0, thread_var=0x55b9bd8ab790) at /test/git-bisect/11.3_dbg/tpool/tpool_generic.cc:578
#13 0x000055b9bb329c3a in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#14 std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#15 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#16 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#17 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#18 0x000014a3be0dc253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x000014a3bdc94b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#20 0x000014a3bdd26a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
{noformat} ]

Roel Van de Paar made changes - 2023-09-08 07:12

Assignee

Roel Van de Paar [ roel ]

Marko Mäkelä [ marko ]

Julien Fritsch made changes - 2023-09-11 05:57

Fix Version/s		10.4 [ 22408 ]
Fix Version/s		10.5 [ 23123 ]
Fix Version/s		10.6 [ 24028 ]
Fix Version/s		11.0 [ 28320 ]
Fix Version/s		11.2 [ 28603 ]
Fix Version/s	N/A [ 14700 ]

Marko Mäkelä made changes - 2023-09-11 09:57

Status

Stalled [ 10000 ]

Needs Feedback [ 10501 ]

Marko Mäkelä made changes - 2023-09-11 09:57

Assignee

Marko Mäkelä [ marko ]

Roel Van de Paar [ roel ]

Julien Fritsch made changes - 2023-10-04 09:17

Assignee	Roel Van de Paar [ roel ]	Daniel Black [ danblack ]
Status	Needs Feedback [ 10501 ]	Open [ 1 ]

Julien Fritsch made changes - 2024-05-03 14:41

Fix Version/s

11.0 [ 28320 ]

Jira Automation (IT) made changes - 2024-07-04 02:11

Zendesk Related Tickets		201827
Zendesk active tickets		201827

Julien Fritsch made changes - 2024-09-10 15:07

Fix Version/s

10.4 [ 22408 ]

Julien Fritsch made changes - 2024-09-13 08:35

Status

Open [ 1 ]

Needs Feedback [ 10501 ]

Jesper Buhl made changes - 2024-09-16 11:58

Link

This issue relates to ~~MDEV-34938~~ [ ~~MDEV-34938~~ ]

Sergei Golubchik made changes - 2024-10-21 14:45

Fix Version/s		N/A [ 14700 ]
Fix Version/s	10.5 [ 23123 ]
Fix Version/s	10.6 [ 24028 ]
Fix Version/s	11.2 [ 28603 ]
Resolution		Incomplete [ 4 ]
Status	Needs Feedback [ 10501 ]	Closed [ 6 ]

MariaDB Server

InnoDB: Failing assertion: fsize != os_offset_t(-1) on CREATE TABLE with many partitions / ENOMEM (out of kernel memory) from fstat handling

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates

Git Integration