Seppo Jaakola added a comment - 2021-02-03 11:04 Should be fixed by this one commit. Our jenkins is showing two galera related failures: Completed: Failed 2/459 tests, 99.56% were successful. Failing test(s): galera_3nodes_sr.GCF-832 galera_sr.galera_sr_ddl_master

Marko Mäkelä added a comment - 2021-02-03 12:09 The code will be refactored further in MDEV-24731 or MDEV-24738 . I have applied the change in that branch. The hang is gone, but I occasionally see wsrep_assert_no_bf_bf_wait() failures. Is that OK?

Marko Mäkelä added a comment - 2021-02-04 17:30 I included the submitted hang fix in the MDEV-24731 commit that I pushed to 10.6. But, I will keep this ticket open, because test would occasionally crash for me.

Marko Mäkelä added a comment - 2021-02-05 09:40 Here is a recent local failure (10.6 with minor changes to reduce sizeof(trx_t::lock) ), using galera-4 26.4.7-1 in Debian unstable: 10.6 de407e7cb4d12574806bf4f7830e73908d5ca9e2-ish CURRENT_TEST: galera.galera_FK_duplicate_client_insert mysqltest: At line 93: query 'commit' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0... … Program terminated with signal SIGABRT, Aborted. #0 __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56 56 ../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory. [Current thread is 1 (Thread 0x7ffa61599700 (LWP 2146017))] #0 __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56 #1 0x0000564523b45a77 in handle_fatal_signal (sig=6) at /mariadb/10.6-MDEV-515/sql/signal_handler.cc:330 #2 <signal handler called> #3 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #4 0x00007ffa84829537 in __GI_abort () at abort.c:79 #5 0x00005645240a8772 in ut_dbg_assertion_failed (expr=expr@entry=0x0, file=<optimized out>, line=<optimized out>, line@entry=501) at /mariadb/10.6-MDEV-515/storage/innobase/ut/ut0dbg.cc:60 #6 0x0000564523f0ff4c in wsrep_assert_no_bf_bf_wait (lock_rec1=lock_rec1@entry=0x0, lock_rec2=lock_rec2@entry=0x7ffa628c85f0, trx1=trx1@entry=0x7ffa628cc648) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:501 #7 0x0000564523f029a0 in lock_rec_has_to_wait (for_locking=<optimized out>, trx=trx@entry=0x7ffa628cc648, type_mode=type_mode@entry=1027, lock2=lock2@entry=0x7ffa628c85f0, lock_is_on_supremum=false) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:646 #8 0x0000564523f13c41 in lock_rec_other_has_conflicting (mode=mode@entry=1027, block=<optimized out>, block@entry=0x7ffa618d7640, heap_no=heap_no@entry=3, trx=0x2, trx@entry=0x7ffa628cc648) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:957 #9 0x0000564523f16685 in lock_rec_lock (impl=false, mode=1027, mode@entry=1636660800, block=block@entry=0x7ffa618d7640, heap_no=heap_no@entry=3, index=index@entry=0x7ffa04023dd8, thr=0x7ffa04945de8) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:1577 #10 0x0000564523f179bf in lock_clust_rec_read_check_and_lock (flags=<optimized out>, flags@entry=0, block=<optimized out>, block@entry=0x7ffa618d7640, rec=0x7ffa61ddc099 "\200", rec@entry=0x3c210fbf08d51700 <error: Cannot access memory at address 0x3c210fbf08d51700>, index=index@entry=0x7ffa04023dd8, offsets=<optimized out>, offsets@entry=0x0, mode=LOCK_X, mode@entry=32762, gap_mode=1024, thr=0x7ffa04945de8) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:5381 #11 0x0000564524035d05 in sel_set_rec_lock (pcur=<optimized out>, pcur@entry=0x7ffa61ddc099, rec=<optimized out>, rec@entry=0x7ffa61596668 "\001", index=index@entry=0x7ffa04023dd8, offsets=<optimized out>, mode=<optimized out>, type=type@entry=1024, thr=0x7ffa04945de8, mtr=0x7ffa61596668) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0sel.cc:1256 #12 0x0000564524031324 in row_search_mvcc (buf=<optimized out>, buf@entry=0x7ffa04027208 "\375\002", mode=<optimized out>, mode@entry=PAGE_CUR_GE, prebuilt=0x7ffa04945438, match_mode=<optimized out>, direction=<optimized out>, direction@entry=0) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0sel.cc:5122 #13 0x0000564523e7e7f5 in ha_innobase::index_read (this=this@entry=0x7ffa04944bf0, buf=0x2 <error: Cannot access memory at address 0x2>, buf@entry=0x7ffa04027208 "\375\002", key_ptr=key_ptr@entry=0x7ffa04027518 "\002", key_len=4, find_flag=find_flag@entry=HA_READ_KEY_EXACT) at /mariadb/10.6-MDEV-515/storage/innobase/handler/ha_innodb.cc:8578 #14 0x0000564523e7efd6 in ha_innobase::rnd_pos (this=0x7ffa04944bf0, buf=0x7ffa04027208 "\375\002", pos=0x7ffa04027518 "\002") at /mariadb/10.6-MDEV-515/storage/innobase/handler/ha_innodb.cc:9067 #15 0x0000564523b4cc31 in handler::ha_rnd_pos (this=this@entry=0x7ffa04944bf0, buf=<optimized out>, buf@entry=0x7ffa04027208 "\375\002", pos=0x7ffa04027518 "\002") at /mariadb/10.6-MDEV-515/sql/handler.cc:3098 #16 0x0000564523b5b9e7 in handler::rnd_pos_by_record (this=0x7ffa04944bf0, record=0x7ffa04027208 "\375\002") at /mariadb/10.6-MDEV-515/sql/handler.h:3930 #17 0x0000564523c86bb0 in handler::ha_rnd_pos_by_record (this=0x7ffa04944bf0, buf=0x7ffa615954a0 "\ap\b") at /mariadb/10.6-MDEV-515/sql/sql_class.h:7063 #18 Rows_log_event::find_row (this=this@entry=0x7ffa0c01f008, rgi=rgi@entry=0x7ffa0c047840) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:7823 #19 0x0000564523c8828b in Update_rows_log_event::do_exec_row (this=0x7ffa0c01f008, rgi=0x7ffa0c047840) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:8274 #20 0x0000564523c808d3 in Rows_log_event::do_apply_event (this=0x7ffa0c01f008, rgi=<optimized out>) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:5673 #21 0x000056452382ce44 in Log_event::apply_event (this=this@entry=0x7ffa0c01f008, rgi=0x7ffa0c047840) at /mariadb/10.6-MDEV-515/sql/log_event.h:1498 #22 0x0000564523e5ae5b in wsrep_apply_events (thd=thd@entry=0x7ffa0c030b68, rli=<optimized out>, events_buf=<optimized out>, buf_len=<optimized out>) at /mariadb/10.6-MDEV-515/sql/wsrep_applier.cc:208 #23 0x0000564523e45335 in apply_events (thd=0x7ffa0c030b68, rli=0x7ffa615954a0, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}, err=@0x7ffa61597230: {buffer_ = std::vector of length 0, capacity 0}) at /mariadb/10.6-MDEV-515/sql/wsrep_high_priority_service.cc:127 #24 Wsrep_replayer_service::apply_write_set (this=0x7ffa61597f10, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}, err=@0x7ffa61597230: {buffer_ = std::vector of length 0, capacity 0}) at /mariadb/10.6-MDEV-515/sql/wsrep_high_priority_service.cc:709 #25 0x00005645243111dd in apply_write_set (server_state=<optimized out>, high_priority_service=@0x7ffa61597f10: {_vptr$high_priority_service = 0x564524b8f438 <vtable for Wsrep_replayer_service+16>, server_state_ = @0x564526956390, must_exit_ = false}, ws_handle=@0x7ffa61597400: {transaction_id_ = {id_ = 46}, opaque_ = 0x7ffa0c01f400}, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/server_state.cpp:327 #26 wsrep::server_state::on_apply (this=<optimized out>, high_priority_service=@0x7ffa61597f10: {_vptr$high_priority_service = 0x564524b8f438 <vtable for Wsrep_replayer_service+16>, server_state_ = @0x564526956390, must_exit_ = false}, ws_handle=@0x7ffa61597400: {transaction_id_ = {id_ = 46}, opaque_ = 0x7ffa0c01f400}, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/server_state.cpp:1136 #27 0x000056452431c2e3 in wsrep::high_priority_service::apply (this=0x7ffa61597f10, ws_handle=@0x7ffa61597400: {transaction_id_ = {id_ = 46}, opaque_ = 0x7ffa0c01f400}, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}) at /mariadb/10.6-MDEV-515/wsrep-lib/include/wsrep/high_priority_service.hpp:47 #28 (anonymous namespace)::apply_cb (ctx=0x7ffa61597f10, wsh=<optimized out>, flags=<optimized out>, buf=<optimized out>, meta=<optimized out>, exit_loop=0x7ffa615978df) at /mariadb/10.6-MDEV-515/wsrep-lib/src/wsrep_provider_v26.cpp:502 #29 0x00007ffa801e3111 in ?? () from /usr/lib/galera/libgalera_smm.so #30 0x00007ffa801f7e03 in ?? () from /usr/lib/galera/libgalera_smm.so #31 0x00007ffa801d580b in ?? () from /usr/lib/galera/libgalera_smm.so #32 0x000056452431d1d1 in wsrep::wsrep_provider_v26::replay (this=<optimized out>, ws_handle=<optimized out>, reply_service=0x0) at /mariadb/10.6-MDEV-515/wsrep-lib/src/wsrep_provider_v26.cpp:870 #33 0x0000564523e426e5 in Wsrep_client_service::replay (this=0x7ffa0c0071b0) at /mariadb/10.6-MDEV-515/sql/wsrep_client_service.cc:280 #34 0x0000564524317d90 in wsrep::transaction::replay (this=this@entry=0x7ffa0c007230, lock=@0x7ffa615983b0: {_M_device = 0x7ffa0c007190, _M_owns = false}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/transaction.cpp:1889 #35 0x000056452431768f in wsrep::transaction::after_statement (this=0x7ffa0c007230) at /mariadb/10.6-MDEV-515/wsrep-lib/src/transaction.cpp:856 #36 0x000056452430943b in wsrep::client_state::after_statement (this=0x7ffa0c0071c8) at /mariadb/10.6-MDEV-515/wsrep-lib/src/client_state.cpp:276 #37 0x00005645238e5db3 in wsrep_after_statement (thd=0x7ffa0c000d48) at /mariadb/10.6-MDEV-515/sql/wsrep_trans_observer.h:426 #38 wsrep_mysql_parse (thd=thd@entry=0x7ffa0c000d48, rawbuf=0x7ffa0c0181c0 "commit", length=6, parser_state=parser_state@entry=0x7ffa61598620) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:7750 #39 0x00005645238e35ee in dispatch_command (command=<optimized out>, command@entry=COM_QUERY, thd=thd@entry=0x7ffa0c000d48, packet=packet@entry=0x7ffa0c010149 "commit", packet_length=packet_length@entry=6) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1820 #40 0x00005645238e4d77 in do_command (thd=0x7ffa0c000d48) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1365 #41 0x0000564523a0d7e9 in do_handle_one_connection (connect=connect@entry=0x564526a9e3f8, put_in_cache=<optimized out>) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1410 #42 0x0000564523a0d65c in handle_one_connection (arg=arg@entry=0x564526a9e3f8) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1312 #43 0x0000564523de01df in pfs_spawn_thread (arg=0x564526f72538) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #44 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #45 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 27 (Thread 0x7ffa6162f700 (LWP 2131139)): #0 0x00007ffa848f73ff in __GI___poll (fds=fds@entry=0x7ffa6162ea98, nfds=nfds@entry=1, timeout=timeout@entry=28800000) at ../sysdeps/unix/sysv/linux/poll.c:29 #1 0x0000564523d2660d in vio_io_wait (vio=vio@entry=0x564527027a18, event=<optimized out>, event@entry=VIO_IO_EVENT_READ, timeout=28800000) at /mariadb/10.6-MDEV-515/vio/viosocket.c:948 #2 0x0000564523d2682d in vio_socket_io_wait (vio=0x564527027a18, event=VIO_IO_EVENT_READ) at /mariadb/10.6-MDEV-515/vio/viosocket.c:116 #3 vio_read (vio=0x564527027a18, buf=<optimized out>, size=<optimized out>) at /mariadb/10.6-MDEV-515/vio/viosocket.c:182 #4 0x0000564523c939c5 in my_real_read (net=net@entry=0x7ffa18001058, complen=<optimized out>, complen@entry=0x7ffa6162ecd0, header=<optimized out>) at /mariadb/10.6-MDEV-515/sql/net_serv.cc:979 #5 0x0000564523c93683 in my_net_read_packet_reallen (net=0x7ffa18001058, read_from_server=<optimized out>, reallen=reallen@entry=0x7ffa6162ed20) at /mariadb/10.6-MDEV-515/sql/net_serv.cc:1260 #6 0x0000564523c93611 in my_net_read_packet (net=0x7ffa6162ea98, read_from_server=1 '\001') at /mariadb/10.6-MDEV-515/sql/net_serv.cc:1244 #7 0x00005645238e4c81 in do_command (thd=0x7ffa18000d48) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1229 #8 0x0000564523a0d7e9 in do_handle_one_connection (connect=0x564526f08968, connect@entry=0x564526ff0ff8, put_in_cache=<optimized out>) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1410 #9 0x0000564523a0d65c in handle_one_connection (arg=arg@entry=0x564526ff0ff8) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1312 #10 0x0000564523de01df in pfs_spawn_thread (arg=0x564527027a18) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 26 (Thread 0x7ffa615e4700 (LWP 2145998)): #0 0x00007ffa848f73ff in __GI___poll (fds=fds@entry=0x7ffa615e3a98, nfds=nfds@entry=1, timeout=timeout@entry=28800000) at ../sysdeps/unix/sysv/linux/poll.c:29 #1 0x0000564523d2660d in vio_io_wait (vio=vio@entry=0x5645269ef228, event=<optimized out>, event@entry=VIO_IO_EVENT_READ, timeout=28800000) at /mariadb/10.6-MDEV-515/vio/viosocket.c:948 #2 0x0000564523d2682d in vio_socket_io_wait (vio=0x5645269ef228, event=VIO_IO_EVENT_READ) at /mariadb/10.6-MDEV-515/vio/viosocket.c:116 #3 vio_read (vio=0x5645269ef228, buf=<optimized out>, size=<optimized out>) at /mariadb/10.6-MDEV-515/vio/viosocket.c:182 #4 0x0000564523c939c5 in my_real_read (net=net@entry=0x7ffa04001058, complen=<optimized out>, complen@entry=0x7ffa615e3cd0, header=<optimized out>) at /mariadb/10.6-MDEV-515/sql/net_serv.cc:979 #5 0x0000564523c93683 in my_net_read_packet_reallen (net=0x7ffa04001058, read_from_server=<optimized out>, reallen=reallen@entry=0x7ffa615e3d20) at /mariadb/10.6-MDEV-515/sql/net_serv.cc:1260 #6 0x0000564523c93611 in my_net_read_packet (net=0x7ffa615e3a98, read_from_server=1 '\001') at /mariadb/10.6-MDEV-515/sql/net_serv.cc:1244 #7 0x00005645238e4c81 in do_command (thd=0x7ffa04000d48) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1229 #8 0x0000564523a0d7e9 in do_handle_one_connection (connect=connect@entry=0x564526f0bcf8, put_in_cache=<optimized out>) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1410 #9 0x0000564523a0d65c in handle_one_connection (arg=arg@entry=0x564526f0bcf8) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1312 #10 0x0000564523de01df in pfs_spawn_thread (arg=0x5645269ef228) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 25 (Thread 0x7ffa614c5700 (LWP 2130702)): #0 futex_wait_cancelable (private=0, expected=0, futex_word=0x564524bd08e0 <buf_pool+1120>) at ../sysdeps/nptl/futex-internal.h:186 #1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x564524bd07e8 <buf_pool+872>, cond=0x564524bd08b8 <buf_pool+1080>) at pthread_cond_wait.c:508 #2 __pthread_cond_wait (cond=cond@entry=0x564524bd08b8 <buf_pool+1080>, mutex=mutex@entry=0x564524bd07e8 <buf_pool+872>) at pthread_cond_wait.c:638 #3 0x000056452429938c in safe_cond_wait (cond=0x564524bd08e0 <buf_pool+1120>, mp=0x564524bd07c0 <buf_pool+832>, file=0x564524329b68 "/mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h", line=1233) at /mariadb/10.6-MDEV-515/mysys/thr_mutex.c:492 #4 0x0000564524135811 in inline_mysql_cond_wait (mutex=<optimized out>, src_file=<optimized out>, src_line=2092, that=<optimized out>) at /mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h:1233 #5 buf_flush_page_cleaner () at /mariadb/10.6-MDEV-515/storage/innobase/buf/buf0flu.cc:2092 #6 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #7 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 24 (Thread 0x7ffa537fe700 (LWP 2130811)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa537fdd58, clockid=1400888496, expected=0, futex_word=0x564526ac1e5c) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa537fdd58, clockid=1400888496, mutex=0x564526ab23a0, cond=0x564526ac1e30) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa537fdd58, clockid=1400888496, mutex=0x564526ab23a0, cond=0x564526ac1e30) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1e30, mutex=0x564526ab23a0, clockid=1400888496, abstime=0x7ffa537fdd58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1e30, __lock=@0x7ffa537fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1e30, __lock=@0x7ffa537fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1e30, __lock=@0x7ffa537fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa537fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1e30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1e30, t=t@entry=0x7ffa537fddf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1e30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 23 (Thread 0x7ffa43fff700 (LWP 2130834)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa43ffed58, clockid=1140845744, expected=0, futex_word=0x564526ac1b58) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa43ffed58, clockid=1140845744, mutex=0x564526ab23a0, cond=0x564526ac1b30) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa43ffed58, clockid=1140845744, mutex=0x564526ab23a0, cond=0x564526ac1b30) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1b30, mutex=0x564526ab23a0, clockid=1140845744, abstime=0x7ffa43ffed58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1b30, __lock=@0x7ffa43ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1b30, __lock=@0x7ffa43ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1b30, __lock=@0x7ffa43ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa43ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1b30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1b30, t=t@entry=0x7ffa43ffedf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1b30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 22 (Thread 0x7ffa608ba700 (LWP 2130755)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa608b9d58, clockid=1619762352, expected=0, futex_word=0x564526ac1f58) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa608b9d58, clockid=1619762352, mutex=0x564526ab23a0, cond=0x564526ac1f30) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa608b9d58, clockid=1619762352, mutex=0x564526ab23a0, cond=0x564526ac1f30) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1f30, mutex=0x564526ab23a0, clockid=1619762352, abstime=0x7ffa608b9d58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1f30, __lock=@0x7ffa608b9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1f30, __lock=@0x7ffa608b9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1f30, __lock=@0x7ffa608b9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa608b9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1f30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1f30, t=t@entry=0x7ffa608b9df8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1f30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 21 (Thread 0x7ffa616c5700 (LWP 2130922)): #0 0x00007ffa84840ba2 in __GI___sigtimedwait (set=0x7ffa616c4d68, info=0x7ffa616c4ce8, timeout=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29 #1 0x00005645237f6dfa in my_sigwait (set=0x7ffa616c4d68, sig=0x7ffa616c4cd0, code=<optimized out>) at /mariadb/10.6-MDEV-515/include/my_pthread.h:195 #2 signal_hand (arg=<optimized out>) at /mariadb/10.6-MDEV-515/sql/mysqld.cc:3031 #3 0x0000564523de01df in pfs_spawn_thread (arg=0x564526a991e8) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #4 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 20 (Thread 0x7ffa517fa700 (LWP 2130823)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa517f9d58, clockid=1367317680, expected=0, futex_word=0x564526ac1c5c) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa517f9d58, clockid=1367317680, mutex=0x564526ab23a0, cond=0x564526ac1c30) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa517f9d58, clockid=1367317680, mutex=0x564526ab23a0, cond=0x564526ac1c30) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1c30, mutex=0x564526ab23a0, clockid=1367317680, abstime=0x7ffa517f9d58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1c30, __lock=@0x7ffa517f9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1c30, __lock=@0x7ffa517f9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1c30, __lock=@0x7ffa517f9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa517f9da8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1c30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1c30, t=t@entry=0x7ffa517f9df8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1c30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 19 (Thread 0x7ffa3bfff700 (LWP 2130840)): #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x0000564524069010 in ssux_lock_low::writer_wait (this=0x7ffa628cc668, l=3221225472) at /mariadb/10.6-MDEV-515/storage/innobase/sync/srw_lock.cc:99 #2 ssux_lock_low::write_lock (this=0x7ffa628cc668, holding_u=false) at /mariadb/10.6-MDEV-515/storage/innobase/sync/srw_lock.cc:228 #3 0x0000564523f1fa40 in ssux_lock_low::wr_lock (this=0x7ffa628cc668) at /mariadb/10.6-MDEV-515/storage/innobase/include/srw_lock.h:96 #4 rw_trx_hash_t::validate_element (trx=0x7ffa628cc648) at /mariadb/10.6-MDEV-515/storage/innobase/include/trx0sys.h:520 #5 0x0000564523f958b8 in rw_trx_hash_t::debug_iterator<trx_sys_t::snapshot_ids_arg> (element=0x7ffa0c04bc18, arg=0x7ffa3bffe6d8) at /mariadb/10.6-MDEV-515/storage/innobase/include/trx0sys.h:542 #6 0x000056452429dd11 in l_find (head=0x7ffa6403cc68, head@entry=0x564526c09ef0, cs=cs@entry=0x0, hashnr=hashnr@entry=0, key=key@entry=0x7ffa3bffe6d8 "pU\371#EV", keylen=keylen@entry=0, cursor=0x7ffa3bffe680, cursor@entry=0x7ffa3bffe670, pins=0x564526c02990, callback=0x564523f95870 <rw_trx_hash_t::debug_iterator<trx_sys_t::snapshot_ids_arg>(rw_trx_hash_element_t*, rw_trx_hash_t::debug_iterator_arg<trx_sys_t::snapshot_ids_arg>*)>) at /mariadb/10.6-MDEV-515/mysys/lf_hash.c:126 #7 0x000056452429db13 in lf_hash_iterate (hash=<optimized out>, pins=0x564526c02990, action=0x564523f95870 <rw_trx_hash_t::debug_iterator<trx_sys_t::snapshot_ids_arg>(rw_trx_hash_element_t*, rw_trx_hash_t::debug_iterator_arg<trx_sys_t::snapshot_ids_arg>*)>, argument=0x7ffa3bffe6d8) at /mariadb/10.6-MDEV-515/mysys/lf_hash.c:518 #8 0x0000564523f9552d in rw_trx_hash_t::iterate<trx_sys_t::snapshot_ids_arg> (this=this@entry=0x564524bd0000 <trx_sys+2496>, caller_trx=caller_trx@entry=0x0, action=0x564523f95570 <trx_sys_t::copy_one_id(rw_trx_hash_element_t*, trx_sys_t::snapshot_ids_arg*)>, argument=0x7ffa3bffe6d8, argument@entry=0x7ffa3bffe728) at /mariadb/10.6-MDEV-515/storage/innobase/include/trx0sys.h:760 #9 0x0000564523f950a3 in trx_sys_t::snapshot_ids (this=<optimized out>, caller_trx=<optimized out>, ids=<optimized out>, max_trx_id=0x564525521310 <purge_sys+16>, min_trx_no=0x564525521340 <purge_sys+64>) at /mariadb/10.6-MDEV-515/storage/innobase/include/trx0sys.h:1024 #10 ReadViewBase::snapshot (this=0x564525521310 <purge_sys+16>, trx=trx@entry=0x0) at /mariadb/10.6-MDEV-515/storage/innobase/read/read0read.cc:175 #11 0x0000564523f94bae in trx_sys_t::clone_oldest_view (this=0x564524bcf640 <trx_sys>, view=0x564525521310 <purge_sys+16>) at /mariadb/10.6-MDEV-515/storage/innobase/read/read0read.cc:247 #12 0x000056452406d83b in purge_sys_t::clone_oldest_view (this=<optimized out>) at /mariadb/10.6-MDEV-515/storage/innobase/include/trx0purge.h:263 #13 trx_purge (n_tasks=4, truncate=false) at /mariadb/10.6-MDEV-515/storage/innobase/trx/trx0purge.cc:1279 #14 0x000056452405cd1f in srv_do_purge (n_total_purged=<optimized out>) at /mariadb/10.6-MDEV-515/storage/innobase/srv/srv0srv.cc:1895 #15 purge_coordinator_callback_low () at /mariadb/10.6-MDEV-515/storage/innobase/srv/srv0srv.cc:1985 #16 purge_coordinator_callback () at /mariadb/10.6-MDEV-515/storage/innobase/srv/srv0srv.cc:2014 #17 0x0000564524227a1d in tpool::task_group::execute (this=0x564525520420 <purge_coordinator_task_group>, t=0x5645255204b0 <purge_coordinator_task>) at /mariadb/10.6-MDEV-515/tpool/task_group.cc:55 #18 0x0000564524225586 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1ab0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:546 #19 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #20 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #21 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 18 (Thread 0x7ffa803e9700 (LWP 2129907)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa803e8da0, clockid=-2143384368, expected=0, futex_word=0x56452552c65c <COND_timer+44>) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa803e8da0, clockid=-2143384368, mutex=0x56452552c5a8 <LOCK_timer+40>, cond=0x56452552c630 <COND_timer>) at pthread_cond_wait.c:520 #2 __pthread_cond_timedwait (cond=cond@entry=0x56452552c630 <COND_timer>, mutex=mutex@entry=0x56452552c5a8 <LOCK_timer+40>, abstime=abstime@entry=0x7ffa803e8da0) at pthread_cond_wait.c:656 #3 0x00005645242995d4 in safe_cond_timedwait (cond=cond@entry=0x56452552c630 <COND_timer>, mp=0x56452552c580 <LOCK_timer>, abstime=abstime@entry=0x7ffa803e8da0, file=0x564524329b68 "/mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h", line=line@entry=1259) at /mariadb/10.6-MDEV-515/mysys/thr_mutex.c:546 #4 0x000056452429a000 in inline_mysql_cond_timedwait (that=<optimized out>, mutex=<optimized out>, abstime=0x7ffa803e8da0, src_file=<optimized out>, src_line=321) at /mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h:1259 #5 timer_handler (arg=arg@entry=0x0) at /mariadb/10.6-MDEV-515/mysys/thr_timer.c:321 #6 0x0000564523de01df in pfs_spawn_thread (arg=0x56452683a828) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #7 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #8 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 17 (Thread 0x7ffa53fff700 (LWP 2130810)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa53ffed58, clockid=1409281200, expected=0, futex_word=0x564526ac1ed8) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa53ffed58, clockid=1409281200, mutex=0x564526ab23a0, cond=0x564526ac1eb0) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa53ffed58, clockid=1409281200, mutex=0x564526ab23a0, cond=0x564526ac1eb0) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1eb0, mutex=0x564526ab23a0, clockid=1409281200, abstime=0x7ffa53ffed58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1eb0, __lock=@0x7ffa53ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1eb0, __lock=@0x7ffa53ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1eb0, __lock=@0x7ffa53ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa53ffeda8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1eb0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1eb0, t=t@entry=0x7ffa53ffedf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1eb0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 16 (Thread 0x7ffa527fc700 (LWP 2130815)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa527fbd58, clockid=1384103088, expected=0, futex_word=0x564526ac1d5c) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa527fbd58, clockid=1384103088, mutex=0x564526ab23a0, cond=0x564526ac1d30) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa527fbd58, clockid=1384103088, mutex=0x564526ab23a0, cond=0x564526ac1d30) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1d30, mutex=0x564526ab23a0, clockid=1384103088, abstime=0x7ffa527fbd58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1d30, __lock=@0x7ffa527fbda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1d30, __lock=@0x7ffa527fbda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1d30, __lock=@0x7ffa527fbda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa527fbda8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1d30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1d30, t=t@entry=0x7ffa527fbdf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1d30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 15 (Thread 0x7ffa51ffb700 (LWP 2130820)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa51ffad58, clockid=1375710384, expected=0, futex_word=0x564526ac1cdc) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa51ffad58, clockid=1375710384, mutex=0x564526ab23a0, cond=0x564526ac1cb0) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa51ffad58, clockid=1375710384, mutex=0x564526ab23a0, cond=0x564526ac1cb0) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1cb0, mutex=0x564526ab23a0, clockid=1375710384, abstime=0x7ffa51ffad58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1cb0, __lock=@0x7ffa51ffada8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1cb0, __lock=@0x7ffa51ffada8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1cb0, __lock=@0x7ffa51ffada8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa51ffada8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1cb0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1cb0, t=t@entry=0x7ffa51ffadf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1cb0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 14 (Thread 0x7ffa800e9700 (LWP 2129989)): #0 futex_wait_cancelable (private=0, expected=0, futex_word=0x7ffa70015b90) at ../sysdeps/nptl/futex-internal.h:186 #1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7ffa70015ae0, cond=0x7ffa70015b68) at pthread_cond_wait.c:508 #2 __pthread_cond_wait (cond=cond@entry=0x7ffa70015b68, mutex=mutex@entry=0x7ffa70015ae0) at pthread_cond_wait.c:638 #3 0x000056452429938c in safe_cond_wait (cond=0x7ffa70015b90, mp=0x7ffa70015ab8, file=0x564524329b68 "/mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h", line=1233) at /mariadb/10.6-MDEV-515/mysys/thr_mutex.c:492 #4 0x0000564523e5ce21 in inline_mysql_cond_wait (that=0x7ffa70015b68, mutex=0x7ffa70015ab8, src_file=<optimized out>, src_line=69) at /mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h:1233 #5 Wsrep_thd_queue::pop_front (this=0x7ffa70015a60) at /mariadb/10.6-MDEV-515/sql/wsrep_thd.h:69 #6 0x0000564523e5b800 in wsrep_rollback_process (rollbacker=0x7ffa70000d48, arg=<optimized out>) at /mariadb/10.6-MDEV-515/sql/wsrep_thd.cc:241 #7 0x0000564523e51cdb in start_wsrep_THD (arg=arg@entry=0x5645269d4e00) at /mariadb/10.6-MDEV-515/sql/wsrep_mysqld.cc:3089 #8 0x0000564523de01df in pfs_spawn_thread (arg=0x5645269ee758) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #9 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #10 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 13 (Thread 0x7ffa634c8700 (LWP 2130002)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa634c7d30, clockid=1665956960, expected=0, futex_word=0x56452548d6f8 <COND_checkpoint+40>) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa634c7d30, clockid=1665956960, mutex=0x56452548d640 <LOCK_checkpoint+40>, cond=0x56452548d6d0 <COND_checkpoint>) at pthread_cond_wait.c:520 #2 __pthread_cond_timedwait (cond=cond@entry=0x56452548d6d0 <COND_checkpoint>, mutex=mutex@entry=0x56452548d640 <LOCK_checkpoint+40>, abstime=abstime@entry=0x7ffa634c7d30) at pthread_cond_wait.c:656 #3 0x00005645242995d4 in safe_cond_timedwait (cond=cond@entry=0x56452548d6d0 <COND_checkpoint>, mp=mp@entry=0x56452548d618 <LOCK_checkpoint>, abstime=abstime@entry=0x7ffa634c7d30, file=0x564524329b68 "/mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h", line=line@entry=1259) at /mariadb/10.6-MDEV-515/mysys/thr_mutex.c:546 #4 0x0000564523d6df7f in inline_mysql_cond_timedwait (that=0x56452548d6d0 <COND_checkpoint>, mutex=0x56452548d618 <LOCK_checkpoint>, abstime=0x7ffa634c7d30, src_file=<optimized out>, src_line=116) at /mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h:1259 #5 my_service_thread_sleep (control=control@entry=0x564524bc3f00 <checkpoint_control>, sleep_time=<optimized out>) at /mariadb/10.6-MDEV-515/storage/maria/ma_servicethread.c:115 #6 0x0000564523d658d1 in ma_checkpoint_background (arg=<optimized out>, arg@entry=0x1e) at /mariadb/10.6-MDEV-515/storage/maria/ma_checkpoint.c:707 #7 0x0000564523de01df in pfs_spawn_thread (arg=0x564526a985a8) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #8 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #9 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 12 (Thread 0x7ffa628c6700 (LWP 2130670)): #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x0000564524228464 in my_getevents (ctx=<optimized out>, min_nr=1, nr=256, ev=0x7ffa628c3df0) at /mariadb/10.6-MDEV-515/tpool/aio_linux.cc:63 #2 tpool::aio_linux::getevent_thread_routine (aio=0x5645269c4060) at /mariadb/10.6-MDEV-515/tpool/aio_linux.cc:107 #3 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #4 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 11 (Thread 0x7ffa6154e700 (LWP 2146018)): #0 0x00007ffa848f73ff in __GI___poll (fds=fds@entry=0x7ffa6154da98, nfds=nfds@entry=1, timeout=timeout@entry=28800000) at ../sysdeps/unix/sysv/linux/poll.c:29 #1 0x0000564523d2660d in vio_io_wait (vio=vio@entry=0x564526cd3748, event=<optimized out>, event@entry=VIO_IO_EVENT_READ, timeout=28800000) at /mariadb/10.6-MDEV-515/vio/viosocket.c:948 #2 0x0000564523d2682d in vio_socket_io_wait (vio=0x564526cd3748, event=VIO_IO_EVENT_READ) at /mariadb/10.6-MDEV-515/vio/viosocket.c:116 #3 vio_read (vio=0x564526cd3748, buf=<optimized out>, size=<optimized out>) at /mariadb/10.6-MDEV-515/vio/viosocket.c:182 #4 0x0000564523c939c5 in my_real_read (net=net@entry=0x7ffa10002198, complen=<optimized out>, complen@entry=0x7ffa6154dcd0, header=<optimized out>) at /mariadb/10.6-MDEV-515/sql/net_serv.cc:979 #5 0x0000564523c93683 in my_net_read_packet_reallen (net=0x7ffa10002198, read_from_server=<optimized out>, reallen=reallen@entry=0x7ffa6154dd20) at /mariadb/10.6-MDEV-515/sql/net_serv.cc:1260 #6 0x0000564523c93611 in my_net_read_packet (net=0x7ffa6154da98, read_from_server=1 '\001') at /mariadb/10.6-MDEV-515/sql/net_serv.cc:1244 #7 0x00005645238e4c81 in do_command (thd=0x7ffa10001e88) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1229 #8 0x0000564523a0d7e9 in do_handle_one_connection (connect=connect@entry=0x564526a9e468, put_in_cache=<optimized out>) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1410 #9 0x0000564523a0d65c in handle_one_connection (arg=arg@entry=0x564526a9e468) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1312 #10 0x0000564523de01df in pfs_spawn_thread (arg=0x564526cd3748) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 10 (Thread 0x7ffa50ff9700 (LWP 2130825)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa50ff8d58, clockid=1358924976, expected=0, futex_word=0x564526ac1bd8) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa50ff8d58, clockid=1358924976, mutex=0x564526ab23a0, cond=0x564526ac1bb0) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa50ff8d58, clockid=1358924976, mutex=0x564526ab23a0, cond=0x564526ac1bb0) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1bb0, mutex=0x564526ab23a0, clockid=1358924976, abstime=0x7ffa50ff8d58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1bb0, __lock=@0x7ffa50ff8da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1bb0, __lock=@0x7ffa50ff8da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1bb0, __lock=@0x7ffa50ff8da8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa50ff8da8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1bb0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1bb0, t=t@entry=0x7ffa50ff8df8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1bb0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 9 (Thread 0x7ffa437fe700 (LWP 2130842)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa437fdd58, clockid=1132453040, expected=0, futex_word=0x564526ac1a58) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa437fdd58, clockid=1132453040, mutex=0x564526ab23a0, cond=0x564526ac1a30) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa437fdd58, clockid=1132453040, mutex=0x564526ab23a0, cond=0x564526ac1a30) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1a30, mutex=0x564526ab23a0, clockid=1132453040, abstime=0x7ffa437fdd58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1a30, __lock=@0x7ffa437fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1a30, __lock=@0x7ffa437fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1a30, __lock=@0x7ffa437fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa437fdda8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1a30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1a30, t=t@entry=0x7ffa437fddf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1a30) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 8 (Thread 0x7ffa78878700 (LWP 2129988)): #0 futex_wait_cancelable (private=0, expected=0, futex_word=0x5645269d5488) at ../sysdeps/nptl/futex-internal.h:186 #1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x5645269d5428, cond=0x5645269d5460) at pthread_cond_wait.c:508 #2 __pthread_cond_wait (cond=0x5645269d5460, mutex=0x5645269d5428) at pthread_cond_wait.c:638 #3 0x00007ffa80266747 in ?? () from /usr/lib/galera/libgalera_smm.so #4 0x00007ffa8025437b in ?? () from /usr/lib/galera/libgalera_smm.so #5 0x00007ffa8024f645 in ?? () from /usr/lib/galera/libgalera_smm.so #6 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #7 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 7 (Thread 0x7ffa8009e700 (LWP 2129990)): #0 __lll_lock_wait (futex=futex@entry=0x5645254fb4a8 <lock_sys+104>, private=0) at lowlevellock.c:52 #1 0x00007ffa852d7843 in __GI___pthread_mutex_lock (mutex=0x5645254fb4a8 <lock_sys+104>) at ../nptl/pthread_mutex_lock.c:80 #2 0x000056452429899f in safe_mutex_lock (mp=0x5645254fb480 <lock_sys+64>, my_flags=0, file=0x56452456a089 "/mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc", line=369) at /mariadb/10.6-MDEV-515/mysys/thr_mutex.c:290 #3 0x0000564523f01f4c in inline_mysql_mutex_lock (that=0x5645254fb480 <lock_sys+64>, src_file=<optimized out>, src_line=369) at /mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h:750 #4 lock_sys_t::mutex_lock (this=<optimized out>) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:369 #5 0x0000564523f1946a in lock_trx_has_sys_table_locks (trx=trx@entry=0x7ffa628c8168) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:5792 #6 0x0000564523feea7c in row_mysql_unfreeze_data_dictionary (trx=trx@entry=0x7ffa628c8168) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0mysql.cc:2031 #7 0x0000564523fcd0dc in row_ins_check_foreign_constraints (table=0x7ffa049097f8, index=<optimized out>, index@entry=0x7ffa0490cbd8, pk=false, entry=entry@entry=0x7ffa64020408, thr=<optimized out>, thr@entry=0x7ffa64025158) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0ins.cc:1988 #8 0x0000564523fcd766 in row_ins_sec_index_entry (index=0x7ffa0490cbd8, entry=0x7ffa64020408, thr=0x7ffa64025158, check_foreign=true) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0ins.cc:3306 #9 0x0000564523fcf173 in row_ins_index_entry (index=0x7ffa0490cbd8, entry=0x7ffa64020408, thr=0x7ffa64025158) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0ins.cc:3382 #10 row_ins_index_entry_step (node=0x7ffa64024f00, thr=0x7ffa64025158) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0ins.cc:3549 #11 row_ins (node=0x7ffa64024f00, thr=<optimized out>) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0ins.cc:3695 #12 row_ins_step (thr=<optimized out>, thr@entry=0x7ffa64025158) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0ins.cc:3837 #13 0x0000564523fed1c0 in row_insert_for_mysql (mysql_rec=<optimized out>, mysql_rec@entry=0x7ffa6403cd40 "\371\002", prebuilt=<optimized out>, ins_mode=<optimized out>) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0mysql.cc:1384 #14 0x0000564523e7ad48 in ha_innobase::write_row (this=0x7ffa6403e5d0, record=0x7ffa6403cd40 "\371\002") at /mariadb/10.6-MDEV-515/storage/innobase/handler/ha_innodb.cc:7370 #15 0x0000564523b57876 in handler::ha_write_row (this=0x7ffa6403e5d0, buf=<optimized out>) at /mariadb/10.6-MDEV-515/sql/handler.cc:7151 #16 0x0000564523c85cd5 in Rows_log_event::write_row (this=this@entry=0x7ffa6403d398, rgi=rgi@entry=0x7ffa6401f660, overwrite=false) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:7281 #17 0x0000564523c8665b in Write_rows_log_event::do_exec_row (this=0x7ffa6403d398, rgi=0x7ffa6401f660) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:7501 #18 0x0000564523c808d3 in Rows_log_event::do_apply_event (this=0x7ffa6403d398, rgi=<optimized out>) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:5673 #19 0x000056452382ce44 in Log_event::apply_event (this=this@entry=0x7ffa6403d398, rgi=0x7ffa6401f660) at /mariadb/10.6-MDEV-515/sql/log_event.h:1498 #20 0x0000564523e5ae5b in wsrep_apply_events (thd=thd@entry=0x7ffa64000d48, rli=<optimized out>, events_buf=<optimized out>, buf_len=<optimized out>) at /mariadb/10.6-MDEV-515/sql/wsrep_applier.cc:208 #21 0x0000564523e44755 in apply_events (thd=0x7ffa64000d48, rli=0x80, data=@0x7ffa8009cef0: {ptr_ = 0x7ffa7b600100, size_ = 167}, err=@0x7ffa8009cd10: {buffer_ = std::vector of length 0, capacity 0}) at /mariadb/10.6-MDEV-515/sql/wsrep_high_priority_service.cc:127 #22 Wsrep_applier_service::apply_write_set (this=0x7ffa8009dcf8, ws_meta=@0x7ffa8009d090: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 8}}, stid_ = {server_id_ = {data_ = {buf = "\240\v\206\316g\225\021\353\223\375~\307\316\264i\250"}}, transaction_id_ = {id_ = 35}, client_id_ = {id_ = 11}}, depends_on_ = {seqno_ = 6}, flags_ = 3}, data=@0x7ffa8009cef0: {ptr_ = 0x7ffa7b600100, size_ = 167}, err=@0x7ffa8009cd10: {buffer_ = std::vector of length 0, capacity 0}) at /mariadb/10.6-MDEV-515/sql/wsrep_high_priority_service.cc:558 #23 0x00005645243111dd in apply_write_set (server_state=<optimized out>, high_priority_service=@0x7ffa8009dcf8: {_vptr$high_priority_service = 0x564524b8f358 <vtable for Wsrep_applier_service+16>, server_state_ = @0x564526956390, must_exit_ = false}, ws_handle=@0x7ffa8009cee0: {transaction_id_ = {id_ = 35}, opaque_ = 0x7ffa64036b50}, ws_meta=@0x7ffa8009d090: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 8}}, stid_ = {server_id_ = {data_ = {buf = "\240\v\206\316g\225\021\353\223\375~\307\316\264i\250"}}, transaction_id_ = {id_ = 35}, client_id_ = {id_ = 11}}, depends_on_ = {seqno_ = 6}, flags_ = 3}, data=@0x7ffa8009cef0: {ptr_ = 0x7ffa7b600100, size_ = 167}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/server_state.cpp:327 #24 wsrep::server_state::on_apply (this=<optimized out>, high_priority_service=@0x7ffa8009dcf8: {_vptr$high_priority_service = 0x564524b8f358 <vtable for Wsrep_applier_service+16>, server_state_ = @0x564526956390, must_exit_ = false}, ws_handle=@0x7ffa8009cee0: {transaction_id_ = {id_ = 35}, opaque_ = 0x7ffa64036b50}, ws_meta=@0x7ffa8009d090: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 8}}, stid_ = {server_id_ = {data_ = {buf = "\240\v\206\316g\225\021\353\223\375~\307\316\264i\250"}}, transaction_id_ = {id_ = 35}, client_id_ = {id_ = 11}}, depends_on_ = {seqno_ = 6}, flags_ = 3}, data=@0x7ffa8009cef0: {ptr_ = 0x7ffa7b600100, size_ = 167}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/server_state.cpp:1136 #25 0x000056452431c2e3 in wsrep::high_priority_service::apply (this=0x7ffa8009dcf8, ws_handle=@0x7ffa8009cee0: {transaction_id_ = {id_ = 35}, opaque_ = 0x7ffa64036b50}, ws_meta=@0x7ffa8009d090: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 8}}, stid_ = {server_id_ = {data_ = {buf = "\240\v\206\316g\225\021\353\223\375~\307\316\264i\250"}}, transaction_id_ = {id_ = 35}, client_id_ = {id_ = 11}}, depends_on_ = {seqno_ = 6}, flags_ = 3}, data=@0x7ffa8009cef0: {ptr_ = 0x7ffa7b600100, size_ = 167}) at /mariadb/10.6-MDEV-515/wsrep-lib/include/wsrep/high_priority_service.hpp:47 #26 (anonymous namespace)::apply_cb (ctx=0x7ffa8009dcf8, wsh=<optimized out>, flags=<optimized out>, buf=<optimized out>, meta=<optimized out>, exit_loop=0x7ffa8009d39f) at /mariadb/10.6-MDEV-515/wsrep-lib/src/wsrep_provider_v26.cpp:502 #27 0x00007ffa801e3111 in ?? () from /usr/lib/galera/libgalera_smm.so #28 0x00007ffa801f2203 in ?? () from /usr/lib/galera/libgalera_smm.so #29 0x00007ffa801f8a55 in ?? () from /usr/lib/galera/libgalera_smm.so #30 0x00007ffa802222a8 in ?? () from /usr/lib/galera/libgalera_smm.so #31 0x00007ffa80222fd7 in ?? () from /usr/lib/galera/libgalera_smm.so #32 0x00007ffa801f93c8 in ?? () from /usr/lib/galera/libgalera_smm.so #33 0x000056452431c948 in wsrep::wsrep_provider_v26::run_applier (this=<optimized out>, applier_ctx=0x80) at /mariadb/10.6-MDEV-515/wsrep-lib/src/wsrep_provider_v26.cpp:743 #34 0x0000564523e5b2e0 in wsrep_replication_process (thd=0x7ffa64000d48, arg=<optimized out>) at /mariadb/10.6-MDEV-515/sql/wsrep_thd.cc:58 #35 0x0000564523e51cdb in start_wsrep_THD (arg=arg@entry=0x5645269ee320) at /mariadb/10.6-MDEV-515/sql/wsrep_mysqld.cc:3089 #36 0x0000564523de01df in pfs_spawn_thread (arg=0x5645269ef228) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #37 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #38 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 6 (Thread 0x7ffa52ffd700 (LWP 2130813)): #0 futex_abstimed_wait_cancelable (private=0, abstime=0x7ffa52ffcd58, clockid=1392495792, expected=0, futex_word=0x564526ac1ddc) at ../sysdeps/nptl/futex-internal.h:323 #1 __pthread_cond_wait_common (abstime=0x7ffa52ffcd58, clockid=1392495792, mutex=0x564526ab23a0, cond=0x564526ac1db0) at pthread_cond_wait.c:520 #2 __pthread_cond_clockwait (abstime=0x7ffa52ffcd58, clockid=1392495792, mutex=0x564526ab23a0, cond=0x564526ac1db0) at pthread_cond_wait.c:677 #3 __pthread_cond_clockwait (cond=0x564526ac1db0, mutex=0x564526ab23a0, clockid=1392495792, abstime=0x7ffa52ffcd58) at pthread_cond_wait.c:665 #4 0x0000564524225102 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1db0, __lock=@0x7ffa52ffcda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:209 #5 std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x564526ac1db0, __lock=@0x7ffa52ffcda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __atime=<optimized out>) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:119 #6 std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x564526ac1db0, __lock=@0x7ffa52ffcda8: {_M_device = 0x564526ab23a0, _M_owns = true}, __rtime=@0x564526ab23c8: {__r = 60000}) at /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/condition_variable:172 #7 tpool::thread_pool_generic::wait_for_tasks (this=this@entry=0x564526ab2290, lk=@0x7ffa52ffcda8: {_M_device = 0x564526ab23a0, _M_owns = true}, thread_data=thread_data@entry=0x564526ac1db0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:446 #8 0x000056452422529e in tpool::thread_pool_generic::get_task (this=this@entry=0x564526ab2290, thread_var=thread_var@entry=0x564526ac1db0, t=t@entry=0x7ffa52ffcdf8) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:499 #9 0x0000564524225594 in tpool::thread_pool_generic::worker_main (this=0x564526ab2290, thread_var=0x564526ac1db0) at /mariadb/10.6-MDEV-515/tpool/tpool_generic.cc:544 #10 0x00007ffa84ab3ed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #11 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #12 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 5 (Thread 0x7ffa6167a700 (LWP 2130968)): #0 futex_wait_cancelable (private=0, expected=0, futex_word=0x564524c4aee4 <COND_slave_background+44>) at ../sysdeps/nptl/futex-internal.h:186 #1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x564524c4deb8 <LOCK_slave_background+40>, cond=0x564524c4aeb8 <COND_slave_background>) at pthread_cond_wait.c:508 #2 __pthread_cond_wait (cond=cond@entry=0x564524c4aeb8 <COND_slave_background>, mutex=mutex@entry=0x564524c4deb8 <LOCK_slave_background+40>) at pthread_cond_wait.c:638 #3 0x000056452429938c in safe_cond_wait (cond=0x564524c4aee4 <COND_slave_background+44>, mp=0x564524c4de90 <LOCK_slave_background>, file=0x564524329b68 "/mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h", line=1222) at /mariadb/10.6-MDEV-515/mysys/thr_mutex.c:492 #4 0x0000564523815ceb in inline_mysql_cond_wait (that=0x564524c4aeb8 <COND_slave_background>, mutex=<optimized out>, src_file=<optimized out>, src_line=524) at /mariadb/10.6-MDEV-515/include/mysql/psi/mysql_thread.h:1222 #5 handle_slave_background (arg=<optimized out>) at /mariadb/10.6-MDEV-515/sql/slave.cc:524 #6 0x0000564523de01df in pfs_spawn_thread (arg=0x564526925428) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #7 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #8 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 4 (Thread 0x7ffa79079700 (LWP 2129986)): #0 0x00007ffa84902116 in epoll_wait (epfd=9, events=0x7ffa79078410, maxevents=128, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30 #1 0x00007ffa8023291f in ?? () from /usr/lib/galera/libgalera_smm.so #2 0x00007ffa80233704 in ?? () from /usr/lib/galera/libgalera_smm.so #3 0x00007ffa8027c8a1 in ?? () from /usr/lib/galera/libgalera_smm.so #4 0x00007ffa8026a3cd in ?? () from /usr/lib/galera/libgalera_smm.so #5 0x00007ffa80270709 in ?? () from /usr/lib/galera/libgalera_smm.so #6 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #7 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 3 (Thread 0x7ffa7b4fd700 (LWP 2129933)): #0 futex_wait_cancelable (private=0, expected=0, futex_word=0x56452699a8d4) at ../sysdeps/nptl/futex-internal.h:186 #1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x56452699a870, cond=0x56452699a8a8) at pthread_cond_wait.c:508 #2 __pthread_cond_wait (cond=0x56452699a8a8, mutex=0x56452699a870) at pthread_cond_wait.c:638 #3 0x00007ffa8021db2e in ?? () from /usr/lib/galera/libgalera_smm.so #4 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #5 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95   Thread 2 (Thread 0x7ffa845d1800 (LWP 2129584)): #0 0x00007ffa848f73ff in __GI___poll (fds=0x7ffffd383a10, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29 #1 0x00005645237fd885 in handle_connections_sockets () at /mariadb/10.6-MDEV-515/sql/mysqld.cc:6003 #2 0x00005645237f8d12 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at /mariadb/10.6-MDEV-515/sql/mysqld.cc:5701 #3 0x00007ffa8482ad0a in __libc_start_main (main=0x5645237f4780 <main(int, char**)>, argc=23, argv=0x7ffffd383e28, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffffd383e18) at ../csu/libc-start.c:308 #4 0x00005645237f46ba in _start ()   Thread 1 (Thread 0x7ffa61599700 (LWP 2146017)): #0 __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56 #1 0x0000564523b45a77 in handle_fatal_signal (sig=6) at /mariadb/10.6-MDEV-515/sql/signal_handler.cc:330 #2 <signal handler called> #3 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #4 0x00007ffa84829537 in __GI_abort () at abort.c:79 #5 0x00005645240a8772 in ut_dbg_assertion_failed (expr=expr@entry=0x0, file=<optimized out>, line=<optimized out>, line@entry=501) at /mariadb/10.6-MDEV-515/storage/innobase/ut/ut0dbg.cc:60 #6 0x0000564523f0ff4c in wsrep_assert_no_bf_bf_wait (lock_rec1=lock_rec1@entry=0x0, lock_rec2=lock_rec2@entry=0x7ffa628c85f0, trx1=trx1@entry=0x7ffa628cc648) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:501 #7 0x0000564523f029a0 in lock_rec_has_to_wait (for_locking=<optimized out>, trx=trx@entry=0x7ffa628cc648, type_mode=type_mode@entry=1027, lock2=lock2@entry=0x7ffa628c85f0, lock_is_on_supremum=false) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:646 #8 0x0000564523f13c41 in lock_rec_other_has_conflicting (mode=mode@entry=1027, block=<optimized out>, block@entry=0x7ffa618d7640, heap_no=heap_no@entry=3, trx=0x2, trx@entry=0x7ffa628cc648) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:957 #9 0x0000564523f16685 in lock_rec_lock (impl=false, mode=1027, mode@entry=1636660800, block=block@entry=0x7ffa618d7640, heap_no=heap_no@entry=3, index=index@entry=0x7ffa04023dd8, thr=0x7ffa04945de8) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:1577 #10 0x0000564523f179bf in lock_clust_rec_read_check_and_lock (flags=<optimized out>, flags@entry=0, block=<optimized out>, block@entry=0x7ffa618d7640, rec=0x7ffa61ddc099 "\200", rec@entry=0x3c210fbf08d51700 <error: Cannot access memory at address 0x3c210fbf08d51700>, index=index@entry=0x7ffa04023dd8, offsets=<optimized out>, offsets@entry=0x0, mode=LOCK_X, mode@entry=32762, gap_mode=1024, thr=0x7ffa04945de8) at /mariadb/10.6-MDEV-515/storage/innobase/lock/lock0lock.cc:5381 #11 0x0000564524035d05 in sel_set_rec_lock (pcur=<optimized out>, pcur@entry=0x7ffa61ddc099, rec=<optimized out>, rec@entry=0x7ffa61596668 "\001", index=index@entry=0x7ffa04023dd8, offsets=<optimized out>, mode=<optimized out>, type=type@entry=1024, thr=0x7ffa04945de8, mtr=0x7ffa61596668) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0sel.cc:1256 #12 0x0000564524031324 in row_search_mvcc (buf=<optimized out>, buf@entry=0x7ffa04027208 "\375\002", mode=<optimized out>, mode@entry=PAGE_CUR_GE, prebuilt=0x7ffa04945438, match_mode=<optimized out>, direction=<optimized out>, direction@entry=0) at /mariadb/10.6-MDEV-515/storage/innobase/row/row0sel.cc:5122 #13 0x0000564523e7e7f5 in ha_innobase::index_read (this=this@entry=0x7ffa04944bf0, buf=0x2 <error: Cannot access memory at address 0x2>, buf@entry=0x7ffa04027208 "\375\002", key_ptr=key_ptr@entry=0x7ffa04027518 "\002", key_len=4, find_flag=find_flag@entry=HA_READ_KEY_EXACT) at /mariadb/10.6-MDEV-515/storage/innobase/handler/ha_innodb.cc:8578 #14 0x0000564523e7efd6 in ha_innobase::rnd_pos (this=0x7ffa04944bf0, buf=0x7ffa04027208 "\375\002", pos=0x7ffa04027518 "\002") at /mariadb/10.6-MDEV-515/storage/innobase/handler/ha_innodb.cc:9067 #15 0x0000564523b4cc31 in handler::ha_rnd_pos (this=this@entry=0x7ffa04944bf0, buf=<optimized out>, buf@entry=0x7ffa04027208 "\375\002", pos=0x7ffa04027518 "\002") at /mariadb/10.6-MDEV-515/sql/handler.cc:3098 #16 0x0000564523b5b9e7 in handler::rnd_pos_by_record (this=0x7ffa04944bf0, record=0x7ffa04027208 "\375\002") at /mariadb/10.6-MDEV-515/sql/handler.h:3930 #17 0x0000564523c86bb0 in handler::ha_rnd_pos_by_record (this=0x7ffa04944bf0, buf=0x7ffa615954a0 "\ap\b") at /mariadb/10.6-MDEV-515/sql/sql_class.h:7063 #18 Rows_log_event::find_row (this=this@entry=0x7ffa0c01f008, rgi=rgi@entry=0x7ffa0c047840) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:7823 #19 0x0000564523c8828b in Update_rows_log_event::do_exec_row (this=0x7ffa0c01f008, rgi=0x7ffa0c047840) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:8274 #20 0x0000564523c808d3 in Rows_log_event::do_apply_event (this=0x7ffa0c01f008, rgi=<optimized out>) at /mariadb/10.6-MDEV-515/sql/log_event_server.cc:5673 #21 0x000056452382ce44 in Log_event::apply_event (this=this@entry=0x7ffa0c01f008, rgi=0x7ffa0c047840) at /mariadb/10.6-MDEV-515/sql/log_event.h:1498 #22 0x0000564523e5ae5b in wsrep_apply_events (thd=thd@entry=0x7ffa0c030b68, rli=<optimized out>, events_buf=<optimized out>, buf_len=<optimized out>) at /mariadb/10.6-MDEV-515/sql/wsrep_applier.cc:208 #23 0x0000564523e45335 in apply_events (thd=0x7ffa0c030b68, rli=0x7ffa615954a0, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}, err=@0x7ffa61597230: {buffer_ = std::vector of length 0, capacity 0}) at /mariadb/10.6-MDEV-515/sql/wsrep_high_priority_service.cc:127 #24 Wsrep_replayer_service::apply_write_set (this=0x7ffa61597f10, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}, err=@0x7ffa61597230: {buffer_ = std::vector of length 0, capacity 0}) at /mariadb/10.6-MDEV-515/sql/wsrep_high_priority_service.cc:709 #25 0x00005645243111dd in apply_write_set (server_state=<optimized out>, high_priority_service=@0x7ffa61597f10: {_vptr$high_priority_service = 0x564524b8f438 <vtable for Wsrep_replayer_service+16>, server_state_ = @0x564526956390, must_exit_ = false}, ws_handle=@0x7ffa61597400: {transaction_id_ = {id_ = 46}, opaque_ = 0x7ffa0c01f400}, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/server_state.cpp:327 #26 wsrep::server_state::on_apply (this=<optimized out>, high_priority_service=@0x7ffa61597f10: {_vptr$high_priority_service = 0x564524b8f438 <vtable for Wsrep_replayer_service+16>, server_state_ = @0x564526956390, must_exit_ = false}, ws_handle=@0x7ffa61597400: {transaction_id_ = {id_ = 46}, opaque_ = 0x7ffa0c01f400}, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/server_state.cpp:1136 #27 0x000056452431c2e3 in wsrep::high_priority_service::apply (this=0x7ffa61597f10, ws_handle=@0x7ffa61597400: {transaction_id_ = {id_ = 46}, opaque_ = 0x7ffa0c01f400}, ws_meta=@0x7ffa615975b0: {gtid_ = {id_ = {data_ = {buf = "\237\237G\326g\225\021\353\241o\337\235\t\233\000\261"}}, seqno_ = {seqno_ = 9}}, stid_ = {server_id_ = {data_ = {buf = "\237\237\025\357g\225\021\353\277\370\336/,\260\037\337"}}, transaction_id_ = {id_ = 46}, client_id_ = {id_ = 12}}, depends_on_ = {seqno_ = 8}, flags_ = 3}, data=@0x7ffa61597410: {ptr_ = 0x7ffa7b600260, size_ = 203}) at /mariadb/10.6-MDEV-515/wsrep-lib/include/wsrep/high_priority_service.hpp:47 #28 (anonymous namespace)::apply_cb (ctx=0x7ffa61597f10, wsh=<optimized out>, flags=<optimized out>, buf=<optimized out>, meta=<optimized out>, exit_loop=0x7ffa615978df) at /mariadb/10.6-MDEV-515/wsrep-lib/src/wsrep_provider_v26.cpp:502 #29 0x00007ffa801e3111 in ?? () from /usr/lib/galera/libgalera_smm.so #30 0x00007ffa801f7e03 in ?? () from /usr/lib/galera/libgalera_smm.so #31 0x00007ffa801d580b in ?? () from /usr/lib/galera/libgalera_smm.so #32 0x000056452431d1d1 in wsrep::wsrep_provider_v26::replay (this=<optimized out>, ws_handle=<optimized out>, reply_service=0x0) at /mariadb/10.6-MDEV-515/wsrep-lib/src/wsrep_provider_v26.cpp:870 #33 0x0000564523e426e5 in Wsrep_client_service::replay (this=0x7ffa0c0071b0) at /mariadb/10.6-MDEV-515/sql/wsrep_client_service.cc:280 #34 0x0000564524317d90 in wsrep::transaction::replay (this=this@entry=0x7ffa0c007230, lock=@0x7ffa615983b0: {_M_device = 0x7ffa0c007190, _M_owns = false}) at /mariadb/10.6-MDEV-515/wsrep-lib/src/transaction.cpp:1889 #35 0x000056452431768f in wsrep::transaction::after_statement (this=0x7ffa0c007230) at /mariadb/10.6-MDEV-515/wsrep-lib/src/transaction.cpp:856 #36 0x000056452430943b in wsrep::client_state::after_statement (this=0x7ffa0c0071c8) at /mariadb/10.6-MDEV-515/wsrep-lib/src/client_state.cpp:276 #37 0x00005645238e5db3 in wsrep_after_statement (thd=0x7ffa0c000d48) at /mariadb/10.6-MDEV-515/sql/wsrep_trans_observer.h:426 #38 wsrep_mysql_parse (thd=thd@entry=0x7ffa0c000d48, rawbuf=0x7ffa0c0181c0 "commit", length=6, parser_state=parser_state@entry=0x7ffa61598620) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:7750 #39 0x00005645238e35ee in dispatch_command (command=<optimized out>, command@entry=COM_QUERY, thd=thd@entry=0x7ffa0c000d48, packet=packet@entry=0x7ffa0c010149 "commit", packet_length=packet_length@entry=6) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1820 #40 0x00005645238e4d77 in do_command (thd=0x7ffa0c000d48) at /mariadb/10.6-MDEV-515/sql/sql_parse.cc:1365 #41 0x0000564523a0d7e9 in do_handle_one_connection (connect=connect@entry=0x564526a9e3f8, put_in_cache=<optimized out>) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1410 #42 0x0000564523a0d65c in handle_one_connection (arg=arg@entry=0x564526a9e3f8) at /mariadb/10.6-MDEV-515/sql/sql_connect.cc:1312 #43 0x0000564523de01df in pfs_spawn_thread (arg=0x564526f72538) at /mariadb/10.6-MDEV-515/storage/perfschema/pfs.cc:2201 #44 0x00007ffa852d4ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 #45 0x00007ffa84901def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Marko Mäkelä added a comment - 2021-02-05 10:31 A similar stack trace occurred with bb-10.6- MDEV-24789 24c082136bc00a5019f7fbe61e14dc4fcf110bfb. The test does often pass, and if I run it alone, it will always pass. Also, it is worth noting that the test refuses to run concurrently with itself: ./mtr --parallel=auto --repeat=10 galera.galera_FK_duplicate_client_insert{,,,,,} would create only one worker. So, I suppose that the only way to repeat this failure is to run something like nice ./mtr --parallel=60 --force --big-test --suite=galera,galera_sr,galera_3nodes which has been my invocation on a system where nproc says 40. My build and tests are located in /dev/shm .