Checking supported features... MariaDB Version 10.4.3-MariaDB-debug - SSL connections supported - binaries are debug compiled Collecting tests... Installing system database... worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- galera.galera_bf_lock_wait 'innodb' [ pass ] 107151 galera.galera_bf_lock_wait 'innodb' [ 2 fail ] Test ended at 2019-02-27 12:18:03 CURRENT_TEST: galera.galera_bf_lock_wait mysqltest: At line 50: query 'drop table t1' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > WHILE 1 DO start transaction; update t1 set b=connection_id() where a=1; commit; END WHILE; END| connect node_1_p1, 127.0.0.1, root, , test, $NODE_MYPORT_1; call p1; connect node_1_p2, 127.0.0.1, root, , test, $NODE_MYPORT_1; call p1; connect node_2_p1, 127.0.0.1, root, , test, $NODE_MYPORT_2; call p1; connect node_2_p2, 127.0.0.1, root, , test, $NODE_MYPORT_2; call p1; connection default; checking error log for 'BF lock wait long' message for 10 times every 10 seconds ... drop table t1; More results from queries before failure can be found in /home/stepan/galera/git/10.4/server/mysql-test/var/log/galera_bf_lock_wait.log Server [mysqld.1 - pid: 48608, winpid: 48608, exit: 256] failed during test run Server log from this test: ----------SERVER LOG START----------- mysqld: /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:422: void wsrep_commit_empty(THD*, bool): Assertion `have_error || !wsrep_current_error(thd)' failed. 190227 12:17:54 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.4.3-MariaDB-debug-log key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=6 max_threads=153 thread_count=15 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63481 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f852c000b00 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f85a804bd30 thread_stack 0x49000 /home/stepan/galera/git/10.4/server/sql/mysqld(my_print_stacktrace+0x40)[0x559dbb4abc56] /home/stepan/galera/git/10.4/server/sql/mysqld(handle_fatal_signal+0x3e2)[0x559dbace4625] sigaction.c:0(__restore_rt)[0x7f85b8d315d0] :0(__GI_raise)[0x7f85b7020207] :0(__GI_abort)[0x7f85b70218f8] :0(__assert_fail_base)[0x7f85b7019026] :0(__GI___assert_fail)[0x7f85b70190d2] sql/wsrep_trans_observer.h:423(wsrep_commit_empty(THD*, bool))[0x559dbace69cf] sql/handler.cc:1681(ha_commit_trans(THD*, bool))[0x559dbacea346] sql/transaction.cc:313(trans_commit(THD*))[0x559dbab28fa8] sql/sql_parse.cc:5849(mysql_execute_command(THD*))[0x559dba9ac6ae] sql/sp_head.cc:3591(sp_instr_stmt::exec_core(THD*, unsigned int*))[0x559dba8c1180] sql/sp_head.cc:3319(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*))[0x559dba8c046e] sql/sp_head.cc:3497(sp_instr_stmt::execute(THD*, unsigned int*))[0x559dba8c0ccc] sql/sp_head.cc:1341(sp_head::execute(THD*, bool))[0x559dba8ba7cb] sql/sp_head.cc:2283(sp_head::execute_procedure(THD*, List*))[0x559dba8bd1f2] sql/sql_parse.cc:2959(do_execute_sp(THD*, sp_head*))[0x559dba9a2018] sql/sql_parse.cc:3201(Sql_cmd_call::execute(THD*))[0x559dba9a2c04] sql/sql_parse.cc:6330(mysql_execute_command(THD*))[0x559dba9ae55b] sql/sql_parse.cc:8141(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x559dba9b3777] sql/sql_parse.cc:7964(wsrep_mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x559dba9b2da6] sql/sql_parse.cc:1804(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x559dba99eb44] sql/sql_parse.cc:1358(do_command(THD*))[0x559dba99d491] sql/sql_connect.cc:1399(do_handle_one_connection(CONNECT*))[0x559dbab14180] sql/sql_connect.cc:1303(handle_one_connection)[0x559dbab13ec4] perfschema/pfs.cc:1864(pfs_spawn_thread)[0x559dbb443bd1] pthread_create.c:0(start_thread)[0x7f85b8d29dd5] /lib64/libc.so.6(clone+0x6d)[0x7f85b70e7ead] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x7f852c02f508): commit Connection ID (thread ID): 29 Status: KILL_QUERY Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file at /home/stepan/galera/git/10.4/server/mysql-test/var/mysqld.1/data/ ----------SERVER LOG END------------- - found 'core.48609' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace from coredump /home/stepan/galera/git/10.4/server/mysql-test/var/log/galera.galera_bf_lock_wait-innodb/mysqld.1/data/core.48609 Core generated by '/home/stepan/galera/git/10.4/server/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 48910] [New LWP 48616] [New LWP 48617] [New LWP 48621] [New LWP 48619] [New LWP 48622] [New LWP 48629] [New LWP 48612] [New LWP 48627] [New LWP 48633] [New LWP 48618] [New LWP 48628] [New LWP 48637] [New LWP 48620] [New LWP 48636] [New LWP 48639] [New LWP 48623] [New LWP 48862] [New LWP 48614] [New LWP 48613] [New LWP 48626] [New LWP 48860] [New LWP 48630] [New LWP 48615] [New LWP 48610] [New LWP 48632] [New LWP 48861] [New LWP 48638] [New LWP 48635] [New LWP 48640] [New LWP 48634] [New LWP 48643] [New LWP 48624] [New LWP 48644] [New LWP 48909] [New LWP 48611] [New LWP 48609] [New LWP 48631] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/home/stepan/galera/git/10.4/server/sql/mysqld --defaults-group-suffix=.1 --def'. Program terminated with signal 6, Aborted. #0 0x00007f85b8d2e9d1 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007f85b8d2e9d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000559dbb4abd44 in my_write_core (sig=6) at /home/stepan/galera/git/10.4/server/mysys/stacktrace.c:481 #2 0x0000559dbace4a51 in handle_fatal_signal (sig=6) at /home/stepan/galera/git/10.4/server/sql/signal_handler.cc:305 #3 #4 0x00007f85b7020207 in raise () from /lib64/libc.so.6 #5 0x00007f85b70218f8 in abort () from /lib64/libc.so.6 #6 0x00007f85b7019026 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f85b70190d2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000559dbace69cf in wsrep_commit_empty (thd=0x7f852c000b00, all=true) at /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:422 #9 0x0000559dbacea346 in ha_commit_trans (thd=0x7f852c000b00, all=true) at /home/stepan/galera/git/10.4/server/sql/handler.cc:1677 #10 0x0000559dbab28fa8 in trans_commit (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/transaction.cc:313 #11 0x0000559dba9ac6ae in mysql_execute_command (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:5849 #12 0x0000559dba8c1180 in sp_instr_stmt::exec_core (this=0x7f852c020ac0, thd=0x7f852c000b00, nextp=0x7f85a80490e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3591 #13 0x0000559dba8c046e in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f852c020b08, thd=0x7f852c000b00, nextp=0x7f85a80490e4, open_tables=false, instr=0x7f852c020ac0) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3319 #14 0x0000559dba8c0ccc in sp_instr_stmt::execute (this=0x7f852c020ac0, thd=0x7f852c000b00, nextp=0x7f85a80490e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3497 #15 0x0000559dba8ba7cb in sp_head::execute (this=0x7f852c01b5e8, thd=0x7f852c000b00, merge_da_on_success=true) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:1341 #16 0x0000559dba8bd1f2 in sp_head::execute_procedure (this=0x7f852c01b5e8, thd=0x7f852c000b00, args=0x7f852c005818) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:2283 #17 0x0000559dba9a2018 in do_execute_sp (thd=0x7f852c000b00, sp=0x7f852c01b5e8) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:2959 #18 0x0000559dba9a2c04 in Sql_cmd_call::execute (this=0x7f852c0140b0, thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:3201 #19 0x0000559dba9ae55b in mysql_execute_command (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:6330 #20 0x0000559dba9b3777 in mysql_parse (thd=0x7f852c000b00, rawbuf=0x7f852c014008 "call p1", length=7, parser_state=0x7f85a804b0b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:8141 #21 0x0000559dba9b2da6 in wsrep_mysql_parse (thd=0x7f852c000b00, rawbuf=0x7f852c014008 "call p1", length=7, parser_state=0x7f85a804b0b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7953 #22 0x0000559dba99eb44 in dispatch_command (command=COM_QUERY, thd=0x7f852c000b00, packet=0x7f852c00a5e1 "call p1", packet_length=7, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1804 #23 0x0000559dba99d491 in do_command (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1358 #24 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #25 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #26 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe8864f0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #27 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #28 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 38 (Thread 0x7f857f7fe700 (LWP 48631)): #0 0x00007f85b8d30d1d in fsync () from /lib64/libpthread.so.0 #1 0x0000559dbafe7d49 in os_file_fsync_posix (file=19) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2458 #2 0x0000559dbafe809f in os_file_flush_func (file=19) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2559 #3 0x0000559dbb230860 in pfs_os_file_flush_func (file=..., src_file=0x559dbb8af490 "/home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc", src_line=827) at /home/stepan/galera/git/10.4/server/storage/innobase/include/os0file.ic:406 #4 0x0000559dbb2335db in fil_flush_low (space=0x559dbe829750, metadata=false) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:827 #5 0x0000559dbb23eeb5 in fil_flush (space_id=4294967280) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4449 #6 0x0000559dbafc3417 in log_write_flush_to_disk_low () at /home/stepan/galera/git/10.4/server/storage/innobase/log/log0log.cc:873 #7 0x0000559dbafc3e79 in log_write_up_to (lsn=883159, flush_to_disk=true, rotate_key=true) at /home/stepan/galera/git/10.4/server/storage/innobase/log/log0log.cc:1111 #8 0x0000559dbafc520c in log_checkpoint (sync=true, write_always=false) at /home/stepan/galera/git/10.4/server/storage/innobase/log/log0log.cc:1494 #9 0x0000559dbb0e2708 in srv_master_do_idle_tasks () at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2256 #10 0x0000559dbb0e2ae6 in srv_master_thread (arg=0x0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2358 #11 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #12 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 37 (Thread 0x7f85b9138880 (LWP 48609)): #0 0x00007f85b70dd20d in poll () from /lib64/libc.so.6 #1 0x0000559dba87742d in handle_connections_sockets () at /home/stepan/galera/git/10.4/server/sql/mysqld.cc:6414 #2 0x0000559dba8766c8 in mysqld_main (argc=161, argv=0x559dbe361140) at /home/stepan/galera/git/10.4/server/sql/mysqld.cc:5850 #3 0x0000559dba86a875 in main (argc=23, argv=0x7ffd60b03b58) at /home/stepan/galera/git/10.4/server/sql/main.cc:25 Thread 36 (Thread 0x7f85ab5fe700 (LWP 48611)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f85b0c376e7 in gu::Lock::wait (cond=..., this=0x7f85ab5fdbf0) at galerautils/src/gu_lock.hpp:45 #2 0x00007f85b0c37779 in galera::ServiceThd::thd_func (arg=0x559dbe4dfab8) at galera/src/galera_service_thd.cpp:30 #3 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 35 (Thread 0x7f85a8097700 (LWP 48909)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f85b0c4f787 in gu::Lock::wait (cond=..., this=0x7f85a80932c0) at galerautils/src/gu_lock.hpp:45 #2 0x00007f85b0c603dc in galera::Monitor::enter (this=this@entry=0x559dbe4e0400, obj=...) at galera/src/monitor.hpp:156 #3 0x00007f85b0c54e87 in galera::ReplicatorSMM::commit_order_enter_local (this=this@entry=0x559dbe4df3a0, trx=...) at galera/src/replicator_smm.cpp:1280 #4 0x00007f85b0c73833 in galera_commit_order_enter (gh=, ws_handle=, meta=) at galera/src/wsrep_provider.cpp:658 #5 0x0000559dbb5665d9 in wsrep::wsrep_provider_v26::commit_order_enter (this=0x559dbe466750, ws_handle=..., ws_meta=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/wsrep_provider_v26.cpp:769 #6 0x0000559dbb55a8f1 in wsrep::transaction::after_statement (this=0x7f8528006a78) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/transaction.cpp:755 #7 0x0000559dbb54abcb in wsrep::client_state::after_statement (this=0x7f8528006a08) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/client_state.cpp:241 #8 0x0000559dba8b7122 in wsrep_after_statement (thd=0x7f8528000b00) at /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:317 #9 0x0000559dba8c124e in sp_instr_stmt::exec_core (this=0x7f8528020ac0, thd=0x7f8528000b00, nextp=0x7f85a80940e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3607 #10 0x0000559dba8c046e in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f8528020b08, thd=0x7f8528000b00, nextp=0x7f85a80940e4, open_tables=false, instr=0x7f8528020ac0) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3319 #11 0x0000559dba8c0ccc in sp_instr_stmt::execute (this=0x7f8528020ac0, thd=0x7f8528000b00, nextp=0x7f85a80940e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3497 #12 0x0000559dba8ba7cb in sp_head::execute (this=0x7f852801b5e8, thd=0x7f8528000b00, merge_da_on_success=true) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:1341 #13 0x0000559dba8bd1f2 in sp_head::execute_procedure (this=0x7f852801b5e8, thd=0x7f8528000b00, args=0x7f8528005818) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:2283 #14 0x0000559dba9a2018 in do_execute_sp (thd=0x7f8528000b00, sp=0x7f852801b5e8) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:2959 #15 0x0000559dba9a2c04 in Sql_cmd_call::execute (this=0x7f85280140b0, thd=0x7f8528000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:3201 #16 0x0000559dba9ae55b in mysql_execute_command (thd=0x7f8528000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:6330 #17 0x0000559dba9b3777 in mysql_parse (thd=0x7f8528000b00, rawbuf=0x7f8528014008 "call p1", length=7, parser_state=0x7f85a80960b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:8141 #18 0x0000559dba9b2da6 in wsrep_mysql_parse (thd=0x7f8528000b00, rawbuf=0x7f8528014008 "call p1", length=7, parser_state=0x7f85a80960b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7953 #19 0x0000559dba99eb44 in dispatch_command (command=COM_QUERY, thd=0x7f8528000b00, packet=0x7f852800a5e1 "call p1", packet_length=7, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1804 #20 0x0000559dba99d491 in do_command (thd=0x7f8528000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1358 #21 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #22 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #23 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe8cec00) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #24 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #25 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 34 (Thread 0x7f85b0079700 (LWP 48644)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1373 in safe_cond_wait (cond=0x7f8540000cd0, mp=0x7f8540000c20, file=0x559dbb5e5460 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1167) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:492 #2 0x0000559dbab1c914 in inline_mysql_cond_wait (that=0x7f8540000cd0, mutex=0x7f8540000c20, src_file=0x559dbb5e5848 "/home/stepan/galera/git/10.4/server/sql/mdl.cc", src_line=1143) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1167 #3 0x0000559dbab1dd9e in MDL_wait::timed_wait (this=0x7f8540000c20, owner=0x7f8540000bd0, abs_timeout=0x7f85b0076bb0, set_status_on_timeout=false, wait_state_name=0x559dbc020050 ) at /home/stepan/galera/git/10.4/server/sql/mdl.cc:1143 #4 0x0000559dbab1faa3 in MDL_context::acquire_lock (this=0x7f8540000c20, mdl_request=0x7f8540015810, lock_wait_timeout=86400) at /home/stepan/galera/git/10.4/server/sql/mdl.cc:2326 #5 0x0000559dbab1ff59 in MDL_context::acquire_locks (this=0x7f8540000c20, mdl_requests=0x7f85b0076cf0, lock_wait_timeout=86400) at /home/stepan/galera/git/10.4/server/sql/mdl.cc:2447 #6 0x0000559dba90c1a4 in lock_table_names (thd=0x7f8540000b00, options=..., tables_start=0x7f85400153d0, tables_end=0x0, lock_wait_timeout=86400, flags=0) at /home/stepan/galera/git/10.4/server/sql/sql_base.cc:4019 #7 0x0000559dbaa72f5d in lock_table_names (thd=0x7f8540000b00, table_list=0x7f85400153d0, table_list_end=0x0, lock_wait_timeout=86400, flags=0) at /home/stepan/galera/git/10.4/server/sql/sql_base.h:247 #8 0x0000559dbaa78197 in mysql_rm_table (thd=0x7f8540000b00, tables=0x7f85400153d0, if_exists=false, drop_temporary=false, drop_sequence=false) at /home/stepan/galera/git/10.4/server/sql/sql_table.cc:2064 #9 0x0000559dba9a915c in mysql_execute_command (thd=0x7f8540000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:5095 #10 0x0000559dba9b3777 in mysql_parse (thd=0x7f8540000b00, rawbuf=0x7f8540015318 "drop table t1", length=13, parser_state=0x7f85b00780b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:8141 #11 0x0000559dba9b2da6 in wsrep_mysql_parse (thd=0x7f8540000b00, rawbuf=0x7f8540015318 "drop table t1", length=13, parser_state=0x7f85b00780b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7953 #12 0x0000559dba99eb44 in dispatch_command (command=COM_QUERY, thd=0x7f8540000b00, packet=0x7f85400283b1 "drop table t1", packet_length=13, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1804 #13 0x0000559dba99d491 in do_command (thd=0x7f8540000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1358 #14 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe91f680) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #15 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe91f680) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #16 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe9b5290) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #17 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #18 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 33 (Thread 0x7f8586ffd700 (LWP 48624)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff2c98 in os_event::timed_wait (this=0x559dbe61fa60, abstime=0x7f8586ffcaf0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:276 #2 0x0000559dbaff2faf in os_event::wait_time_low (this=0x559dbe61fa60, time_in_usec=1000000, reset_sig_count=2) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:398 #3 0x0000559dbaff314f in os_event_wait_time_low (event=0x559dbe61fa60, time_in_usec=1000000, reset_sig_count=2) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:498 #4 0x0000559dbb1c92ab in pc_sleep_if_needed (next_loop_time=1551262677102, sig_count=2, cur_time=1551262676102) at /home/stepan/galera/git/10.4/server/storage/innobase/buf/buf0flu.cc:2593 #5 0x0000559dbb1ca1e4 in buf_flush_page_cleaner_coordinator () at /home/stepan/galera/git/10.4/server/storage/innobase/buf/buf0flu.cc:3052 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 32 (Thread 0x7f85b00c4700 (LWP 48643)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1373 in safe_cond_wait (cond=0x559dbc1fa3e0 , mp=0x559dbc1fd7a0 , file=0x559dbb57cae8 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1167) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:492 #2 0x0000559dba89bed6 in inline_mysql_cond_wait (that=0x559dbc1fa3e0 , mutex=0x559dbc1fd7a0 , src_file=0x559dbb57d650 "/home/stepan/galera/git/10.4/server/sql/slave.cc", src_line=519) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1167 #3 0x0000559dba89d768 in handle_slave_background (arg=0x0) at /home/stepan/galera/git/10.4/server/sql/slave.cc:519 #4 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe5e36c0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 31 (Thread 0x7f857dffb700 (LWP 48634)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff337d in os_event::wait (this=0x559dbe61f760) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:159 #2 0x0000559dbaff2e03 in os_event::wait_low (this=0x559dbe61f760, reset_sig_count=5084) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:326 #3 0x0000559dbaff3174 in os_event_wait_low (event=0x559dbe61f760, reset_sig_count=0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:515 #4 0x0000559dbb0df381 in srv_resume_thread (slot=0x559dbc037980 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:901 #5 0x0000559dbb0e2f85 in srv_worker_thread (arg=0x0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2483 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 30 (Thread 0x7f85b010f700 (LWP 48640)): #0 0x00007f85b702140a in sigwaitinfo () from /lib64/libc.so.6 #1 0x0000559dba86a8f4 in my_sigwait (set=0x7f85b010ecf0, sig=0x7f85b010ecb4, code=0x7f85b010ecb8) at /home/stepan/galera/git/10.4/server/include/my_pthread.h:196 #2 0x0000559dba870f34 in signal_hand (arg=0x0) at /home/stepan/galera/git/10.4/server/sql/mysqld.cc:3230 #3 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe5e31d0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #4 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7f857d7fa700 (LWP 48635)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff337d in os_event::wait (this=0x559dbe61f860) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:159 #2 0x0000559dbaff2e03 in os_event::wait_low (this=0x559dbe61f860, reset_sig_count=4095) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:326 #3 0x0000559dbaff3174 in os_event_wait_low (event=0x559dbe61f860, reset_sig_count=0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:515 #4 0x0000559dbb0df381 in srv_resume_thread (slot=0x559dbc0379f0 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:901 #5 0x0000559dbb0e2f85 in srv_worker_thread (arg=0x0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2483 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7f856b7fe700 (LWP 48638)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff337d in os_event::wait (this=0x559dbe61fae0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:159 #2 0x0000559dbaff2e03 in os_event::wait_low (this=0x559dbe61fae0, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:326 #3 0x0000559dbaff3174 in os_event_wait_low (event=0x559dbe61fae0, reset_sig_count=0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:515 #4 0x0000559dbb1a80b4 in buf_resize_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/buf/buf0buf.cc:3097 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7f85a812d700 (LWP 48861)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f85b0c4f787 in gu::Lock::wait (cond=..., this=0x7f85a81292c0) at galerautils/src/gu_lock.hpp:45 #2 0x00007f85b0c603dc in galera::Monitor::enter (this=this@entry=0x559dbe4e0400, obj=...) at galera/src/monitor.hpp:156 #3 0x00007f85b0c54e87 in galera::ReplicatorSMM::commit_order_enter_local (this=this@entry=0x559dbe4df3a0, trx=...) at galera/src/replicator_smm.cpp:1280 #4 0x00007f85b0c73833 in galera_commit_order_enter (gh=, ws_handle=, meta=) at galera/src/wsrep_provider.cpp:658 #5 0x0000559dbb5665d9 in wsrep::wsrep_provider_v26::commit_order_enter (this=0x559dbe466750, ws_handle=..., ws_meta=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/wsrep_provider_v26.cpp:769 #6 0x0000559dbb55a8f1 in wsrep::transaction::after_statement (this=0x7f853c006a78) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/transaction.cpp:755 #7 0x0000559dbb54abcb in wsrep::client_state::after_statement (this=0x7f853c006a08) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/client_state.cpp:241 #8 0x0000559dba8b7122 in wsrep_after_statement (thd=0x7f853c000b00) at /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:317 #9 0x0000559dba8c124e in sp_instr_stmt::exec_core (this=0x7f853c02b050, thd=0x7f853c000b00, nextp=0x7f85a812a0e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3607 #10 0x0000559dba8c046e in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f853c02b098, thd=0x7f853c000b00, nextp=0x7f85a812a0e4, open_tables=false, instr=0x7f853c02b050) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3319 #11 0x0000559dba8c0ccc in sp_instr_stmt::execute (this=0x7f853c02b050, thd=0x7f853c000b00, nextp=0x7f85a812a0e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3497 #12 0x0000559dba8ba7cb in sp_head::execute (this=0x7f853c025b78, thd=0x7f853c000b00, merge_da_on_success=true) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:1341 #13 0x0000559dba8bd1f2 in sp_head::execute_procedure (this=0x7f853c025b78, thd=0x7f853c000b00, args=0x7f853c005818) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:2283 #14 0x0000559dba9a2018 in do_execute_sp (thd=0x7f853c000b00, sp=0x7f853c025b78) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:2959 #15 0x0000559dba9a2c04 in Sql_cmd_call::execute (this=0x7f853c01fb80, thd=0x7f853c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:3201 #16 0x0000559dba9ae55b in mysql_execute_command (thd=0x7f853c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:6330 #17 0x0000559dba9b3777 in mysql_parse (thd=0x7f853c000b00, rawbuf=0x7f853c01fad8 "call p1", length=7, parser_state=0x7f85a812c0b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:8141 #18 0x0000559dba9b2da6 in wsrep_mysql_parse (thd=0x7f853c000b00, rawbuf=0x7f853c01fad8 "call p1", length=7, parser_state=0x7f85a812c0b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7953 #19 0x0000559dba99eb44 in dispatch_command (command=COM_QUERY, thd=0x7f853c000b00, packet=0x7f853c0179a1 "call p1", packet_length=7, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1804 #20 0x0000559dba99d491 in do_command (thd=0x7f853c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1358 #21 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #22 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #23 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe9466c0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #24 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #25 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7f857effd700 (LWP 48632)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff337d in os_event::wait (this=0x559dbe61f6e0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:159 #2 0x0000559dbaff2e03 in os_event::wait_low (this=0x559dbe61f6e0, reset_sig_count=1269) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:326 #3 0x0000559dbaff3174 in os_event_wait_low (event=0x559dbe61f6e0, reset_sig_count=1269) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:515 #4 0x0000559dbb0df381 in srv_resume_thread (slot=0x559dbc037948 , sig_count=1269, wait=true, timeout_usec=0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:901 #5 0x0000559dbb0e3392 in srv_purge_coordinator_suspend (slot=0x559dbc037948 , rseg_history_len=0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2614 #6 0x0000559dbb0e3628 in srv_purge_coordinator_thread (arg=0x0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2689 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7f85b1782700 (LWP 48610)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1675 in safe_cond_timedwait (cond=0x559dbca99d80 , mp=0x559dbca99cc0 , abstime=0x7f85b1781d70, file=0x559dbb93d420 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:546 #2 0x0000559dbb4b262f in inline_mysql_cond_timedwait (that=0x559dbca99d80 , mutex=0x559dbca99cc0 , abstime=0x7f85b1781d70, src_file=0x559dbb93d468 "/home/stepan/galera/git/10.4/server/mysys/thr_timer.c", src_line=292) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1204 #3 0x0000559dbb4b3192 in timer_handler (arg=0x0) at /home/stepan/galera/git/10.4/server/mysys/thr_timer.c:292 #4 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe4a8bd0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 24 (Thread 0x7f85b0569700 (LWP 48615)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1373 in safe_cond_wait (cond=0x7f8594016ac8, mp=0x7f8594016a18, file=0x559dbb61e5c0 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:492 #2 0x0000559dbac03438 in inline_mysql_cond_wait (that=0x7f8594016ac8, mutex=0x7f8594016a18, src_file=0x559dbb61e8e0 "/home/stepan/galera/git/10.4/server/sql/wsrep_thd.h", src_line=69) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1178 #3 0x0000559dbac060d8 in Wsrep_thd_queue::pop_front (this=0x7f85940169c0) at /home/stepan/galera/git/10.4/server/sql/wsrep_thd.h:69 #4 0x0000559dbac04eb5 in wsrep_post_rollback_process (post_rollbacker=0x7f8594000b00, arg=0x0) at /home/stepan/galera/git/10.4/server/sql/wsrep_thd.cc:285 #5 0x0000559dbabf5576 in start_wsrep_THD (arg=0x559dbe5274f0) at /home/stepan/galera/git/10.4/server/sql/wsrep_mysqld.cc:2707 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f857ffff700 (LWP 48630)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff2c98 in os_event::timed_wait (this=0x559dbe7bf800, abstime=0x7f857fffeb60) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:276 #2 0x0000559dbaff2faf in os_event::wait_time_low (this=0x559dbe7bf800, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:398 #3 0x0000559dbaff314f in os_event_wait_time_low (event=0x559dbe7bf800, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:498 #4 0x0000559dbb141889 in ib_wqueue_timedwait (wq=0x559dbe7bf750, wait_in_usecs=5000000) at /home/stepan/galera/git/10.4/server/storage/innobase/ut/ut0wqueue.cc:163 #5 0x0000559dbb282362 in fts_optimize_thread (arg=0x559dbe7bf750) at /home/stepan/galera/git/10.4/server/storage/innobase/fts/fts0opt.cc:2897 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7f85a8178700 (LWP 48860)): #0 0x00007f85b70dd20d in poll () from /lib64/libc.so.6 #1 0x0000559dbb51cfe8 in vio_io_wait (vio=0x559dbe89b500, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/stepan/galera/git/10.4/server/vio/viosocket.c:976 #2 0x0000559dbb51b608 in vio_socket_io_wait (vio=0x559dbe89b500, event=VIO_IO_EVENT_READ) at /home/stepan/galera/git/10.4/server/vio/viosocket.c:118 #3 0x0000559dbb51b7ee in vio_read (vio=0x559dbe89b500, buf=0x7f853800a5e0 "\a", size=4) at /home/stepan/galera/git/10.4/server/vio/viosocket.c:195 #4 0x0000559dbae4d8f0 in my_real_read (net=0x7f8538000df8, complen=0x7f85a8177810, header=1 '\001') at /home/stepan/galera/git/10.4/server/sql/net_serv.cc:962 #5 0x0000559dbae4e177 in my_net_read_packet_reallen (net=0x7f8538000df8, read_from_server=1 '\001', reallen=0x7f85a8177878) at /home/stepan/galera/git/10.4/server/sql/net_serv.cc:1243 #6 0x0000559dbae4e12a in my_net_read_packet (net=0x7f8538000df8, read_from_server=1 '\001') at /home/stepan/galera/git/10.4/server/sql/net_serv.cc:1227 #7 0x0000559dba99cdc9 in do_command (thd=0x7f8538000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1231 #8 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #9 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #10 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe466b90) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #11 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #12 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f8585df6700 (LWP 48626)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff2c98 in os_event::timed_wait (this=0x559dbe7f7e10, abstime=0x7f8585df5d40) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:276 #2 0x0000559dbaff2faf in os_event::wait_time_low (this=0x559dbe7f7e10, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:398 #3 0x0000559dbaff314f in os_event_wait_time_low (event=0x559dbe7f7e10, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:498 #4 0x0000559dbafbfad6 in lock_wait_timeout_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/lock/lock0wait.cc:543 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f85a8979700 (LWP 48613)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f85b0c11f47 in gu::Lock::wait (cond=..., this=0x7f85a8978850) at galerautils/src/gu_lock.hpp:45 #2 0x00007f85b0c14a8b in RecvBuf::front (this=this@entry=0x559dbe517878, timeout=...) at gcs/src/gcs_gcomm.cpp:124 #3 0x00007f85b0c12626 in gcomm_recv (backend=, msg=0x559dbe4e7970, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:673 #4 0x00007f85b0c065cc in core_msg_recv (backend=backend@entry=0x559dbe4e7ab8, recv_msg=recv_msg@entry=0x559dbe4e7970, timeout=timeout@entry=9223372035999999999) at gcs/src/gcs_core.cpp:480 #5 0x00007f85b0c09844 in gcs_core_recv (conn=0x559dbe4e7910, recv_act=recv_act@entry=0x7f85a8978d70, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1184 #6 0x00007f85b0c10ab8 in gcs_recv_thread (arg=0x559dbe4e7690) at gcs/src/gcs.cpp:1398 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f85b05b4700 (LWP 48614)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1373 in safe_cond_wait (cond=0x7f85a0016a08, mp=0x7f85a0016958, file=0x559dbb61e5c0 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:492 #2 0x0000559dbac03438 in inline_mysql_cond_wait (that=0x7f85a0016a08, mutex=0x7f85a0016958, src_file=0x559dbb61e8e0 "/home/stepan/galera/git/10.4/server/sql/wsrep_thd.h", src_line=69) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1178 #3 0x0000559dbac060d8 in Wsrep_thd_queue::pop_front (this=0x7f85a0016900) at /home/stepan/galera/git/10.4/server/sql/wsrep_thd.h:69 #4 0x0000559dbac04af4 in wsrep_rollback_process (rollbacker=0x7f85a0001d50, arg=0x0) at /home/stepan/galera/git/10.4/server/sql/wsrep_thd.cc:143 #5 0x0000559dbabf5576 in start_wsrep_THD (arg=0x559dbe530ec0) at /home/stepan/galera/git/10.4/server/sql/wsrep_mysqld.cc:2707 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f85a80e2700 (LWP 48862)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f85b0c4f787 in gu::Lock::wait (cond=..., this=0x7f85a80de2c0) at galerautils/src/gu_lock.hpp:45 #2 0x00007f85b0c603dc in galera::Monitor::enter (this=this@entry=0x559dbe4e0400, obj=...) at galera/src/monitor.hpp:156 #3 0x00007f85b0c54e87 in galera::ReplicatorSMM::commit_order_enter_local (this=this@entry=0x559dbe4df3a0, trx=...) at galera/src/replicator_smm.cpp:1280 #4 0x00007f85b0c73833 in galera_commit_order_enter (gh=, ws_handle=, meta=) at galera/src/wsrep_provider.cpp:658 #5 0x0000559dbb5665d9 in wsrep::wsrep_provider_v26::commit_order_enter (this=0x559dbe466750, ws_handle=..., ws_meta=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/wsrep_provider_v26.cpp:769 #6 0x0000559dbb55a8f1 in wsrep::transaction::after_statement (this=0x7f8530006a78) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/transaction.cpp:755 #7 0x0000559dbb54abcb in wsrep::client_state::after_statement (this=0x7f8530006a08) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/client_state.cpp:241 #8 0x0000559dba8b7122 in wsrep_after_statement (thd=0x7f8530000b00) at /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:317 #9 0x0000559dba8c124e in sp_instr_stmt::exec_core (this=0x7f8530020ac0, thd=0x7f8530000b00, nextp=0x7f85a80df0e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3607 #10 0x0000559dba8c046e in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f8530020b08, thd=0x7f8530000b00, nextp=0x7f85a80df0e4, open_tables=false, instr=0x7f8530020ac0) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3319 #11 0x0000559dba8c0ccc in sp_instr_stmt::execute (this=0x7f8530020ac0, thd=0x7f8530000b00, nextp=0x7f85a80df0e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3497 #12 0x0000559dba8ba7cb in sp_head::execute (this=0x7f853001b5e8, thd=0x7f8530000b00, merge_da_on_success=true) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:1341 #13 0x0000559dba8bd1f2 in sp_head::execute_procedure (this=0x7f853001b5e8, thd=0x7f8530000b00, args=0x7f8530005818) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:2283 #14 0x0000559dba9a2018 in do_execute_sp (thd=0x7f8530000b00, sp=0x7f853001b5e8) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:2959 #15 0x0000559dba9a2c04 in Sql_cmd_call::execute (this=0x7f8530014810, thd=0x7f8530000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:3201 #16 0x0000559dba9ae55b in mysql_execute_command (thd=0x7f8530000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:6330 #17 0x0000559dba9b3777 in mysql_parse (thd=0x7f8530000b00, rawbuf=0x7f8530014008 "call p1", length=7, parser_state=0x7f85a80e10b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:8141 #18 0x0000559dba9b2da6 in wsrep_mysql_parse (thd=0x7f8530000b00, rawbuf=0x7f8530014008 "call p1", length=7, parser_state=0x7f85a80e10b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7953 #19 0x0000559dba99eb44 in dispatch_command (command=COM_QUERY, thd=0x7f8530000b00, packet=0x7f853000a5e1 "call p1", packet_length=7, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1804 #20 0x0000559dba99d491 in do_command (thd=0x7f8530000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1358 #21 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #22 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #23 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe916f20) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #24 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #25 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f85877fe700 (LWP 48623)): #0 0x00007f85b88fa644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559dbafe669a in LinuxAIOHandler::collect (this=0x7f85877fdb00) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:1915 #2 0x0000559dbafe6bc3 in LinuxAIOHandler::poll (this=0x7f85877fdb00, m1=0x7f85877fdba8, m2=0x7f85877fdbb0, request=0x7f85877fdbe0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2060 #3 0x0000559dbafe6d26 in os_aio_linux_handler (global_segment=5, m1=0x7f85877fdba8, m2=0x7f85877fdbb0, request=0x7f85877fdbe0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2114 #4 0x0000559dbafeb84a in os_aio_handler (segment=5, m1=0x7f85877fdba8, m2=0x7f85877fdbb0, request=0x7f85877fdbe0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:5665 #5 0x0000559dbb23e963 in fil_aio_wait (segment=5) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4344 #6 0x0000559dbb0e4e97 in io_handler_thread (arg=0x559dbc275d88 ) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0start.cc:325 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f856affd700 (LWP 48639)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1373 in safe_cond_wait (cond=0x7f856affcc70, mp=0x7f856affccb0, file=0x559dbb7831c0 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:492 #2 0x0000559dbaf09d2b in inline_mysql_cond_wait (that=0x7f856affcc70, mutex=0x7f856affccb0, src_file=0x559dbb784b88 "/home/stepan/galera/git/10.4/server/storage/innobase/handler/ha_innodb.cc", src_line=294) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1178 #3 0x0000559dbaf0e967 in thd_destructor_proxy () at /home/stepan/galera/git/10.4/server/storage/innobase/handler/ha_innodb.cc:294 #4 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe8609b0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f857cff9700 (LWP 48636)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff337d in os_event::wait (this=0x559dbe61f9e0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:159 #2 0x0000559dbaff2e03 in os_event::wait_low (this=0x559dbe61f9e0, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:326 #3 0x0000559dbaff3174 in os_event_wait_low (event=0x559dbe61f9e0, reset_sig_count=0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:515 #4 0x0000559dbb1bf43f in buf_dump_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/buf/buf0dump.cc:833 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f8591006700 (LWP 48620)): #0 0x00007f85b88fa644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559dbafe669a in LinuxAIOHandler::collect (this=0x7f8591005b00) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:1915 #2 0x0000559dbafe6bc3 in LinuxAIOHandler::poll (this=0x7f8591005b00, m1=0x7f8591005ba8, m2=0x7f8591005bb0, request=0x7f8591005be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2060 #3 0x0000559dbafe6d26 in os_aio_linux_handler (global_segment=2, m1=0x7f8591005ba8, m2=0x7f8591005bb0, request=0x7f8591005be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2114 #4 0x0000559dbafeb84a in os_aio_handler (segment=2, m1=0x7f8591005ba8, m2=0x7f8591005bb0, request=0x7f8591005be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:5665 #5 0x0000559dbb23e963 in fil_aio_wait (segment=2) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4344 #6 0x0000559dbb0e4e97 in io_handler_thread (arg=0x559dbc275d70 ) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0start.cc:325 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f856bfff700 (LWP 48637)): #0 0x00007f85b8d30e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x0000559dbaff367f in os_thread_sleep (tm=1000000) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0thread.cc:225 #2 0x0000559dbb199fca in btr_defragment_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/btr/btr0defragment.cc:702 #3 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f8584df4700 (LWP 48628)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff2c98 in os_event::timed_wait (this=0x559dbe61f960, abstime=0x7f8584df3d20) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:276 #2 0x0000559dbaff2faf in os_event::wait_time_low (this=0x559dbe61f960, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:398 #3 0x0000559dbaff314f in os_event_wait_time_low (event=0x559dbe61f960, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:498 #4 0x0000559dbb0e1068 in srv_monitor_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:1672 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f8592008700 (LWP 48618)): #0 0x00007f85b88fa644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559dbafe669a in LinuxAIOHandler::collect (this=0x7f8592007b00) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:1915 #2 0x0000559dbafe6bc3 in LinuxAIOHandler::poll (this=0x7f8592007b00, m1=0x7f8592007ba8, m2=0x7f8592007bb0, request=0x7f8592007be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2060 #3 0x0000559dbafe6d26 in os_aio_linux_handler (global_segment=0, m1=0x7f8592007ba8, m2=0x7f8592007bb0, request=0x7f8592007be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2114 #4 0x0000559dbafeb84a in os_aio_handler (segment=0, m1=0x7f8592007ba8, m2=0x7f8592007bb0, request=0x7f8592007be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:5665 #5 0x0000559dbb23e963 in fil_aio_wait (segment=0) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4344 #6 0x0000559dbb0e4e97 in io_handler_thread (arg=0x559dbc275d60 ) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0start.cc:325 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f857e7fc700 (LWP 48633)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff337d in os_event::wait (this=0x559dbe61f7e0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:159 #2 0x0000559dbaff2e03 in os_event::wait_low (this=0x559dbe61f7e0, reset_sig_count=4439) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:326 #3 0x0000559dbaff3174 in os_event_wait_low (event=0x559dbe61f7e0, reset_sig_count=0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:515 #4 0x0000559dbb0df381 in srv_resume_thread (slot=0x559dbc0379b8 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:901 #5 0x0000559dbb0e2f85 in srv_worker_thread (arg=0x0) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:2483 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f85855f5700 (LWP 48627)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff2c98 in os_event::timed_wait (this=0x559dbe61f8e0, abstime=0x7f85855f4ba0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:276 #2 0x0000559dbaff2faf in os_event::wait_time_low (this=0x559dbe61f8e0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:398 #3 0x0000559dbaff314f in os_event_wait_time_low (event=0x559dbe61f8e0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:498 #4 0x0000559dbb0e158a in srv_error_monitor_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0srv.cc:1836 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f85a917a700 (LWP 48612)): #0 0x00007f85b70e8483 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f85b0bd8dd8 in asio::detail::epoll_reactor::run (this=0x559dbe518010, block=block@entry=true, ops=...) at asio/asio/detail/impl/epoll_reactor.ipp:391 #2 0x00007f85b0bdd4d0 in asio::detail::task_io_service::do_run_one (this=this@entry=0x559dbe517eb0, lock=..., this_thread=..., ec=...) at asio/asio/detail/impl/task_io_service.ipp:355 #3 0x00007f85b0bdd782 in asio::detail::task_io_service::run (this=0x559dbe517eb0, ec=...) at asio/asio/detail/impl/task_io_service.ipp:148 #4 0x00007f85b0bf51dd in asio::io_service::run (this=0x559dbe5169e0) at asio/asio/impl/io_service.ipp:58 #5 0x00007f85b0bf5931 in gcomm::AsioProtonet::event_loop (this=0x559dbe516940, period=...) at gcomm/src/asio_protonet.cpp:116 #6 0x00007f85b0c139cd in GCommConn::run (this=0x559dbe517690) at gcs/src/gcs_gcomm.cpp:498 #7 0x00007f85b0c16a79 in GCommConn::run_fn (arg=) at gcs/src/gcs_gcomm.cpp:209 #8 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #9 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f85865f7700 (LWP 48629)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbaff2c98 in os_event::timed_wait (this=0x559dbe829e30, abstime=0x7f85865f6d60) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:276 #2 0x0000559dbaff2faf in os_event::wait_time_low (this=0x559dbe829e30, time_in_usec=10000000, reset_sig_count=1) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:398 #3 0x0000559dbaff314f in os_event_wait_time_low (event=0x559dbe829e30, time_in_usec=10000000, reset_sig_count=0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0event.cc:498 #4 0x0000559dbb2285ff in dict_stats_thread () at /home/stepan/galera/git/10.4/server/storage/innobase/dict/dict0stats_bg.cc:460 #5 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f8587fff700 (LWP 48622)): #0 0x00007f85b88fa644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559dbafe669a in LinuxAIOHandler::collect (this=0x7f8587ffeb00) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:1915 #2 0x0000559dbafe6bc3 in LinuxAIOHandler::poll (this=0x7f8587ffeb00, m1=0x7f8587ffeba8, m2=0x7f8587ffebb0, request=0x7f8587ffebe0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2060 #3 0x0000559dbafe6d26 in os_aio_linux_handler (global_segment=4, m1=0x7f8587ffeba8, m2=0x7f8587ffebb0, request=0x7f8587ffebe0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2114 #4 0x0000559dbafeb84a in os_aio_handler (segment=4, m1=0x7f8587ffeba8, m2=0x7f8587ffebb0, request=0x7f8587ffebe0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:5665 #5 0x0000559dbb23e963 in fil_aio_wait (segment=4) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4344 #6 0x0000559dbb0e4e97 in io_handler_thread (arg=0x559dbc275d80 ) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0start.cc:325 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f8591807700 (LWP 48619)): #0 0x00007f85b88fa644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559dbafe669a in LinuxAIOHandler::collect (this=0x7f8591806b00) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:1915 #2 0x0000559dbafe6bc3 in LinuxAIOHandler::poll (this=0x7f8591806b00, m1=0x7f8591806ba8, m2=0x7f8591806bb0, request=0x7f8591806be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2060 #3 0x0000559dbafe6d26 in os_aio_linux_handler (global_segment=1, m1=0x7f8591806ba8, m2=0x7f8591806bb0, request=0x7f8591806be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2114 #4 0x0000559dbafeb84a in os_aio_handler (segment=1, m1=0x7f8591806ba8, m2=0x7f8591806bb0, request=0x7f8591806be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:5665 #5 0x0000559dbb23e963 in fil_aio_wait (segment=1) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4344 #6 0x0000559dbb0e4e97 in io_handler_thread (arg=0x559dbc275d68 ) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0start.cc:325 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f8590805700 (LWP 48621)): #0 0x00007f85b88fa644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559dbafe669a in LinuxAIOHandler::collect (this=0x7f8590804b00) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:1915 #2 0x0000559dbafe6bc3 in LinuxAIOHandler::poll (this=0x7f8590804b00, m1=0x7f8590804ba8, m2=0x7f8590804bb0, request=0x7f8590804be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2060 #3 0x0000559dbafe6d26 in os_aio_linux_handler (global_segment=3, m1=0x7f8590804ba8, m2=0x7f8590804bb0, request=0x7f8590804be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:2114 #4 0x0000559dbafeb84a in os_aio_handler (segment=3, m1=0x7f8590804ba8, m2=0x7f8590804bb0, request=0x7f8590804be0) at /home/stepan/galera/git/10.4/server/storage/innobase/os/os0file.cc:5665 #5 0x0000559dbb23e963 in fil_aio_wait (segment=3) at /home/stepan/galera/git/10.4/server/storage/innobase/fil/fil0fil.cc:4344 #6 0x0000559dbb0e4e97 in io_handler_thread (arg=0x559dbc275d78 ) at /home/stepan/galera/git/10.4/server/storage/innobase/srv/srv0start.cc:325 #7 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f859385a700 (LWP 48617)): #0 0x00007f85b8d2dd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559dbb4b1675 in safe_cond_timedwait (cond=0x559dbca875e0 , mp=0x559dbca87520 , abstime=0x7f8593859cb0, file=0x559dbb8f9030 "/home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/galera/git/10.4/server/mysys/thr_mutex.c:546 #2 0x0000559dbb31af27 in inline_mysql_cond_timedwait (that=0x559dbca875e0 , mutex=0x559dbca87520 , abstime=0x7f8593859cb0, src_file=0x559dbb8f9078 "/home/stepan/galera/git/10.4/server/storage/maria/ma_servicethread.c", src_line=116) at /home/stepan/galera/git/10.4/server/include/mysql/psi/mysql_thread.h:1204 #3 0x0000559dbb31b44b in my_service_thread_sleep (control=0x559dbc03bd00 , sleep_time=30000000000) at /home/stepan/galera/git/10.4/server/storage/maria/ma_servicethread.c:115 #4 0x0000559dbb30f8e4 in ma_checkpoint_background (arg=0x1e) at /home/stepan/galera/git/10.4/server/storage/maria/ma_checkpoint.c:707 #5 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe5d7a50) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #6 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f85b051e700 (LWP 48616)): #0 0x00007f85b8d2d965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f85b0c4f787 in gu::Lock::wait (cond=..., this=0x7f85b051b490) at galerautils/src/gu_lock.hpp:45 #2 0x00007f85b0c603dc in galera::Monitor::enter (this=this@entry=0x559dbe4e0400, obj=...) at galera/src/monitor.hpp:156 #3 0x00007f85b0c53b48 in galera::ReplicatorSMM::commit_order_enter_remote (this=this@entry=0x559dbe4df3a0, trx=...) at galera/src/replicator_smm.cpp:1342 #4 0x00007f85b0c7385b in galera_commit_order_enter (gh=, ws_handle=, meta=) at galera/src/wsrep_provider.cpp:663 #5 0x0000559dbb5665d9 in wsrep::wsrep_provider_v26::commit_order_enter (this=0x559dbe466750, ws_handle=..., ws_meta=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/wsrep_provider_v26.cpp:769 #6 0x0000559dbb55ab6b in wsrep::transaction::after_applying (this=0x7f8598006a78) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/transaction.cpp:793 #7 0x0000559dbabe53d5 in wsrep::client_state::after_applying (this=0x7f8598006a08) at /home/stepan/galera/git/10.4/server/wsrep-lib/include/wsrep/client_state.hpp:254 #8 0x0000559dbabe3dd7 in Wsrep_high_priority_service::log_dummy_write_set (this=0x7f85b051d4d0, ws_handle=..., ws_meta=...) at /home/stepan/galera/git/10.4/server/sql/wsrep_high_priority_service.cc:432 #9 0x0000559dbb54eb4e in apply_write_set (server_state=..., high_priority_service=..., ws_handle=..., ws_meta=..., data=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/server_state.cpp:155 #10 0x0000559dbb5524bd in wsrep::server_state::on_apply (this=0x559dbe4a9be0, high_priority_service=..., ws_handle=..., ws_meta=..., data=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/server_state.cpp:966 #11 0x0000559dbb567cd1 in wsrep::high_priority_service::apply (this=0x7f85b051d4d0, ws_handle=..., ws_meta=..., data=...) at /home/stepan/galera/git/10.4/server/wsrep-lib/include/wsrep/high_priority_service.hpp:46 #12 0x0000559dbb5653b8 in (anonymous namespace)::apply_cb (ctx=0x7f85b051d4d0, wsh=0x7f85b051c3e0, flags=67, buf=0x7f85b051c420, meta=0x7f85b051c6b0, exit_loop=0x7f85b051c5ef) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/wsrep_provider_v26.cpp:489 #13 0x00007f85b0c20a91 in galera::TrxHandleSlave::apply (this=this@entry=0x7f85980516c0, recv_ctx=recv_ctx@entry=0x7f85b051d4d0, apply_cb=0x559dbb565195 <(anonymous namespace)::apply_cb(void*, wsrep_ws_handle_t const*, uint32_t, wsrep_buf_t const*, wsrep_trx_meta_t const*, wsrep_bool_t*)>, meta=..., exit_loop=exit_loop@entry=@0x7f85b051c5ef: false) at galera/src/trx_handle.cpp:418 #14 0x00007f85b0c53502 in galera::ReplicatorSMM::apply_trx (this=this@entry=0x559dbe4df3a0, recv_ctx=recv_ctx@entry=0x7f85b051d4d0, ts=...) at galera/src/replicator_smm.cpp:489 #15 0x00007f85b0c591b0 in galera::ReplicatorSMM::process_trx (this=0x559dbe4df3a0, recv_ctx=0x7f85b051d4d0, ts_ptr=...) at galera/src/replicator_smm.cpp:2114 #16 0x00007f85b0c3a37b in galera::GcsActionSource::process_writeset (this=this@entry=0x559dbe4e7070, recv_ctx=0x7f85b051d4d0, act=..., exit_loop=@0x7f85b051d29e: false) at galera/src/gcs_action_source.cpp:62 #17 0x00007f85b0c3a555 in galera::GcsActionSource::dispatch (this=this@entry=0x559dbe4e7070, recv_ctx=recv_ctx@entry=0x7f85b051d4d0, act=..., exit_loop=@0x7f85b051d29e: false) at galera/src/gcs_action_source.cpp:109 #18 0x00007f85b0c3a92d in galera::GcsActionSource::process (this=0x559dbe4e7070, recv_ctx=0x7f85b051d4d0, exit_loop=@0x7f85b051d29e: false) at galera/src/gcs_action_source.cpp:182 #19 0x00007f85b0c597a3 in galera::ReplicatorSMM::async_recv (this=0x559dbe4df3a0, recv_ctx=0x7f85b051d4d0) at galera/src/replicator_smm.cpp:383 #20 0x0000559dbb566076 in wsrep::wsrep_provider_v26::run_applier (this=0x559dbe466750, applier_ctx=0x7f85b051d4d0) at /home/stepan/galera/git/10.4/server/wsrep-lib/src/wsrep_provider_v26.cpp:690 #21 0x0000559dbac039e7 in wsrep_replication_process (thd=0x7f8598000b00, arg=0x0) at /home/stepan/galera/git/10.4/server/sql/wsrep_thd.cc:61 #22 0x0000559dbabf5576 in start_wsrep_THD (arg=0x559dbe52fd20) at /home/stepan/galera/git/10.4/server/sql/wsrep_mysqld.cc:2707 #23 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #24 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f85a804c700 (LWP 48910)): #0 0x00007f85b8d2e9d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000559dbb4abd44 in my_write_core (sig=6) at /home/stepan/galera/git/10.4/server/mysys/stacktrace.c:481 #2 0x0000559dbace4a51 in handle_fatal_signal (sig=6) at /home/stepan/galera/git/10.4/server/sql/signal_handler.cc:305 #3 #4 0x00007f85b7020207 in raise () from /lib64/libc.so.6 #5 0x00007f85b70218f8 in abort () from /lib64/libc.so.6 #6 0x00007f85b7019026 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f85b70190d2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000559dbace69cf in wsrep_commit_empty (thd=0x7f852c000b00, all=true) at /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:422 #9 0x0000559dbacea346 in ha_commit_trans (thd=0x7f852c000b00, all=true) at /home/stepan/galera/git/10.4/server/sql/handler.cc:1677 #10 0x0000559dbab28fa8 in trans_commit (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/transaction.cc:313 #11 0x0000559dba9ac6ae in mysql_execute_command (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:5849 #12 0x0000559dba8c1180 in sp_instr_stmt::exec_core (this=0x7f852c020ac0, thd=0x7f852c000b00, nextp=0x7f85a80490e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3591 #13 0x0000559dba8c046e in sp_lex_keeper::reset_lex_and_exec_core (this=0x7f852c020b08, thd=0x7f852c000b00, nextp=0x7f85a80490e4, open_tables=false, instr=0x7f852c020ac0) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3319 #14 0x0000559dba8c0ccc in sp_instr_stmt::execute (this=0x7f852c020ac0, thd=0x7f852c000b00, nextp=0x7f85a80490e4) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:3497 #15 0x0000559dba8ba7cb in sp_head::execute (this=0x7f852c01b5e8, thd=0x7f852c000b00, merge_da_on_success=true) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:1341 #16 0x0000559dba8bd1f2 in sp_head::execute_procedure (this=0x7f852c01b5e8, thd=0x7f852c000b00, args=0x7f852c005818) at /home/stepan/galera/git/10.4/server/sql/sp_head.cc:2283 #17 0x0000559dba9a2018 in do_execute_sp (thd=0x7f852c000b00, sp=0x7f852c01b5e8) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:2959 #18 0x0000559dba9a2c04 in Sql_cmd_call::execute (this=0x7f852c0140b0, thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:3201 #19 0x0000559dba9ae55b in mysql_execute_command (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:6330 #20 0x0000559dba9b3777 in mysql_parse (thd=0x7f852c000b00, rawbuf=0x7f852c014008 "call p1", length=7, parser_state=0x7f85a804b0b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:8141 #21 0x0000559dba9b2da6 in wsrep_mysql_parse (thd=0x7f852c000b00, rawbuf=0x7f852c014008 "call p1", length=7, parser_state=0x7f85a804b0b0, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7953 #22 0x0000559dba99eb44 in dispatch_command (command=COM_QUERY, thd=0x7f852c000b00, packet=0x7f852c00a5e1 "call p1", packet_length=7, is_com_multi=false, is_next_command=false) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1804 #23 0x0000559dba99d491 in do_command (thd=0x7f852c000b00) at /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:1358 #24 0x0000559dbab14180 in do_handle_one_connection (connect=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1399 #25 0x0000559dbab13ec4 in handle_one_connection (arg=0x559dbe8fd6c0) at /home/stepan/galera/git/10.4/server/sql/sql_connect.cc:1302 #26 0x0000559dbb443bd1 in pfs_spawn_thread (arg=0x559dbe8864f0) at /home/stepan/galera/git/10.4/server/storage/perfschema/pfs.cc:1862 #27 0x00007f85b8d29dd5 in start_thread () from /lib64/libpthread.so.0 #28 0x00007f85b70e7ead in clone () from /lib64/libc.so.6 - saving '/home/stepan/galera/git/10.4/server/mysql-test/var/log/galera.galera_bf_lock_wait-innodb/' to '/home/stepan/galera/git/10.4/server/mysql-test/var/log/galera.galera_bf_lock_wait-innodb/' ***Warnings generated in error logs during shutdown after running tests: galera.galera_bf_lock_wait mysqld: /home/stepan/galera/git/10.4/server/sql/wsrep_trans_observer.h:422: void wsrep_commit_empty(THD*, bool): Assertion `have_error || !wsrep_current_error(thd)' failed. Attempting backtrace. You can use the following information to find out galera.galera_bf_lock_wait 'innodb' [ 3 pass ] 106339 ***Warnings generated in error logs during shutdown after running tests: galera.galera_bf_lock_wait mysqld: /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7936: void wsrep_prepare_for_autocommit_retry(THD*, char*, uint, Parser_state*): Assertion `thd->wsrep_trx().active() == false' failed. Attempting backtrace. You can use the following information to find out mysqld: /home/stepan/galera/git/10.4/server/sql/sql_parse.cc:7936: void wsrep_prepare_for_autocommit_retry(THD*, char*, uint, Parser_state*): Assertion `thd->wsrep_trx().active() == false' failed. -------------------------------------------------------------------------- The servers were restarted 1 times Spent 213.490 of 382 seconds executing testcases Completed: Failed 1/3 tests, 66.67% were successful. Failing test(s): galera.galera_bf_lock_wait The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html Errors/warnings were found in logfiles during server shutdown after running the following sequence(s) of tests: galera.galera_bf_lock_wait galera.galera_bf_lock_wait mysql-test-run: *** ERROR: there were failing test cases