Checking supported features... MariaDB Version 10.4.9-MariaDB-debug - SSL connections supported - binaries are debug compiled - binaries built with wsrep patch Using suites: galera_sr Collecting tests... Installing system database... Using parallel: 4 worker[2] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039 worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 worker[4] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079 worker[3] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059 ============================================================================== TEST WORKER RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- 191025 22:26:52 +3.078 galera_sr.GCF-1060 [ disabled ] MDEV-20848 Galera test failure on galera_sr.GCF_1060 191025 22:26:52 +0.000 galera_sr.GCF-437 [ disabled ] test relies on InnoDB redo log size limitation 191025 22:26:52 +0.000 galera_sr.galera_sr_table_contents [ disabled ] missing file 191025 22:27:01 +8.273 galera_sr.galera_sr_gtid 'innodb' w4 [ pass ] 2190 191025 22:27:01 +0.452 galera_sr.galera_sr_log_bin 'innodb' w3 [ pass ] 2660 191025 22:27:02 +1.070 galera_sr.mysql-wsrep-features#32 'innodb' w1 [ pass ] 3592 191025 22:27:19 +16.752 galera_sr.mysql-wsrep-features#136 'innodb' w4 [ pass ] 2134 191025 22:27:19 +0.012 galera_sr.galera_sr_v1_row_events 'innodb' w3 [ pass ] 2103 191025 22:27:37 +18.414 galera_sr.GCF-627 'innodb' w4 [ pass ] 2172 191025 22:27:37 +0.026 galera_sr.galera_sr_create_drop 'innodb' w3 [ pass ] 2125 191025 22:27:38 +0.902 galera_sr.GCF-1018B 'innodb' w1 [ pass ] 20273 191025 22:27:40 +1.756 galera_sr.galera_sr_ddl_master 'innodb' w3 [ pass ] 2186 191025 22:27:42 +1.813 galera_sr.GCF-845 'innodb' w4 [ pass ] 4119 191025 22:27:42 +0.682 galera_sr.galera_sr_ddl_schema 'innodb' w3 [ pass ] 2182 191025 22:27:44 +1.726 galera_sr.GCF-851 'innodb' w4 [ pass ] 2100 191025 22:27:45 +0.874 galera_sr.galera_sr_ddl_slave 'innodb' w3 [ pass ] 2221 191025 22:27:48 +2.516 galera_sr.GCF-867 'innodb' w4 [ pass ] 2980 191025 22:27:48 +0.291 galera_sr.galera_sr_ddl_unrelated 'innodb' w3 [ pass ] 2460 191025 22:27:50 +2.177 galera_sr.GCF-889 'innodb' w4 [ pass ] 2118 191025 22:27:50 +0.419 galera_sr.galera_sr_dupkey_error 'innodb' w3 [ pass ] 2298 191025 22:27:53 +2.299 galera_sr.GCF-900 'innodb' w4 [ pass ] 2375 191025 22:27:57 +4.292 galera_sr.galera_sr_fk_conflict 'innodb' w3 [ pass ] 6285 191025 22:27:57 +0.279 galera_sr.galera_sr_bf_abort 'innodb' w4 [ pass ] 4166 191025 22:28:00 +2.677 galera_sr.galera_sr_blob 'innodb' w4 [ pass ] 2315 191025 22:28:06 +5.780 galera_sr.galera_sr_cc_master 'innodb' w4 [ fail ] Test ended at 2019-10-25 22:28:06 CURRENT_TEST: galera_sr.galera_sr_cc_master mysqltest: At line 44: query 'SET GLOBAL wsrep_cluster_address = ''' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > SET AUTOCOMMIT=OFF; START TRANSACTION; INSERT INTO t1 VALUES (1); INSERT INTO t1 VALUES (2); INSERT INTO t1 VALUES (3); INSERT INTO t1 VALUES (4); INSERT INTO t1 VALUES (5); SELECT COUNT(*) FROM mysql.wsrep_streaming_log; COUNT(*) 5 connection node_1; SELECT COUNT(*) FROM mysql.wsrep_streaming_log; COUNT(*) 5 connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2; connection node_2a; SET SESSION wsrep_sync_wait=0; SET GLOBAL wsrep_cluster_address = ''; More results from queries before failure can be found in /home/stepan/mariadb/10.4/git/mysql-test/var/4/log/galera_sr_cc_master.log Server [mysqld.2 - pid: 12603, winpid: 12603, exit: 256] failed during test run Server log from this test: ----------SERVER LOG START----------- 2019-10-25 22:28:02 140 [Note] WSREP: Stop replication by 140 2019-10-25 22:28:02 140 [Note] WSREP: Server status change synced -> disconnecting 2019-10-25 22:28:02 140 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-10-25 22:28:02 140 [Note] WSREP: Closing send monitor... 2019-10-25 22:28:02 140 [Note] WSREP: Closed send monitor. 2019-10-25 22:28:02 140 [Note] WSREP: gcomm: terminating thread 2019-10-25 22:28:02 140 [Note] WSREP: gcomm: joining thread 2019-10-25 22:28:02 140 [Note] WSREP: gcomm: closing backend 2019-10-25 22:28:03 140 [Note] WSREP: view(view_id(NON_PRIM,7c15e3e0,2) memb { 7c79d6ce,0 } joined { } left { } partitioned { 7c15e3e0,0 }) 2019-10-25 22:28:03 140 [Note] WSREP: PC protocol downgrade 1 -> 0 2019-10-25 22:28:03 140 [Note] WSREP: view((empty)) 2019-10-25 22:28:03 140 [Note] WSREP: gcomm: closed 2019-10-25 22:28:03 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 2019-10-25 22:28:03 0 [Note] WSREP: Flow-control interval: [16, 16] 2019-10-25 22:28:03 0 [Note] WSREP: Received NON-PRIMARY. 2019-10-25 22:28:03 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 216) 2019-10-25 22:28:03 0 [Note] WSREP: New SELF-LEAVE. 2019-10-25 22:28:03 0 [Note] WSREP: Flow-control interval: [0, 0] 2019-10-25 22:28:03 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. 2019-10-25 22:28:03 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: -1) 2019-10-25 22:28:03 0 [Note] WSREP: RECV thread exiting 0: Success 2019-10-25 22:28:03 140 [Note] WSREP: recv_thread() joined. 2019-10-25 22:28:03 140 [Note] WSREP: Closing replication queue. 2019-10-25 22:28:03 140 [Note] WSREP: Closing slave action queue. 2019-10-25 22:28:03 2 [Note] WSREP: ####### processing CC -1, local, ordered 2019-10-25 22:28:03 2 [Note] WSREP: ####### My UUID: 7c79d6ce-f75d-11e9-8a80-13083ff677cf 2019-10-25 22:28:03 2 [Note] WSREP: ####### ST not required 2019-10-25 22:28:03 2 [Note] WSREP: ================================================ View: id: 7c1cbeb7-f75d-11e9-a0f5-662e63655745:-1 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 0 members(1): 0: 7c79d6ce-f75d-11e9-8a80-13083ff677cf, cnt7glr11.localdomain ================================================= 2019-10-25 22:28:03 2 [Note] WSREP: Non-primary view 2019-10-25 22:28:03 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-10-25 22:28:03 2 [Note] WSREP: ####### processing CC -1, local, ordered 2019-10-25 22:28:03 2 [Note] WSREP: ####### My UUID: 7c79d6ce-f75d-11e9-8a80-13083ff677cf 2019-10-25 22:28:03 2 [Note] WSREP: ####### ST not required 2019-10-25 22:28:03 2 [Note] WSREP: ================================================ View: id: 7c1cbeb7-f75d-11e9-a0f5-662e63655745:-1 status: non-primary protocol_version: 4 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: yes own_index: -1 members(0): ================================================= 2019-10-25 22:28:03 2 [Note] WSREP: Non-primary view 2019-10-25 22:28:03 2 [Note] WSREP: Server status change disconnecting -> disconnected 2019-10-25 22:28:03 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-10-25 22:28:03 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-10-25 22:28:03 2 [Note] WSREP: Applier thread exiting ret: 0 thd: 2 2019-10-25 22:28:03 140 [Note] WSREP: killing local connection: 134 2019-10-25 22:28:03 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) mysqld: /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_socket.h:779: inline_mysql_socket_recv: Assertion `mysql_socket.fd != -1' failed. 191025 22:28:03 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.4.9-MariaDB-debug-log key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=2 max_threads=153 thread_count=9 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63618 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f5d1000a040 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f5d3086cd30 thread_stack 0x49000 /home/stepan/mariadb/10.4/git/sql/mysqld(my_print_stacktrace+0x40)[0x559feed99008] mysys/stacktrace.c:269(my_print_stacktrace)[0x559fee5055dd] sigaction.c:0(__restore_rt)[0x7f5d4b4a55d0] :0(__GI_raise)[0x7f5d49793207] :0(__GI_abort)[0x7f5d497948f8] :0(__assert_fail_base)[0x7f5d4978c026] :0(__GI___assert_fail)[0x7f5d4978c0d2] /home/stepan/mariadb/10.4/git/sql/mysqld(+0x1464638)[0x559feee07638] psi/mysql_socket.h:781(inline_mysql_socket_recv)[0x559feee07d2e] sql/net_serv.cc:962(my_real_read(st_net*, unsigned long*, char))[0x559fee6861e3] sql/net_serv.cc:1243(my_net_read_packet_reallen)[0x559fee686ae1] sql/net_serv.cc:1228(my_net_read_packet)[0x559fee686a94] sql/sql_parse.cc:1232(do_command(THD*))[0x559fee19d9de] sql/sql_connect.cc:1412(do_handle_one_connection(CONNECT*))[0x559fee327d51] sql/sql_connect.cc:1317(handle_one_connection)[0x559fee327a80] perfschema/pfs.cc:1864(pfs_spawn_thread)[0x559feed274a5] pthread_create.c:0(start_thread)[0x7f5d4b49ddd5] /lib64/libc.so.6(clone+0x6d)[0x7f5d4985aead] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x0): Connection ID (thread ID): 134 Status: KILL_CONNECTION Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file... Working directory at /home/stepan/mariadb/10.4/git/mysql-test/var/4/mysqld.2/data Resource Limits: Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 4096 23005 processes Max open files 1024 1024 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 23005 23005 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us Core pattern: |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h ----------SERVER LOG END------------- - found 'core.12605' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace from coredump /home/stepan/mariadb/10.4/git/mysql-test/var/4/log/galera_sr.galera_sr_cc_master-innodb/mysqld.2/data/core.12605 Core generated by '/home/stepan/mariadb/10.4/git/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 13163] [New LWP 12614] [New LWP 13078] [New LWP 12648] [New LWP 13082] [New LWP 14442] [New LWP 13131] [New LWP 13080] [New LWP 12613] [New LWP 13137] [New LWP 13132] [New LWP 13067] [New LWP 13138] [New LWP 13133] [New LWP 13130] [New LWP 13139] [New LWP 13135] [New LWP 13134] [New LWP 13155] [New LWP 13152] [New LWP 13136] [New LWP 13153] [New LWP 13151] [New LWP 13154] [New LWP 12605] [New LWP 13162] [New LWP 13081] [New LWP 13079] [New LWP 13077] [New LWP 13076] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/home/stepan/mariadb/10.4/git/sql/mysqld --defaults-group-suffix=.2 --defaults-'. Program terminated with signal 6, Aborted. #0 0x00007f5d4b4a29d1 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007f5d4b4a29d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000559feed990f6 in my_write_core (sig=6) at /home/stepan/mariadb/10.4/git/mysys/stacktrace.c:481 #2 0x0000559fee505964 in handle_fatal_signal (sig=6) at /home/stepan/mariadb/10.4/git/sql/signal_handler.cc:343 #3 #4 0x00007f5d49793207 in raise () from /lib64/libc.so.6 #5 0x00007f5d497948f8 in abort () from /lib64/libc.so.6 #6 0x00007f5d4978c026 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f5d4978c0d2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000559feee07638 in inline_mysql_socket_recv (src_file=0x559fef391f70 "/home/stepan/mariadb/10.4/git/vio/viosocket.c", src_line=190, mysql_socket=..., buf=0x7f5d10028ba0, n=4, flags=64) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_socket.h:779 #9 0x0000559feee07d2e in vio_read (vio=0x559ff0c0e6a0, buf=0x7f5d10028ba0 "\001", size=4) at /home/stepan/mariadb/10.4/git/vio/viosocket.c:189 #10 0x0000559fee6861e3 in my_real_read (net=0x7f5d1000a338, complen=0x7f5d3086c810, header=1 '\001') at /home/stepan/mariadb/10.4/git/sql/net_serv.cc:962 #11 0x0000559fee686ae1 in my_net_read_packet_reallen (net=0x7f5d1000a338, read_from_server=1 '\001', reallen=0x7f5d3086c878) at /home/stepan/mariadb/10.4/git/sql/net_serv.cc:1243 #12 0x0000559fee686a94 in my_net_read_packet (net=0x7f5d1000a338, read_from_server=1 '\001') at /home/stepan/mariadb/10.4/git/sql/net_serv.cc:1227 #13 0x0000559fee19d9de in do_command (thd=0x7f5d1000a040) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:1232 #14 0x0000559fee327d51 in do_handle_one_connection (connect=0x559ff0cff900) at /home/stepan/mariadb/10.4/git/sql/sql_connect.cc:1412 #15 0x0000559fee327a80 in handle_one_connection (arg=0x559ff0cff900) at /home/stepan/mariadb/10.4/git/sql/sql_connect.cc:1316 #16 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff0d01c90) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #17 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #18 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 30 (Thread 0x7f5d17fff700 (LWP 13076)): #0 0x00007f5d4b06e644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559fee845dad in LinuxAIOHandler::collect (this=0x7f5d17ffeb00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1809 #2 0x0000559fee8463f6 in LinuxAIOHandler::poll (this=0x7f5d17ffeb00, m1=0x7f5d17ffeba8, m2=0x7f5d17ffebb0, request=0x7f5d17ffebe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1968 #3 0x0000559fee846559 in os_aio_linux_handler (global_segment=0, m1=0x7f5d17ffeba8, m2=0x7f5d17ffebb0, request=0x7f5d17ffebe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:2022 #4 0x0000559fee84b2c0 in os_aio_handler (segment=0, m1=0x7f5d17ffeba8, m2=0x7f5d17ffebb0, request=0x7f5d17ffebe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:5561 #5 0x0000559feeb014f9 in fil_aio_wait (segment=0) at /home/stepan/mariadb/10.4/git/storage/innobase/fil/fil0fil.cc:4357 #6 0x0000559fee972e42 in io_handler_thread (arg=0x559fefbaa980 ) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0start.cc:324 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7f5d177fe700 (LWP 13077)): #0 0x00007f5d4b06e644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559fee845dad in LinuxAIOHandler::collect (this=0x7f5d177fdb00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1809 #2 0x0000559fee8463f6 in LinuxAIOHandler::poll (this=0x7f5d177fdb00, m1=0x7f5d177fdba8, m2=0x7f5d177fdbb0, request=0x7f5d177fdbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1968 #3 0x0000559fee846559 in os_aio_linux_handler (global_segment=1, m1=0x7f5d177fdba8, m2=0x7f5d177fdbb0, request=0x7f5d177fdbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:2022 #4 0x0000559fee84b2c0 in os_aio_handler (segment=1, m1=0x7f5d177fdba8, m2=0x7f5d177fdbb0, request=0x7f5d177fdbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:5561 #5 0x0000559feeb014f9 in fil_aio_wait (segment=1) at /home/stepan/mariadb/10.4/git/storage/innobase/fil/fil0fil.cc:4357 #6 0x0000559fee972e42 in io_handler_thread (arg=0x559fefbaa988 ) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0start.cc:324 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7f5d167fc700 (LWP 13079)): #0 0x00007f5d4b06e644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559fee845dad in LinuxAIOHandler::collect (this=0x7f5d167fbb00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1809 #2 0x0000559fee8463f6 in LinuxAIOHandler::poll (this=0x7f5d167fbb00, m1=0x7f5d167fbba8, m2=0x7f5d167fbbb0, request=0x7f5d167fbbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1968 #3 0x0000559fee846559 in os_aio_linux_handler (global_segment=3, m1=0x7f5d167fbba8, m2=0x7f5d167fbbb0, request=0x7f5d167fbbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:2022 #4 0x0000559fee84b2c0 in os_aio_handler (segment=3, m1=0x7f5d167fbba8, m2=0x7f5d167fbbb0, request=0x7f5d167fbbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:5561 #5 0x0000559feeb014f9 in fil_aio_wait (segment=3) at /home/stepan/mariadb/10.4/git/storage/innobase/fil/fil0fil.cc:4357 #6 0x0000559fee972e42 in io_handler_thread (arg=0x559fefbaa998 ) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0start.cc:324 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7f5d157fa700 (LWP 13081)): #0 0x00007f5d4b06e644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559fee845dad in LinuxAIOHandler::collect (this=0x7f5d157f9b00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1809 #2 0x0000559fee8463f6 in LinuxAIOHandler::poll (this=0x7f5d157f9b00, m1=0x7f5d157f9ba8, m2=0x7f5d157f9bb0, request=0x7f5d157f9be0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1968 #3 0x0000559fee846559 in os_aio_linux_handler (global_segment=5, m1=0x7f5d157f9ba8, m2=0x7f5d157f9bb0, request=0x7f5d157f9be0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:2022 #4 0x0000559fee84b2c0 in os_aio_handler (segment=5, m1=0x7f5d157f9ba8, m2=0x7f5d157f9bb0, request=0x7f5d157f9be0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:5561 #5 0x0000559feeb014f9 in fil_aio_wait (segment=5) at /home/stepan/mariadb/10.4/git/storage/innobase/fil/fil0fil.cc:4357 #6 0x0000559fee972e42 in io_handler_thread (arg=0x559fefbaa9a8 ) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0start.cc:324 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7f5d40063700 (LWP 13162)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559feed9e769 in safe_cond_wait (cond=0x559fefb2c0a0 , mp=0x559fefb2f320 , file=0x559feee6d040 "/home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h", line=1167) at /home/stepan/mariadb/10.4/git/mysys/thr_mutex.c:492 #2 0x0000559fee09008e in inline_mysql_cond_wait (that=0x559fefb2c0a0 , mutex=0x559fefb2f320 , src_file=0x559feee6db80 "/home/stepan/mariadb/10.4/git/sql/slave.cc", src_line=519) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h:1167 #3 0x0000559fee0919cb in handle_slave_background (arg=0x0) at /home/stepan/mariadb/10.4/git/sql/slave.cc:519 #4 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff0c76690) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7f5d4b8ae880 (LWP 12605)): #0 0x00007f5d4985020d in poll () from /lib64/libc.so.6 #1 0x0000559fee06a2db in handle_connections_sockets () at /home/stepan/mariadb/10.4/git/sql/mysqld.cc:6455 #2 0x0000559fee069576 in mysqld_main (argc=163, argv=0x559ff0642130) at /home/stepan/mariadb/10.4/git/sql/mysqld.cc:5891 #3 0x0000559fee05d095 in main (argc=23, argv=0x7ffd7ed249d8) at /home/stepan/mariadb/10.4/git/sql/main.cc:25 Thread 24 (Thread 0x7f5cf4ff9700 (LWP 13154)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559feed9e769 in safe_cond_wait (cond=0x7f5cf4ff8c70, mp=0x7f5cf4ff8cb0, file=0x559fef08adc0 "/home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/mariadb/10.4/git/mysys/thr_mutex.c:492 #2 0x0000559fee74715d in inline_mysql_cond_wait (that=0x7f5cf4ff8c70, mutex=0x7f5cf4ff8cb0, src_file=0x559fef08c888 "/home/stepan/mariadb/10.4/git/storage/innobase/handler/ha_innodb.cc", src_line=288) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h:1178 #3 0x0000559fee74ca55 in thd_destructor_proxy () at /home/stepan/mariadb/10.4/git/storage/innobase/handler/ha_innodb.cc:288 #4 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff0bb39e0) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f5cf67fc700 (LWP 13151)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee854377 in os_event::wait (this=0x559ff09094f0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:158 #2 0x0000559fee853e6b in os_event::wait_low (this=0x559ff09094f0, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:325 #3 0x0000559fee85416e in os_event_wait_low (event=0x559ff09094f0, reset_sig_count=0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:502 #4 0x0000559feea71e86 in buf_dump_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/buf/buf0dump.cc:833 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7f5cf57fa700 (LWP 13153)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee854377 in os_event::wait (this=0x559ff09095f0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:158 #2 0x0000559fee853e6b in os_event::wait_low (this=0x559ff09095f0, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:325 #3 0x0000559fee85416e in os_event_wait_low (event=0x559ff09095f0, reset_sig_count=0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:502 #4 0x0000559feea577d9 in buf_resize_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/buf/buf0buf.cc:3199 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f5d08ff9700 (LWP 13136)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee854377 in os_event::wait (this=0x559ff09091f0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:158 #2 0x0000559fee853e6b in os_event::wait_low (this=0x559ff09091f0, reset_sig_count=257) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:325 #3 0x0000559fee85416e in os_event_wait_low (event=0x559ff09091f0, reset_sig_count=257) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:502 #4 0x0000559fee96c328 in srv_resume_thread (slot=0x559fef966c10 , sig_count=257, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:909 #5 0x0000559fee970aa0 in srv_purge_coordinator_suspend (slot=0x559fef966c10 , rseg_history_len=16) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2657 #6 0x0000559fee970dd1 in srv_purge_coordinator_thread (arg=0x0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2738 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f5cf5ffb700 (LWP 13152)): #0 0x00007f5d4b4a4e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x0000559fee8546b4 in os_thread_sleep (tm=1000000) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0thread.cc:225 #2 0x0000559feea46dd3 in btr_defragment_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/btr/btr0defragment.cc:705 #3 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f5d400ae700 (LWP 13155)): #0 0x00007f5d4979440a in sigwaitinfo () from /lib64/libc.so.6 #1 0x0000559fee05d114 in my_sigwait (set=0x7f5d400adcf0, sig=0x7f5d400adcb4, code=0x7f5d400adcb8) at /home/stepan/mariadb/10.4/git/include/my_pthread.h:198 #2 0x0000559fee063a8d in signal_hand (arg=0x0) at /home/stepan/mariadb/10.4/git/sql/mysqld.cc:3243 #3 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff08c3710) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #4 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f5d097fa700 (LWP 13134)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee853d00 in os_event::timed_wait (this=0x559ff0aadee0, abstime=0x7f5d097f9b70) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:275 #2 0x0000559fee853fa9 in os_event::wait_time_low (this=0x559ff0aadee0, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:385 #3 0x0000559fee854149 in os_event_wait_time_low (event=0x559ff0aadee0, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:485 #4 0x0000559fee9dc563 in ib_wqueue_timedwait (wq=0x559ff0aaddb0, wait_in_usecs=5000000) at /home/stepan/mariadb/10.4/git/storage/innobase/ut/ut0wqueue.cc:163 #5 0x0000559feeb4f23d in fts_optimize_thread (arg=0x559ff0aaddb0) at /home/stepan/mariadb/10.4/git/storage/innobase/fts/fts0opt.cc:2865 #6 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f5d0bfff700 (LWP 13135)): #0 0x00007f5d4b4a4e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x0000559fee8546b4 in os_thread_sleep (tm=1000000) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0thread.cc:225 #2 0x0000559fee96fd65 in srv_master_sleep () at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2334 #3 0x0000559fee96fecd in srv_master_thread (arg=0x0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2375 #4 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f5cf6ffd700 (LWP 13139)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee854377 in os_event::wait (this=0x559ff0909370) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:158 #2 0x0000559fee853e6b in os_event::wait_low (this=0x559ff0909370, reset_sig_count=902) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:325 #3 0x0000559fee85416e in os_event_wait_low (event=0x559ff0909370, reset_sig_count=0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:502 #4 0x0000559fee96c328 in srv_resume_thread (slot=0x559fef966f10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:909 #5 0x0000559fee9705e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f5d0b7fe700 (LWP 13130)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee853d00 in os_event::timed_wait (this=0x559ff0b446e0, abstime=0x7f5d0b7fdd50) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:275 #2 0x0000559fee853fa9 in os_event::wait_time_low (this=0x559ff0b446e0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:385 #3 0x0000559fee854149 in os_event_wait_time_low (event=0x559ff0b446e0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:485 #4 0x0000559fee816527 in lock_wait_timeout_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/lock/lock0wait.cc:522 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f5d09ffb700 (LWP 13133)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee853d00 in os_event::timed_wait (this=0x559ff0b76580, abstime=0x7f5d09ffad70) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:275 #2 0x0000559fee853fa9 in os_event::wait_time_low (this=0x559ff0b76580, time_in_usec=10000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:385 #3 0x0000559fee854149 in os_event_wait_time_low (event=0x559ff0b76580, time_in_usec=10000000, reset_sig_count=0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:485 #4 0x0000559feeae86ca in dict_stats_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/dict/dict0stats_bg.cc:476 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f5cf77fe700 (LWP 13138)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee854377 in os_event::wait (this=0x559ff0909270) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:158 #2 0x0000559fee853e6b in os_event::wait_low (this=0x559ff0909270, reset_sig_count=898) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:325 #3 0x0000559fee85416e in os_event_wait_low (event=0x559ff0909270, reset_sig_count=0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:502 #4 0x0000559fee96c328 in srv_resume_thread (slot=0x559fef966d10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:909 #5 0x0000559fee9705e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f5d31e6f700 (LWP 13067)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559feed9ea6b in safe_cond_timedwait (cond=0x559ff03bb5e0 , mp=0x559ff03bb520 , abstime=0x7f5d31e6ecb0, file=0x559fef206850 "/home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/git/mysys/thr_mutex.c:546 #2 0x0000559feebf6109 in inline_mysql_cond_timedwait (that=0x559ff03bb5e0 , mutex=0x559ff03bb520 , abstime=0x7f5d31e6ecb0, src_file=0x559fef206890 "/home/stepan/mariadb/10.4/git/storage/maria/ma_servicethread.c", src_line=116) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h:1204 #3 0x0000559feebf664d in my_service_thread_sleep (control=0x559fef96d780 , sleep_time=30000000000) at /home/stepan/mariadb/10.4/git/storage/maria/ma_servicethread.c:115 #4 0x0000559feebea50a in ma_checkpoint_background (arg=0x1e) at /home/stepan/mariadb/10.4/git/storage/maria/ma_checkpoint.c:707 #5 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff08c1030) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #6 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f5d0a7fc700 (LWP 13132)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee853d00 in os_event::timed_wait (this=0x559ff0909470, abstime=0x7f5d0a7fbd30) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:275 #2 0x0000559fee853fa9 in os_event::wait_time_low (this=0x559ff0909470, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:385 #3 0x0000559fee854149 in os_event_wait_time_low (event=0x559ff0909470, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:485 #4 0x0000559fee96e20f in srv_monitor_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:1687 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f5cf7fff700 (LWP 13137)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee854377 in os_event::wait (this=0x559ff09092f0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:158 #2 0x0000559fee853e6b in os_event::wait_low (this=0x559ff09092f0, reset_sig_count=893) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:325 #3 0x0000559fee85416e in os_event_wait_low (event=0x559ff09092f0, reset_sig_count=0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:502 #4 0x0000559fee96c328 in srv_resume_thread (slot=0x559fef966e10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:909 #5 0x0000559fee9705e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f5d43f36700 (LWP 12613)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559feed9ea6b in safe_cond_timedwait (cond=0x559ff03cdce0 , mp=0x559ff03cdc20 , abstime=0x7f5d43f35d70, file=0x559fef24ed80 "/home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/git/mysys/thr_mutex.c:546 #2 0x0000559feed9faae in inline_mysql_cond_timedwait (that=0x559ff03cdce0 , mutex=0x559ff03cdc20 , abstime=0x7f5d43f35d70, src_file=0x559fef24edc0 "/home/stepan/mariadb/10.4/git/mysys/thr_timer.c", src_line=292) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h:1204 #3 0x0000559feeda0671 in timer_handler (arg=0x0) at /home/stepan/mariadb/10.4/git/mysys/thr_timer.c:292 #4 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff0788250) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f5d15ffb700 (LWP 13080)): #0 0x00007f5d4b06e644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559fee845dad in LinuxAIOHandler::collect (this=0x7f5d15ffab00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1809 #2 0x0000559fee8463f6 in LinuxAIOHandler::poll (this=0x7f5d15ffab00, m1=0x7f5d15ffaba8, m2=0x7f5d15ffabb0, request=0x7f5d15ffabe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1968 #3 0x0000559fee846559 in os_aio_linux_handler (global_segment=4, m1=0x7f5d15ffaba8, m2=0x7f5d15ffabb0, request=0x7f5d15ffabe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:2022 #4 0x0000559fee84b2c0 in os_aio_handler (segment=4, m1=0x7f5d15ffaba8, m2=0x7f5d15ffabb0, request=0x7f5d15ffabe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:5561 #5 0x0000559feeb014f9 in fil_aio_wait (segment=4) at /home/stepan/mariadb/10.4/git/storage/innobase/fil/fil0fil.cc:4357 #6 0x0000559fee972e42 in io_handler_thread (arg=0x559fefbaa9a0 ) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0start.cc:324 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f5d0affd700 (LWP 13131)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee853d00 in os_event::timed_wait (this=0x559ff09093f0, abstime=0x7f5d0affcbb0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:275 #2 0x0000559fee853fa9 in os_event::wait_time_low (this=0x559ff09093f0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:385 #3 0x0000559fee854149 in os_event_wait_time_low (event=0x559ff09093f0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:485 #4 0x0000559fee96e789 in srv_error_monitor_thread () at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0srv.cc:1851 #5 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f5d30822700 (LWP 14442)): #0 0x00007f5d49821e2d in nanosleep () from /lib64/libc.so.6 #1 0x00007f5d49821cc4 in sleep () from /lib64/libc.so.6 #2 0x0000559fee415642 in wsrep_close_client_connections (wait_to_end=1 '\001', except_caller_thd=0x7f5cbc000b00) at /home/stepan/mariadb/10.4/git/sql/wsrep_mysqld.cc:2344 #3 0x0000559fee40effc in wsrep_stop_replication (thd=0x7f5cbc000b00) at /home/stepan/mariadb/10.4/git/sql/wsrep_mysqld.cc:943 #4 0x0000559fee4218ae in wsrep_cluster_address_update (self=0x559fefb548a0 , thd=0x7f5cbc000b00, type=SHOW_OPT_GLOBAL) at /home/stepan/mariadb/10.4/git/sql/wsrep_var.cc:483 #5 0x0000559fee08a958 in sys_var::update (this=0x559fefb548a0 , thd=0x7f5cbc000b00, var=0x7f5cbc011f60) at /home/stepan/mariadb/10.4/git/sql/set_var.cc:209 #6 0x0000559fee08c6f4 in set_var::update (this=0x7f5cbc011f60, thd=0x7f5cbc000b00) at /home/stepan/mariadb/10.4/git/sql/set_var.cc:837 #7 0x0000559fee08c36e in sql_set_variables (thd=0x7f5cbc000b00, var_list=0x7f5cbc005930, free=true) at /home/stepan/mariadb/10.4/git/sql/set_var.cc:740 #8 0x0000559fee1aa179 in mysql_execute_command (thd=0x7f5cbc000b00) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:4944 #9 0x0000559fee1b47b5 in mysql_parse (thd=0x7f5cbc000b00, rawbuf=0x7f5cbc011df8 "SET GLOBAL wsrep_cluster_address = ''", length=37, parser_state=0x7f5d30821030, is_com_multi=false, is_next_command=false) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:7912 #10 0x0000559fee1b3d68 in wsrep_mysql_parse (thd=0x7f5cbc000b00, rawbuf=0x7f5cbc011df8 "SET GLOBAL wsrep_cluster_address = ''", length=37, parser_state=0x7f5d30821030, is_com_multi=false, is_next_command=false) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:7716 #11 0x0000559fee19fa77 in dispatch_command (command=COM_QUERY, thd=0x7f5cbc000b00, packet=0x7f5cbc008331 "SET GLOBAL wsrep_cluster_address = ''", packet_length=37, is_com_multi=false, is_next_command=false) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:1826 #12 0x0000559fee19e18a in do_command (thd=0x7f5cbc000b00) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:1359 #13 0x0000559fee327d51 in do_handle_one_connection (connect=0x559ff0cff900) at /home/stepan/mariadb/10.4/git/sql/sql_connect.cc:1412 #14 0x0000559fee327a80 in handle_one_connection (arg=0x559ff0cff900) at /home/stepan/mariadb/10.4/git/sql/sql_connect.cc:1316 #15 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff0745ee0) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #16 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #17 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f5d14ff9700 (LWP 13082)): #0 0x00007f5d4b4a1d12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559fee853d00 in os_event::timed_wait (this=0x559ff0909570, abstime=0x7f5d14ff8b00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:275 #2 0x0000559fee853fa9 in os_event::wait_time_low (this=0x559ff0909570, time_in_usec=213000, reset_sig_count=2) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:385 #3 0x0000559fee854149 in os_event_wait_time_low (event=0x559ff0909570, time_in_usec=213000, reset_sig_count=2) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0event.cc:485 #4 0x0000559feea7e358 in pc_sleep_if_needed (next_loop_time=188004048, sig_count=2, cur_time=188003835) at /home/stepan/mariadb/10.4/git/storage/innobase/buf/buf0flu.cc:2643 #5 0x0000559feea7f4fc in buf_flush_page_cleaner_coordinator () at /home/stepan/mariadb/10.4/git/storage/innobase/buf/buf0flu.cc:3102 #6 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f5d40321700 (LWP 12648)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x0000559feed9e769 in safe_cond_wait (cond=0x7f5d24015a68, mp=0x7f5d240159b8, file=0x559feef1aa20 "/home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/mariadb/10.4/git/mysys/thr_mutex.c:492 #2 0x0000559fee4261b4 in inline_mysql_cond_wait (that=0x7f5d24015a68, mutex=0x7f5d240159b8, src_file=0x559feef1ac30 "/home/stepan/mariadb/10.4/git/sql/wsrep_thd.h", src_line=69) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_thread.h:1178 #3 0x0000559fee429416 in Wsrep_thd_queue::pop_front (this=0x7f5d24015960) at /home/stepan/mariadb/10.4/git/sql/wsrep_thd.h:69 #4 0x0000559fee427e23 in wsrep_rollback_process (rollbacker=0x7f5d24001d50, arg=0x559ff0819dc0) at /home/stepan/mariadb/10.4/git/sql/wsrep_thd.cc:266 #5 0x0000559fee416e70 in start_wsrep_THD (arg=0x559ff0819dc0) at /home/stepan/mariadb/10.4/git/sql/wsrep_mysqld.cc:2708 #6 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff07dc8f0) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f5d16ffd700 (LWP 13078)): #0 0x00007f5d4b06e644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x0000559fee845dad in LinuxAIOHandler::collect (this=0x7f5d16ffcb00) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1809 #2 0x0000559fee8463f6 in LinuxAIOHandler::poll (this=0x7f5d16ffcb00, m1=0x7f5d16ffcba8, m2=0x7f5d16ffcbb0, request=0x7f5d16ffcbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:1968 #3 0x0000559fee846559 in os_aio_linux_handler (global_segment=2, m1=0x7f5d16ffcba8, m2=0x7f5d16ffcbb0, request=0x7f5d16ffcbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:2022 #4 0x0000559fee84b2c0 in os_aio_handler (segment=2, m1=0x7f5d16ffcba8, m2=0x7f5d16ffcbb0, request=0x7f5d16ffcbe0) at /home/stepan/mariadb/10.4/git/storage/innobase/os/os0file.cc:5561 #5 0x0000559feeb014f9 in fil_aio_wait (segment=2) at /home/stepan/mariadb/10.4/git/storage/innobase/fil/fil0fil.cc:4357 #6 0x0000559fee972e42 in io_handler_thread (arg=0x559fefbaa990 ) at /home/stepan/mariadb/10.4/git/storage/innobase/srv/srv0start.cc:324 #7 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f5d42fa6700 (LWP 12614)): #0 0x00007f5d4b4a1965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f5d433fcdd7 in gu::Lock::wait (cond=..., this=0x7f5d42fa5bf0) at galerautils/src/gu_lock.hpp:45 #2 0x00007f5d433fce69 in galera::ServiceThd::thd_func (arg=0x559ff07bc698) at galera/src/galera_service_thd.cpp:30 #3 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f5d3086d700 (LWP 13163)): #0 0x00007f5d4b4a29d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000559feed990f6 in my_write_core (sig=6) at /home/stepan/mariadb/10.4/git/mysys/stacktrace.c:481 #2 0x0000559fee505964 in handle_fatal_signal (sig=6) at /home/stepan/mariadb/10.4/git/sql/signal_handler.cc:343 #3 #4 0x00007f5d49793207 in raise () from /lib64/libc.so.6 #5 0x00007f5d497948f8 in abort () from /lib64/libc.so.6 #6 0x00007f5d4978c026 in __assert_fail_base () from /lib64/libc.so.6 #7 0x00007f5d4978c0d2 in __assert_fail () from /lib64/libc.so.6 #8 0x0000559feee07638 in inline_mysql_socket_recv (src_file=0x559fef391f70 "/home/stepan/mariadb/10.4/git/vio/viosocket.c", src_line=190, mysql_socket=..., buf=0x7f5d10028ba0, n=4, flags=64) at /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_socket.h:779 #9 0x0000559feee07d2e in vio_read (vio=0x559ff0c0e6a0, buf=0x7f5d10028ba0 "\001", size=4) at /home/stepan/mariadb/10.4/git/vio/viosocket.c:189 #10 0x0000559fee6861e3 in my_real_read (net=0x7f5d1000a338, complen=0x7f5d3086c810, header=1 '\001') at /home/stepan/mariadb/10.4/git/sql/net_serv.cc:962 #11 0x0000559fee686ae1 in my_net_read_packet_reallen (net=0x7f5d1000a338, read_from_server=1 '\001', reallen=0x7f5d3086c878) at /home/stepan/mariadb/10.4/git/sql/net_serv.cc:1243 #12 0x0000559fee686a94 in my_net_read_packet (net=0x7f5d1000a338, read_from_server=1 '\001') at /home/stepan/mariadb/10.4/git/sql/net_serv.cc:1227 #13 0x0000559fee19d9de in do_command (thd=0x7f5d1000a040) at /home/stepan/mariadb/10.4/git/sql/sql_parse.cc:1232 #14 0x0000559fee327d51 in do_handle_one_connection (connect=0x559ff0cff900) at /home/stepan/mariadb/10.4/git/sql/sql_connect.cc:1412 #15 0x0000559fee327a80 in handle_one_connection (arg=0x559ff0cff900) at /home/stepan/mariadb/10.4/git/sql/sql_connect.cc:1316 #16 0x0000559feed274a5 in pfs_spawn_thread (arg=0x559ff0d01c90) at /home/stepan/mariadb/10.4/git/storage/perfschema/pfs.cc:1862 #17 0x00007f5d4b49ddd5 in start_thread () from /lib64/libpthread.so.0 #18 0x00007f5d4985aead in clone () from /lib64/libc.so.6 Compressed file /home/stepan/mariadb/10.4/git/mysql-test/var/4/log/galera_sr.galera_sr_cc_master-innodb/mysqld.2/data/core.12605 - saving '/home/stepan/mariadb/10.4/git/mysql-test/var/4/log/galera_sr.galera_sr_cc_master-innodb/' to '/home/stepan/mariadb/10.4/git/mysql-test/var/log/galera_sr.galera_sr_cc_master-innodb/' ***Warnings generated in error logs during shutdown after running tests: galera_sr.GCF-627 galera_sr.GCF-900 galera_sr.GCF-889 galera_sr.galera_sr_cc_master galera_sr.galera_sr_blob galera_sr.GCF-851 galera_sr.GCF-867 galera_sr.galera_sr_bf_abort galera_sr.GCF-845 mysqld: /home/stepan/mariadb/10.4/git/include/mysql/psi/mysql_socket.h:779: inline_mysql_socket_recv: Assertion `mysql_socket.fd != -1' failed. Attempting backtrace. You can use the following information to find out 191025 22:28:37 +30.938 galera_sr.galera_sr_cc_no_primary 'innodb' w4 [ pass ] 14889 191025 22:28:42 +5.647 galera_sr.galera_sr_cc_slave 'innodb' w4 [ pass ] 5296 191025 22:28:45 +2.788 galera_sr.galera_sr_concurrent 'innodb' w4 [ pass ] 2495 191025 22:28:48 +2.660 galera_sr.galera_sr_conflict 'innodb' w4 [ pass ] 2309 191025 22:28:50 +2.590 galera_sr.galera_sr_conflict_on_commit 'innodb' w4 [ pass ] 2263 191025 22:28:54 +3.648 galera_sr.galera_sr_conflict_on_commit2 'innodb' w4 [ pass ] 3289 191025 22:28:57 +2.651 galera_sr.galera_sr_conflict_with_rollback_master 'innodb' w4 [ pass ] 2356 191025 22:28:59 +2.768 galera_sr.galera_sr_parallel_apply 'innodb' w4 [ pass ] 2413 191025 22:29:02 +2.989 galera_sr.galera_sr_rollback 'innodb' w4 [ pass ] 2671 191025 22:29:05 +2.623 galera_sr.galera_sr_rollback_retry 'innodb' w4 [ pass ] 2310 191025 22:29:08 +3.026 galera_sr.galera_sr_rollback_savepoint 'innodb' w4 [ pass ] 2673 191025 22:29:11 +2.784 galera_sr.galera_sr_rollback_statement 'innodb' w4 [ pass ] 2484 191025 22:29:21 +10.025 galera_sr.galera_sr_shutdown_master 'innodb' w4 [ pass ] 9731 191025 22:29:31 +10.185 galera_sr.galera_sr_shutdown_slave 'innodb' w4 [ pass ] 9864 191025 22:29:34 +3.213 galera_sr.galera_sr_transaction_replay 'innodb' w4 [ pass ] 2791 191025 22:29:37 +2.513 galera_sr.galera_sr_unit_statements 'innodb' w4 [ pass ] 2237 191025 22:29:40 +3.660 galera_sr.galera_sr_ws_size 'innodb' w4 [ pass ] 3355 191025 22:29:45 +4.920 galera_sr.galera_sr_ws_size2 'innodb' w4 [ pass ] 4506 191025 22:29:48 +2.593 galera_sr.galera_var_ignore_apply_errors_sr 'innodb' w4 [ pass ] 2249 191025 22:29:51 +3.194 galera_sr.mysql-wsrep#215 'innodb' w4 [ pass ] 2872 191025 22:29:54 +2.693 galera_sr.mysql-wsrep-features#138 'innodb' w4 [ pass ] 2355 191025 22:29:56 +2.596 galera_sr.mysql-wsrep-features#14 'innodb' w4 [ pass ] 2248 191025 22:30:00 +3.071 galera_sr.mysql-wsrep-features#148 'innodb' w4 [ pass ] 2810 191025 22:30:02 +2.498 galera_sr.mysql-wsrep-features#15 'innodb' w4 [ pass ] 2222 191025 22:30:06 +3.783 galera_sr.galera_sr_large_fragment 'innodb' w2 [ pass ] 181504 191025 22:30:09 +3.488 galera_sr.mysql-wsrep-features#165 'innodb' w4 [ pass ] 6938 191025 22:30:12 +2.673 galera_sr.mysql-wsrep-features#22 'innodb' w4 [ pass ] 2343 191025 22:30:17 +4.705 galera_sr.mysql-wsrep-features#27 'innodb' w4 [ pass ] 4380 191025 22:30:19 +2.632 galera_sr.mysql-wsrep-features#35 'innodb' w4 [ pass ] 2339 191025 22:30:24 +4.906 galera_sr.mysql-wsrep-features#93 'innodb' w2 [ pass ] 2214 191025 22:30:27 +2.737 galera_sr.mysql-wsrep-features#96 'innodb' w2 [ pass ] 2453 191025 22:30:38 +10.689 galera_sr.MDEV-18585 'innodb' w4 [ pass ] 2299 191025 22:31:02 +23.956 galera_sr.galera_sr_insert_select 'innodb' w3 [ pass ] 183353 191025 22:31:04 +2.337 galera_sr.galera_sr_kill_slave 'innodb' w2 [ pass ] 20891 191025 22:31:04 +0.211 galera_sr.galera_sr_kill_connection 'innodb' w3 [ pass ] 2298 191025 22:31:08 +3.471 galera_sr.galera_sr_kill_query 'innodb' w3 [ pass ] 3079 191025 22:31:12 +4.521 galera_sr.galera_sr_kill_all_norecovery 'innodb' w4 [ pass ] 18411 191025 22:31:15 +2.535 galera_sr.galera_sr_load_data 'innodb' w3 [ pass ] 6659 191025 22:31:17 +2.515 galera_sr.galera_sr_myisam 'innodb' w3 [ pass ] 2247 191025 22:31:36 +18.358 galera_sr.galera_sr_kill_all_nobootstrap 'innodb' w3 [ pass ] 18038 191025 22:31:51 +15.231 galera_sr.galera_sr_kill_all_pcrecovery 'innodb' w3 [ pass ] 14907 191025 22:32:02 +10.917 galera_sr.galera_sr_mysqldump_sst 'innodb' w4 [ pass ] 33596 191025 22:32:05 +3.221 galera_sr.galera_sr_load_data_splitting 'innodb' w3 [ pass ] 13626 191025 22:32:17 +11.771 galera_sr.mdev_18631 'innodb' w4 [ pass ] 2227 191025 22:33:15 +58.160 galera_sr.galera-features#56 'innodb' w1 [ pass ] 336118 191025 22:33:20 +5.010 galera_sr.GCF-1008 'innodb' w1 [ pass ] 4629 191025 22:33:23 +2.702 galera_sr.GCF-1018 'innodb' w1 [ pass ] 2322 191025 22:34:13 +49.992 galera_sr.galera_sr_small_gcache 'innodb' w2 [ pass ] 171805 191025 22:35:35 +82.575 galera_sr.galera_sr_many_fragments 'innodb' w4 [ pass ] 182220 191025 22:35:55 +19.659 galera_sr.mysql-wsrep-features#9 'innodb' w4 [ pass ] 19296 191025 22:35:57 +2.557 galera_sr.GCF-1051 'innodb' w4 [ pass ] 2241 191025 22:36:00 +2.725 galera_sr.GCF-561 'innodb' w4 [ pass ] 2414 191025 22:36:03 +2.752 galera_sr.GCF-571 'innodb' w4 [ pass ] 2398 191025 22:36:03 +0.167 galera_sr.GCF-1043A 'innodb' w1 [ pass ] 160048 191025 22:36:05 +2.473 galera_sr.GCF-572 'innodb' w4 [ pass ] 2240 191025 22:36:08 +2.796 galera_sr.GCF-580 'innodb' w4 [ pass ] 2501 191025 22:36:10 +1.599 galera_sr.GCF-585 'innodb' w1 [ pass ] 6515 191025 22:36:11 +0.960 galera_sr.GCF-597 'innodb' w4 [ pass ] 2289 191025 22:36:13 +1.795 galera_sr.GCF-623 'innodb' w1 [ pass ] 2466 191025 22:36:13 +0.817 galera_sr.GCF-620 'innodb' w4 [ pass ] 2274 191025 22:36:29 +15.164 galera_sr.GCF-1043B 'innodb' w2 [ pass ] 121579 191025 22:36:30 +1.650 galera_sr.mysql-wsrep-features#8 'innodb' w3 [ pass ] 264868 -------------------------------------------------------------------------- The servers were restarted 15 times Spent 1979.767 of 591 seconds executing testcases Completed: Failed 1/84 tests, 98.81% were successful. Failing test(s): galera_sr.galera_sr_cc_master The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html Errors/warnings were found in logfiles during server shutdown after running the following sequence(s) of tests: galera_sr.GCF-627 galera_sr.GCF-900 galera_sr.GCF-889 galera_sr.galera_sr_cc_master galera_sr.galera_sr_blob galera_sr.GCF-851 galera_sr.GCF-867 galera_sr.galera_sr_bf_abort galera_sr.GCF-845 191025 22:36:40 +10.079 mysql-test-run: *** ERROR: there were failing test cases