Checking supported features... MariaDB Version 10.4.9-MariaDB-debug - SSL connections supported - binaries are debug compiled - binaries built with wsrep patch Using suites: galera Collecting tests... Installing system database... worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- 191027 22:58:13 +19.816 galera.GCF-1081 [ disabled ] MDEV-18283 Galera test failure on galera.GCF-1081 191027 22:58:13 +0.000 galera.MW-329 [ disabled ] MDEV-19962 Galera test failure on MW-329 191027 22:58:13 +0.000 galera.MW-360 [ disabled ] needs rewrite to be MariaDB gtid compatible 191027 22:58:13 +0.000 galera.MW-388 [ disabled ] MDEV-19803 Long semaphore wait error on galera.MW-388 191027 22:58:13 +0.000 galera.galera_account_management [ disabled ] MariaDB 10.0 does not support ALTER USER 191027 22:58:13 +0.000 galera.galera_as_master_gtid [ disabled ] Requires MySQL GTID 191027 22:58:13 +0.000 galera.galera_as_master_gtid_change_master [ disabled ] Requires MySQL GTID 191027 22:58:13 +0.000 galera.galera_as_slave_gtid_replicate_do_db_cc [ disabled ] Requires MySQL GTID 191027 22:58:13 +0.000 galera.galera_as_slave_preordered [ disabled ] wsrep-preordered feature not merged to MariaDB 191027 22:58:13 +0.000 galera.galera_as_slave_replication_bundle [ disabled ] MDEV-15785 OPTION_GTID_BEGIN is set in Gtid_log_event::do_apply_event() 191027 22:58:13 +0.000 galera.galera_bf_abort_group_commit [ disabled ] MDEV-18282 Galera test failure on galera.galera_bf_abort_group_commit 191027 22:58:13 +0.000 galera.galera_binlog_rows_query_log_events [ disabled ] MariaDB does not support binlog_rows_query_log_events 191027 22:58:13 +0.000 galera.galera_binlog_stmt_autoinc [ disabled ] MDEV-19959 Galera test failure on galera_binlog_stmt_autoinc 191027 22:58:13 +0.000 galera.galera_concurrent_ctas [ disabled ] MDEV-18180 Galera test failure on galera.galera_concurrent_ctas 191027 22:58:13 +0.000 galera.galera_encrypt_tmp_files [ disabled ] Get error failed to enable encryption of temporary files 191027 22:58:13 +0.000 galera.galera_flush [ disabled ] MariaDB does not have global.thread_statistics 191027 22:58:13 +0.000 galera.galera_gcache_recover_manytrx [ disabled ] MDEV-18834 Galera test failure 191027 22:58:13 +0.000 galera.galera_ist_mariabackup [ disabled ] MDEV-18829 test leaves port open 191027 22:58:13 +0.000 galera.galera_ist_progress [ disabled ] MDEV-15236 fails when trying to read transfer status 191027 22:58:13 +0.000 galera.galera_kill_largechanges [ disabled ] MDEV-18179 Galera test failure on galera.galera_kill_largechanges 191027 22:58:13 +0.000 galera.galera_kill_nochanges [ disabled ] MDEV-18280 Galera test failure on galera_split_brain and galera_kill_nochanges 191027 22:58:13 +0.000 galera.galera_many_tables_nopk [ disabled ] MDEV-18182 Galera test failure on galera.galera_many_tables_nopk 191027 22:58:13 +0.000 galera.galera_migrate [ disabled ] MariaDB does not support START SLAVE USER 191027 22:58:13 +0.000 galera.galera_split_brain [ disabled ] MDEV-18280 Galera test failure on galera_split_brain and galera_kill_nochanges 191027 22:58:13 +0.000 galera.galera_ssl_upgrade [ disabled ] MDEV-19950 Galera test failure on galera_ssl_upgrade 191027 22:58:13 +0.000 galera.galera_var_node_address [ disabled ] MDEV-20485 Galera test failure 191027 22:58:13 +0.000 galera.galera_var_notify_cmd [ disabled ] MDEV-20600 Galera test galera_var_notify_cmd causes hang 191027 22:58:13 +0.000 galera.galera_var_reject_queries [ disabled ] assertion in inline_mysql_socket_send 191027 22:58:13 +0.000 galera.galera_var_retry_autocommit [ disabled ] MDEV-18181 Galera test failure on galera.galera_var_retry_autocommit 191027 22:58:13 +0.000 galera.galera_wan [ disabled ] MDEV-17259 Test failure on galera.galera_wan 191027 22:58:13 +0.000 galera.mysql-wsrep#198 [ disabled ] MDEV-18935 Galera test mysql-wsrep#198 sporaric assertion transaction.cpp:362: int wsrep::transaction::before_commit(): Assertion `state() == s_executing || state() == s_committing || state() == s_must_abort || state() == s_replaying' failed. 191027 22:58:13 +0.000 galera.partition [ disabled ] MDEV-19958 Galera test failure on galera.partition 191027 22:58:13 +0.000 galera.query_cache [ disabled ] MDEV-15805 Test failure on galera.query_cache 191027 22:58:27 +13.712 galera.galera_binlog_checksum 'innodb' [ pass ] 4321 191027 22:58:50 +22.904 galera.MW-313 'innodb' [ pass ] 4212 191027 22:58:54 +4.178 galera.MW-86-wait1 'innodb' [ pass ] 3715 191027 22:59:00 +6.116 galera.MW-86-wait8 'innodb' [ pass ] 5720 191027 22:59:23 +23.154 galera.galera_binlog_event_max_size_min 'innodb' [ pass ] 4448 191027 23:00:53 +90.154 galera.galera_binlog_event_max_size_max 'innodb' [ pass ] 67215 191027 23:01:15 +21.477 galera.enforce_storage_engine2 'innodb' [ pass ] 3857 191027 23:01:38 +23.403 galera.galera_log_output_csv 'innodb' [ pass ] 4656 191027 23:02:03 +24.977 galera.MW-44 'innodb' [ pass ] 6271 191027 23:02:27 +24.449 galera.galera_mdev_13787 'innodb' [ pass ] 4223 191027 23:02:56 +28.637 galera.galera_applier_ftwrl_table_alter 'innodb' [ pass ] 9628 191027 23:03:37 +40.842 galera.galera_bf_background_statistics 'innodb' [ pass ] 20944 191027 23:03:59 +22.084 galera.galera_gtid 'innodb' [ pass ] 3457 191027 23:04:26 +26.672 galera.galera_forced_binlog_format 'innodb' [ pass ] 7988 191027 23:04:49 +23.066 galera.galera_sbr_binlog 'innodb' [ pass ] 3601 191027 23:05:13 +24.122 galera.galera_log_bin 'innodb' [ pass ] 6362 191027 23:05:36 +22.666 galera.galera_v1_row_events 'innodb' [ pass ] 3264 191027 23:05:58 +22.601 galera.galera_var_innodb_disallow_writes 'innodb' [ pass ] 4770 191027 23:06:05 +6.846 galera.mdev_18730 'innodb' [ pass ] 6533 191027 23:06:54 +49.203 galera.galera_flush_local 'innodb' [ pass ] 30410 191027 23:07:17 +22.767 galera.galera_query_cache 'innodb' [ pass ] 5114 191027 23:07:24 +6.700 galera.galera_query_cache_sync_wait 'innodb' [ pass ] 6383 191027 23:09:05 +101.605 galera.mysql-wsrep#201 'innodb' [ pass ] 82542 191027 23:09:27 +21.323 galera.galera_var_auto_inc_control_on 'innodb' [ pass ] 3608 191027 23:09:52 +25.293 galera.galera_sync_wait_upto 'innodb' [ pass ] 6399 191027 23:10:11 +19.465 galera.galera_wsrep_new_cluster 'innodb' [ pass ] 2780 191027 23:10:26 +14.541 galera.galera_wsrep_log_conficts 'innodb' [ pass ] 3800 191027 23:10:46 +19.811 galera.galera_var_sst_auth 'innodb' [ pass ] 2077 191027 23:11:24 +37.824 galera.GAL-419 'innodb' [ pass ] 18954 191027 23:18:44 +440.488 galera.MW-328A 'innodb' [ pass ] 439466 191027 23:27:08 +503.717 galera.MW-328B 'innodb' [ pass ] 501258 191027 23:42:27 +919.664 galera.MW-328C 'innodb' [ pass ] 917973 191027 23:44:20 +112.160 galera.galera_bf_lock_wait 'innodb' [ pass ] 109240 191027 23:45:12 +52.128 galera.galera_fulltext 'innodb' [ pass ] 51521 191027 23:45:18 +6.576 galera.galera_many_columns 'innodb' [ pass ] 5689 191027 23:48:16 +177.428 galera.galera_many_tables_pk 'innodb' [ pass ] 176369 191027 23:48:53 +37.538 galera.galera_parallel_autoinc_largetrx 'innodb' [ pass ] 27025 191027 23:55:11 +378.122 galera.galera_parallel_autoinc_manytrx 'innodb' [ pass ] 358555 191027 23:55:35 +23.910 galera.galera_pc_recovery 'innodb' [ pass ] 15693 191027 23:58:37 +181.656 galera.galera_rsu_add_pk 'innodb' [ pass ] 161271 191028 0:03:32 +295.156 galera.galera_rsu_drop_pk 'innodb' [ pass ] 294898 191028 0:04:30 +58.318 galera.galera_toi_ddl_error 'innodb' [ pass ] 57749 191028 0:05:15 +44.161 galera.galera_toi_ddl_fk_inworker[1] Test still running: galera.galera_gcs_fc_limit worker[1] Test still running: galera.galera_gcs_fc_limit worker[1] Test still running: galera.galera_gcs_fc_limit worker[1] Test still running: galera.galera_gcs_fc_limit worker[1] Trying to dump core for [mysqltest - pid: 2111, winpid: 2111] worker[1] Trying to dump core for [mysqld.1 - pid: 31798, winpid: 31798] worker[1] Trying to dump core for [mysqld.2 - pid: 1655, winpid: 1655] sert 'innodb' [ pass ] 43271 191028 0:05:56 +41.905 galera.galera_toi_ddl_fk_update 'innodb' [ pass ] 41627 191028 0:06:31 +34.327 galera.galera_var_load_data_splitting 'innodb' [ pass ] 32017 191028 0:08:43 +132.031 galera.galera_wsrep_desync_wsrep_on 'innodb' [ pass ] 131261 191028 0:08:49 +6.311 galera.GAL-382 'innodb' [ pass ] 5648 191028 0:09:10 +20.577 galera.GAL-401 'innodb' [ pass ] 20081 191028 0:09:27 +17.039 galera.GAL-480 'innodb' [ pass ] 16448 191028 0:09:31 +4.513 galera.GCF-939 'innodb' [ pass ] 4141 191028 0:09:38 +7.018 galera.MDEV-16509 'innodb' [ pass ] 6440 191028 0:09:47 +8.359 galera.MDEV-20225 'innodb' [ pass ] 7814 191028 0:10:20 +33.425 galera.MW-252 'innodb' [ pass ] 32699 191028 0:10:33 +12.948 galera.MW-258 'innodb' [ pass ] 12520 191028 0:10:39 +5.974 galera.MW-259 'innodb' [ pass ] 5528 191028 0:10:47 +8.343 galera.MW-285 'innodb' [ pass ] 7842 191028 0:10:52 +4.448 galera.MW-286 'innodb' [ pass ] 3912 191028 0:11:01 +9.384 galera.MW-292 'innodb' [ pass ] 8892 191028 0:11:06 +5.344 galera.MW-309 'innodb' [ pass ] 4647 191028 0:11:13 +6.804 galera.MW-328D 'innodb' [ pass ] 6244 191028 0:11:21 +7.385 galera.MW-328E 'innodb' [ pass ] 6904 191028 0:11:25 +4.634 galera.MW-357 'innodb' [ pass ] 4185 191028 0:12:00 +34.838 galera.MW-369 'innodb' [ pass ] 34201 191028 0:12:31 +30.574 galera.MW-402 'innodb' [ pass ] 29933 191028 0:12:34 +3.701 galera.MW-416 'innodb' [ pass ] 3165 191028 0:12:39 +4.738 galera.basic 'innodb' [ pass ] 4197 191028 0:12:44 +4.526 galera.binlog_checksum 'innodb' [ pass ] 3900 191028 0:13:02 +18.466 galera.create 'innodb' [ pass ] 17930 191028 0:13:07 +4.922 galera.enforce_storage_engine 'innodb' [ pass ] 4251 191028 0:13:32 +24.441 galera.ev51914 'innodb' [ pass ] 24059 191028 0:13:43 +11.896 galera.fk 'innodb' [ pass ] 10946 191028 0:14:03 +20.031 galera.galera#500 'innodb' [ pass ] 19521 191028 0:14:06 +2.496 galera.galera#505 'innodb' [ pass ] 2040 191028 0:15:03 +56.803 galera.galera_FK_duplicate_client_insert 'innodb' [ pass ] 56169 191028 0:15:26 +23.652 galera.galera_admin 'innodb' [ pass ] 23063 191028 0:15:34 +7.623 galera.galera_alter_engine_innodb 'innodb' [ pass ] 6880 191028 0:15:39 +5.103 galera.galera_alter_engine_myisam 'innodb' [ pass ] 4602 191028 0:15:46 +6.769 galera.galera_alter_table_force 'innodb' [ pass ] 6197 191028 0:15:51 +4.909 galera.galera_applier_ftwrl_table 'innodb' [ pass ] 4278 191028 0:15:55 +4.118 galera.galera_bf_abort 'innodb' [ pass ] 3562 191028 0:15:59 +4.414 galera.galera_bf_abort_at_after_statement 'innodb' [ pass ] 3964 191028 0:16:04 +4.997 galera.galera_bf_abort_flush_for_export 'innodb' [ pass ] 4535 191028 0:16:09 +4.433 galera.galera_bf_abort_for_update 'innodb' [ pass ] 3888 191028 0:16:13 +4.705 galera.galera_bf_abort_ftwrl 'innodb' [ pass ] 4081 191028 0:16:18 +4.961 galera.galera_bf_abort_get_lock 'innodb' [ pass ] 4515 191028 0:16:23 +4.100 galera.galera_bf_abort_lock_table 'innodb' [ pass ] 3649 191028 0:16:36 +13.679 galera.galera_bf_abort_shutdown 'innodb' [ pass ] 12996 191028 0:16:41 +4.426 galera.galera_bf_abort_sleep 'innodb' [ pass ] 3906 191028 0:16:47 +6.432 galera.galera_binlog_cache_size 'innodb' [ pass ] 5593 191028 0:17:00 +12.866 galera.galera_binlog_row_image 'innodb' [ pass ] 12428 191028 0:17:05 +4.951 galera.galera_change_user 'innodb' [ pass ] 4449 191028 0:17:07 +2.523 galera.galera_commit_empty 'innodb' [ pass ] 2044 191028 0:17:13 +5.156 galera.galera_create_function 'innodb' [ pass ] 4636 191028 0:17:20 +7.121 galera.galera_create_procedure 'innodb' [ pass ] 6635 191028 0:17:47 +27.089 galera.galera_create_table_as_select 'innodb' [ pass ] 26410 191028 0:17:58 +10.988 galera.galera_create_table_like 'innodb' [ pass ] 10526 191028 0:18:10 +12.211 galera.galera_create_trigger 'innodb' [ pass ] 11723 191028 0:18:21 +11.288 galera.galera_ddl_multiline 'innodb' [ pass ] 10795 191028 0:18:24 +2.589 galera.galera_defaults 'innodb' [ pass ] 2041 191028 0:18:32 +8.300 galera.galera_delete_limit 'innodb' [ pass ] 7792 191028 0:18:42 +9.355 galera.galera_desync_overlapped 'innodb' [ pass ] 7699 191028 0:19:24 +42.110 galera.galera_drop_database 'innodb' [ pass ] 41707 191028 0:19:30 +6.248 galera.galera_drop_multi 'innodb' [ pass ] 5661 191028 0:19:36 +6.306 galera.galera_enum 'innodb' [ pass ] 5824 191028 0:19:41 +5.011 galera.galera_events 'innodb' [ pass ] 4494 191028 0:20:06 +25.300 galera.galera_events2 'innodb' [ pass ] 24717 191028 0:20:16 +9.077 galera.galera_fk_cascade_delete 'innodb' [ pass ] 8455 191028 0:20:24 +8.879 galera.galera_fk_cascade_update 'innodb' [ pass ] 8453 191028 0:20:31 +6.633 galera.galera_fk_conflict 'innodb' [ pass ] 6088 191028 0:20:38 +6.938 galera.galera_fk_mismatch 'innodb' [ pass ] 6431 191028 0:20:47 +9.081 galera.galera_fk_multicolumn 'innodb' [ pass ] 8625 191028 0:20:55 +7.892 galera.galera_fk_multitable 'innodb' [ pass ] 7375 191028 0:21:02 +7.204 galera.galera_fk_no_pk 'innodb' [ pass ] 6822 191028 0:21:07 +4.992 galera.galera_fk_selfreferential 'innodb' [ pass ] 4503 191028 0:21:14 +6.846 galera.galera_fk_setnull 'innodb' [ pass ] 6386 191028 0:21:20 +6.021 galera.galera_ftwrl 'innodb' [ pass ] 5607 191028 0:21:26 +6.245 galera.galera_ftwrl_drain 'innodb' [ pass ] 5765 191028 1:52:29 +5462.431 galera.galera_gcs_fc_limit 'innodb' [ fail ] timeout after 5400 seconds Test ended at 2019-10-28 01:52:29 Test case timeout after 5400 seconds == /home/stepan/mariadb/10.4/github/mysql-test/var/log/galera_gcs_fc_limit.log == connection node_2; connection node_1; CREATE TABLE t1 (f1 INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT,B INTEGER) ENGINE=InnoDB; INSERT INTO t1 VALUES (1,1); connection node_2; SET SESSION wsrep_sync_wait=15; SELECT COUNT(*) FROM t1; COUNT(*) 1 SET GLOBAL wsrep_provider_options = 'gcs.fc_limit=1'; LOCK TABLE t1 WRITE; connection node_1; FLUSH STATUS; INSERT INTO t1 VALUES (2,2); INSERT INTO t1 VALUES (3,3); INSERT INTO t1 VALUES (4,4); == /home/stepan/mariadb/10.4/github/mysql-test/var/tmp/analyze-timeout-mysqld.2.err == mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/home/stepan/mariadb/10.4/github/mysql-test/var/tmp/mysqld.2.soc' (111) == /home/stepan/mariadb/10.4/github/mysql-test/var/tmp/analyze-timeout-mysqld.1.err == mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/home/stepan/mariadb/10.4/github/mysql-test/var/tmp/mysqld.1.soc' (111) - found 'core.31799' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace from coredump /home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_gcs_fc_limit-innodb/mysqld.1/data/core.31799 Core generated by '/home/stepan/mariadb/10.4/github/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 31799] [New LWP 31804] [New LWP 31801] [New LWP 31806] [New LWP 31803] [New LWP 31805] [New LWP 31810] [New LWP 31812] [New LWP 31807] [New LWP 31821] [New LWP 31813] [New LWP 31822] [New LWP 31815] [New LWP 31828] [New LWP 31816] [New LWP 31802] [New LWP 31818] [New LWP 31823] [New LWP 31824] [New LWP 31832] [New LWP 31833] [New LWP 31827] [New LWP 31826] [New LWP 32066] [New LWP 31811] [New LWP 31817] [New LWP 31829] [New LWP 31825] [New LWP 31809] [New LWP 31808] [New LWP 31819] [New LWP 31800] [New LWP 31820] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/home/stepan/mariadb/10.4/github/sql/mysqld --defaults-group-suffix=.1 --defaul'. Program terminated with signal 6, Aborted. #0 0x00007fd30ed4f9d1 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007fd30ed4f9d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x000055612cc300f6 in my_write_core (sig=6) at /home/stepan/mariadb/10.4/github/mysys/stacktrace.c:481 #2 0x000055612c39c964 in handle_fatal_signal (sig=6) at /home/stepan/mariadb/10.4/github/sql/signal_handler.cc:343 #3 #4 0x00007fd30d0fd20d in poll () from /lib64/libc.so.6 #5 0x000055612bf012db in handle_connections_sockets () at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:6455 #6 0x000055612bf00576 in mysqld_main (argc=162, argv=0x55612f220130) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:5891 #7 0x000055612bef4095 in main (argc=23, argv=0x7ffe548a0e98) at /home/stepan/mariadb/10.4/github/sql/main.cc:25 Thread 33 (Thread 0x7fd2dbfff700 (LWP 31820)): #0 0x00007fd30ed51d1d in fsync () from /lib64/libpthread.so.0 #1 0x000055612c6de5a7 in os_file_fsync_posix (file=19) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2366 #2 0x000055612c6de8e2 in os_file_flush_func (file=19) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2459 #3 0x000055612c9887f5 in pfs_os_file_flush_func (file=..., src_file=0x55612d054420 "/home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc", src_line=751) at /home/stepan/mariadb/10.4/github/storage/innobase/include/os0file.ic:406 #4 0x000055612c98b186 in fil_flush_low (space=0x55612f746cc0, metadata=false) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:751 #5 0x000055612c998b0f in fil_flush (space_id=4294967280) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4462 #6 0x000055612c6b13f8 in log_write_flush_to_disk_low () at /home/stepan/mariadb/10.4/github/storage/innobase/log/log0log.cc:809 #7 0x000055612c6b1ede in log_write_up_to (lsn=97593560, flush_to_disk=true, rotate_key=false) at /home/stepan/mariadb/10.4/github/storage/innobase/log/log0log.cc:1046 #8 0x000055612c6b2046 in log_buffer_sync_in_background (flush=true) at /home/stepan/mariadb/10.4/github/storage/innobase/log/log0log.cc:1091 #9 0x000055612c805c7f in srv_sync_log_buffer_in_background () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:1997 #10 0x000055612c80628c in srv_master_do_active_tasks () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2156 #11 0x000055612c806f93 in srv_master_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2381 #12 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #13 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 32 (Thread 0x7fd3077e3700 (LWP 31800)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35a6b in safe_cond_timedwait (cond=0x55612e266ce0 , mp=0x55612e266c20 , abstime=0x7fd3077e2d70, file=0x55612d0e8450 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x000055612cc36aae in inline_mysql_cond_timedwait (that=0x55612e266ce0 , mutex=0x55612e266c20 , abstime=0x7fd3077e2d70, src_file=0x55612d0e8498 "/home/stepan/mariadb/10.4/github/mysys/thr_timer.c", src_line=292) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x000055612cc37671 in timer_handler (arg=0x0) at /home/stepan/mariadb/10.4/github/mysys/thr_timer.c:292 #4 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f3661c0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 31 (Thread 0x7fd2d97fa700 (LWP 31819)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6ead00 in os_event::timed_wait (this=0x55612f676660, abstime=0x7fd2d97f9b70) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x000055612c6eafa9 in os_event::wait_time_low (this=0x55612f676660, time_in_usec=5000000, reset_sig_count=7) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x000055612c6eb149 in os_event_wait_time_low (event=0x55612f676660, time_in_usec=5000000, reset_sig_count=7) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x000055612c873563 in ib_wqueue_timedwait (wq=0x55612f66fa70, wait_in_usecs=5000000) at /home/stepan/mariadb/10.4/github/storage/innobase/ut/ut0wqueue.cc:163 #5 0x000055612c9e623d in fts_optimize_thread (arg=0x55612f66fa70) at /home/stepan/mariadb/10.4/github/storage/innobase/fts/fts0opt.cc:2865 #6 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 30 (Thread 0x7fd2e77fe700 (LWP 31808)): #0 0x00007fd30e91b644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x000055612c6dcdad in LinuxAIOHandler::collect (this=0x7fd2e77fdb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x000055612c6dd3f6 in LinuxAIOHandler::poll (this=0x7fd2e77fdb00, m1=0x7fd2e77fdba8, m2=0x7fd2e77fdbb0, request=0x7fd2e77fdbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x000055612c6dd559 in os_aio_linux_handler (global_segment=1, m1=0x7fd2e77fdba8, m2=0x7fd2e77fdbb0, request=0x7fd2e77fdbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x000055612c6e22c0 in os_aio_handler (segment=1, m1=0x7fd2e77fdba8, m2=0x7fd2e77fdbb0, request=0x7fd2e77fdbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x000055612c9984f9 in fil_aio_wait (segment=1) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x000055612c809e42 in io_handler_thread (arg=0x55612da43988 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7fd2e6ffd700 (LWP 31809)): #0 0x00007fd30e91b644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x000055612c6dcdad in LinuxAIOHandler::collect (this=0x7fd2e6ffcb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x000055612c6dd3f6 in LinuxAIOHandler::poll (this=0x7fd2e6ffcb00, m1=0x7fd2e6ffcba8, m2=0x7fd2e6ffcbb0, request=0x7fd2e6ffcbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x000055612c6dd559 in os_aio_linux_handler (global_segment=2, m1=0x7fd2e6ffcba8, m2=0x7fd2e6ffcbb0, request=0x7fd2e6ffcbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x000055612c6e22c0 in os_aio_handler (segment=2, m1=0x7fd2e6ffcba8, m2=0x7fd2e6ffcbb0, request=0x7fd2e6ffcbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x000055612c9984f9 in fil_aio_wait (segment=2) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x000055612c809e42 in io_handler_thread (arg=0x55612da43990 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7fd2caffd700 (LWP 31825)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f4da4c0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f4da4c0, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f4da4c0, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c908e86 in buf_dump_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0dump.cc:833 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7fd3040bb700 (LWP 31829)): #0 0x00007fd30d04140a in sigwaitinfo () from /lib64/libc.so.6 #1 0x000055612bef4114 in my_sigwait (set=0x7fd3040bacf0, sig=0x7fd3040bacb4, code=0x7fd3040bacb8) at /home/stepan/mariadb/10.4/github/include/my_pthread.h:198 #2 0x000055612befaa8d in signal_hand (arg=0x0) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:3243 #3 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f7b8770) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #4 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7fd2da7fc700 (LWP 31817)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6ead00 in os_event::timed_wait (this=0x55612f4da440, abstime=0x7fd2da7fbd30) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x000055612c6eafa9 in os_event::wait_time_low (this=0x55612f4da440, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x000055612c6eb149 in os_event_wait_time_low (event=0x55612f4da440, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x000055612c80520f in srv_monitor_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:1687 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7fd2e5ffb700 (LWP 31811)): #0 0x00007fd30e91b644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x000055612c6dcdad in LinuxAIOHandler::collect (this=0x7fd2e5ffab00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x000055612c6dd3f6 in LinuxAIOHandler::poll (this=0x7fd2e5ffab00, m1=0x7fd2e5ffaba8, m2=0x7fd2e5ffabb0, request=0x7fd2e5ffabe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x000055612c6dd559 in os_aio_linux_handler (global_segment=4, m1=0x7fd2e5ffaba8, m2=0x7fd2e5ffabb0, request=0x7fd2e5ffabe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x000055612c6e22c0 in os_aio_handler (segment=4, m1=0x7fd2e5ffaba8, m2=0x7fd2e5ffabb0, request=0x7fd2e5ffabe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x000055612c9984f9 in fil_aio_wait (segment=4) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x000055612c809e42 in io_handler_thread (arg=0x55612da439a0 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 24 (Thread 0x7fd2fc6a3700 (LWP 32066)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35a6b in safe_cond_timedwait (cond=0x55612d9c5060 , mp=0x55612d9c7a20 , abstime=0x7fd2fc6a2c60, file=0x55612ccf3d68 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x000055612bef4a15 in inline_mysql_cond_timedwait (that=0x55612d9c5060 , mutex=0x55612d9c7a20 , abstime=0x7fd2fc6a2c60, src_file=0x55612ccf5db8 "/home/stepan/mariadb/10.4/github/sql/mysqld.cc", src_line=2686) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x000055612bef9b2c in cache_thread (thd=0x7fd298000b00) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:2685 #4 0x000055612bef9dd2 in one_thread_per_connection_end (thd=0x7fd298000b00, put_in_cache=true) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:2771 #5 0x000055612c1bedcf in do_handle_one_connection (connect=0x55612f7d4510) at /home/stepan/mariadb/10.4/github/sql/sql_connect.cc:1423 #6 0x000055612c1bea80 in handle_one_connection (arg=0x55612f7d4510) at /home/stepan/mariadb/10.4/github/sql/sql_connect.cc:1316 #7 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f323ee0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #8 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #9 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7fd2ca7fc700 (LWP 31826)): #0 0x00007fd30ed51e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x000055612c6eb6b4 in os_thread_sleep (tm=1000000) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0thread.cc:225 #2 0x000055612c8dddd3 in btr_defragment_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/btr/btr0defragment.cc:705 #3 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7fd2c9ffb700 (LWP 31827)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f4da5c0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f4da5c0, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f4da5c0, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c8ee7d9 in buf_resize_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0buf.cc:3199 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7fd2fc6ee700 (LWP 31833)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35a6b in safe_cond_timedwait (cond=0x55612d9c5060 , mp=0x55612d9c7a20 , abstime=0x7fd2fc6edc60, file=0x55612ccf3d68 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x000055612bef4a15 in inline_mysql_cond_timedwait (that=0x55612d9c5060 , mutex=0x55612d9c7a20 , abstime=0x7fd2fc6edc60, src_file=0x55612ccf5db8 "/home/stepan/mariadb/10.4/github/sql/mysqld.cc", src_line=2686) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x000055612bef9b2c in cache_thread (thd=0x7fd2a0000b00) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:2685 #4 0x000055612bef9dd2 in one_thread_per_connection_end (thd=0x7fd2a0000b00, put_in_cache=true) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:2771 #5 0x000055612c1bedcf in do_handle_one_connection (connect=0x55612f8b7a00) at /home/stepan/mariadb/10.4/github/sql/sql_connect.cc:1423 #6 0x000055612c1bea80 in handle_one_connection (arg=0x55612f8b7a00) at /home/stepan/mariadb/10.4/github/sql/sql_connect.cc:1316 #7 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f8d8f40) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #8 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #9 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7fd304070700 (LWP 31832)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35769 in safe_cond_wait (cond=0x55612d9c50a0 , mp=0x55612d9c8320 , file=0x55612cd040c0 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1167) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:492 #2 0x000055612bf2708e in inline_mysql_cond_wait (that=0x55612d9c50a0 , mutex=0x55612d9c8320 , src_file=0x55612cd04c20 "/home/stepan/mariadb/10.4/github/sql/slave.cc", src_line=519) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1167 #3 0x000055612bf289cb in handle_slave_background (arg=0x0) at /home/stepan/mariadb/10.4/github/sql/slave.cc:519 #4 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f8dde50) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7fd2cb7fe700 (LWP 31824)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f4da2c0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f4da2c0, reset_sig_count=3439) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f4da2c0, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c803328 in srv_resume_thread (slot=0x55612d7ffe10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x000055612c8075e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7fd2c3fff700 (LWP 31823)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f4da240) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f4da240, reset_sig_count=4258) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f4da240, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c803328 in srv_resume_thread (slot=0x55612d7ffd10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x000055612c8075e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7fd2d9ffb700 (LWP 31818)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6ead00 in os_event::timed_wait (this=0x55612f7473a0, abstime=0x7fd2d9ffad70) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x000055612c6eafa9 in os_event::wait_time_low (this=0x55612f7473a0, time_in_usec=10000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x000055612c6eb149 in os_event_wait_time_low (event=0x55612f7473a0, time_in_usec=10000000, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x000055612c97f6ca in dict_stats_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/dict/dict0stats_bg.cc:476 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7fd2ff67e700 (LWP 31802)): #0 0x00007fd30d108483 in epoll_wait () from /lib64/libc.so.6 #1 0x00007fd306c4a548 in asio::detail::epoll_reactor::run (this=0x55612f3d2c30, block=block@entry=true, ops=...) at asio/asio/detail/impl/epoll_reactor.ipp:391 #2 0x00007fd306c4ec40 in asio::detail::task_io_service::do_run_one (this=this@entry=0x55612f3d2ad0, lock=..., this_thread=..., ec=...) at asio/asio/detail/impl/task_io_service.ipp:355 #3 0x00007fd306c4eef2 in asio::detail::task_io_service::run (this=0x55612f3d2ad0, ec=...) at asio/asio/detail/impl/task_io_service.ipp:148 #4 0x00007fd306c6697d in asio::io_service::run (this=0x55612f3d27e0) at asio/asio/impl/io_service.ipp:58 #5 0x00007fd306c670d1 in gcomm::AsioProtonet::event_loop (this=0x55612f3d2740, period=...) at gcomm/src/asio_protonet.cpp:116 #6 0x00007fd306c85f3d in GCommConn::run (this=0x55612f3d2130) at gcs/src/gcs_gcomm.cpp:498 #7 0x00007fd306c88fe9 in GCommConn::run_fn (arg=) at gcs/src/gcs_gcomm.cpp:209 #8 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #9 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7fd2daffd700 (LWP 31816)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6ead00 in os_event::timed_wait (this=0x55612f4da3c0, abstime=0x7fd2daffcbb0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x000055612c6eafa9 in os_event::wait_time_low (this=0x55612f4da3c0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x000055612c6eb149 in os_event_wait_time_low (event=0x55612f4da3c0, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x000055612c805789 in srv_error_monitor_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:1851 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7fd2c97fa700 (LWP 31828)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35769 in safe_cond_wait (cond=0x7fd2c97f9c70, mp=0x7fd2c97f9cb0, file=0x55612cf22ce0 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:492 #2 0x000055612c5de15d in inline_mysql_cond_wait (that=0x7fd2c97f9c70, mutex=0x7fd2c97f9cb0, src_file=0x55612cf247f8 "/home/stepan/mariadb/10.4/github/storage/innobase/handler/ha_innodb.cc", src_line=288) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1178 #3 0x000055612c5e3a55 in thd_destructor_proxy () at /home/stepan/mariadb/10.4/github/storage/innobase/handler/ha_innodb.cc:288 #4 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f782c90) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7fd2db7fe700 (LWP 31815)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6ead00 in os_event::timed_wait (this=0x55612f715380, abstime=0x7fd2db7fdd50) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x000055612c6eafa9 in os_event::wait_time_low (this=0x55612f715380, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x000055612c6eb149 in os_event_wait_time_low (event=0x55612f715380, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x000055612c6ad527 in lock_wait_timeout_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/lock/lock0wait.cc:522 #5 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7fd2cbfff700 (LWP 31822)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f4da340) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f4da340, reset_sig_count=3233) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f4da340, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c803328 in srv_resume_thread (slot=0x55612d7fff10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x000055612c8075e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7fd2e4ff9700 (LWP 31813)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f66eb90) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f66eb90, reset_sig_count=3795) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f66eb90, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c6b1a72 in log_write_up_to (lsn=97593560, flush_to_disk=true, rotate_key=false) at /home/stepan/mariadb/10.4/github/storage/innobase/log/log0log.cc:924 #5 0x000055612c90f048 in buf_flush_write_block_low (bpage=0x7fd2fc90de00, flush_type=BUF_FLUSH_LIST, sync=false) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:999 #6 0x000055612c90fd4b in buf_flush_page (buf_pool=0x55612f5c02b0, bpage=0x7fd2fc90de00, flush_type=BUF_FLUSH_LIST, sync=false) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:1210 #7 0x000055612c910824 in buf_flush_try_neighbors (page_id=..., flush_type=BUF_FLUSH_LIST, n_flushed=2, n_to_flush=200) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:1437 #8 0x000055612c910bde in buf_flush_page_and_try_neighbors (bpage=0x7fd2fc90de00, flush_type=BUF_FLUSH_LIST, n_to_flush=200, count=0x7fd2e4ff88b8) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:1511 #9 0x000055612c9119e0 in buf_do_flush_list_batch (buf_pool=0x55612f5c02b0, min_n=200, lsn_limit=18446744073709551615) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:1769 #10 0x000055612c9120bf in buf_flush_batch (buf_pool=0x55612f5c02b0, flush_type=BUF_FLUSH_LIST, min_n=200, lsn_limit=18446744073709551615, n=0x7fd2e4ff8b60) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:1839 #11 0x000055612c912886 in buf_flush_do_batch (buf_pool=0x55612f5c02b0, type=BUF_FLUSH_LIST, min_n=200, lsn_limit=18446744073709551615, n=0x7fd2e4ff8b60) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:1997 #12 0x000055612c912b96 in buf_flush_lists (min_n=200, lsn_limit=18446744073709551615, n_processed=0x7fd2e4ff8bd0) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:2103 #13 0x000055612c916e17 in buf_flush_page_cleaner_coordinator () at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:3255 #14 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #15 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7fd2d8ff9700 (LWP 31821)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612c6eb377 in os_event::wait (this=0x55612f4da1c0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x000055612c6eae6b in os_event::wait_low (this=0x55612f4da1c0, reset_sig_count=1154) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x000055612c6eb16e in os_event_wait_low (event=0x55612f4da1c0, reset_sig_count=1154) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x000055612c803328 in srv_resume_thread (slot=0x55612d7ffc10 , sig_count=1154, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x000055612c807aa0 in srv_purge_coordinator_suspend (slot=0x55612d7ffc10 , rseg_history_len=33) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2657 #6 0x000055612c807dd1 in srv_purge_coordinator_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2738 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7fd2e7fff700 (LWP 31807)): #0 0x00007fd30e91b644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x000055612c6dcdad in LinuxAIOHandler::collect (this=0x7fd2e7ffeb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x000055612c6dd3f6 in LinuxAIOHandler::poll (this=0x7fd2e7ffeb00, m1=0x7fd2e7ffeba8, m2=0x7fd2e7ffebb0, request=0x7fd2e7ffebe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x000055612c6dd559 in os_aio_linux_handler (global_segment=0, m1=0x7fd2e7ffeba8, m2=0x7fd2e7ffebb0, request=0x7fd2e7ffebe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x000055612c6e22c0 in os_aio_handler (segment=0, m1=0x7fd2e7ffeba8, m2=0x7fd2e7ffebb0, request=0x7fd2e7ffebe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x000055612c9984f9 in fil_aio_wait (segment=0) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x000055612c809e42 in io_handler_thread (arg=0x55612da43980 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7fd2e57fa700 (LWP 31812)): #0 0x00007fd30e91b644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x000055612c6dcdad in LinuxAIOHandler::collect (this=0x7fd2e57f9b00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x000055612c6dd3f6 in LinuxAIOHandler::poll (this=0x7fd2e57f9b00, m1=0x7fd2e57f9ba8, m2=0x7fd2e57f9bb0, request=0x7fd2e57f9be0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x000055612c6dd559 in os_aio_linux_handler (global_segment=5, m1=0x7fd2e57f9ba8, m2=0x7fd2e57f9bb0, request=0x7fd2e57f9be0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x000055612c6e22c0 in os_aio_handler (segment=5, m1=0x7fd2e57f9ba8, m2=0x7fd2e57f9bb0, request=0x7fd2e57f9be0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x000055612c9984f9 in fil_aio_wait (segment=5) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x000055612c809e42 in io_handler_thread (arg=0x55612da439a8 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7fd2e67fc700 (LWP 31810)): #0 0x00007fd30e91b644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x000055612c6dcdad in LinuxAIOHandler::collect (this=0x7fd2e67fbb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x000055612c6dd3f6 in LinuxAIOHandler::poll (this=0x7fd2e67fbb00, m1=0x7fd2e67fbba8, m2=0x7fd2e67fbbb0, request=0x7fd2e67fbbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x000055612c6dd559 in os_aio_linux_handler (global_segment=3, m1=0x7fd2e67fbba8, m2=0x7fd2e67fbbb0, request=0x7fd2e67fbbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x000055612c6e22c0 in os_aio_handler (segment=3, m1=0x7fd2e67fbba8, m2=0x7fd2e67fbbb0, request=0x7fd2e67fbbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x000055612c9984f9 in fil_aio_wait (segment=3) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x000055612c809e42 in io_handler_thread (arg=0x55612da43998 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7fd3042e3700 (LWP 31805)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fd306b9be42 in fifo_lock_get (q=q@entry=0x7fd305f33010) at galerautils/src/gu_fifo.c:262 #2 0x00007fd306b9c4b1 in gu_fifo_get_head (q=0x7fd305f33010, err=0x7fd3042e201c) at galerautils/src/gu_fifo.c:330 #3 0x00007fd306c7fd1c in gcs_recv (conn=0x55612f3a2280, action=0x7fd3042e2060) at gcs/src/gcs.cpp:2024 #4 0x00007fd306cacf6c in galera::GcsActionSource::process (this=0x55612f3a1c60, recv_ctx=0x7fd3042e2480, exit_loop=@0x7fd3042e224e: false) at galera/src/gcs_action_source.cpp:161 #5 0x00007fd306ccbf24 in galera::ReplicatorSMM::async_recv (this=0x55612f399fa0, recv_ctx=0x7fd3042e2480) at galera/src/replicator_smm.cpp:391 #6 0x000055612ccebe7a in wsrep::wsrep_provider_v26::run_applier (this=0x55612f3241f0, applier_ctx=0x7fd3042e2480) at /home/stepan/mariadb/10.4/github/wsrep-lib/src/wsrep_provider_v26.cpp:727 #7 0x000055612c2bd787 in wsrep_replication_process (thd=0x7fd2e8000b00, arg=0x55612f3ebb60) at /home/stepan/mariadb/10.4/github/sql/wsrep_thd.cc:62 #8 0x000055612c2ade70 in start_wsrep_THD (arg=0x55612f3ebb60) at /home/stepan/mariadb/10.4/github/sql/wsrep_mysqld.cc:2708 #9 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f3eabf0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #10 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #11 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7fd2fee7d700 (LWP 31803)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fd306c844b7 in gu::Lock::wait (cond=..., this=0x7fd2fee7c850) at galerautils/src/gu_lock.hpp:45 #2 0x00007fd306c86ffb in RecvBuf::front (this=this@entry=0x55612f3d2318, timeout=...) at gcs/src/gcs_gcomm.cpp:124 #3 0x00007fd306c84b96 in gcomm_recv (backend=, msg=0x55612f3a2560, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:673 #4 0x00007fd306c78b2c in core_msg_recv (backend=backend@entry=0x55612f3a26c0, recv_msg=recv_msg@entry=0x55612f3a2560, timeout=timeout@entry=9223372035999999999) at gcs/src/gcs_core.cpp:480 #5 0x00007fd306c7bda4 in gcs_core_recv (conn=0x55612f3a2500, recv_act=recv_act@entry=0x7fd2fee7cd70, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1184 #6 0x00007fd306c83028 in gcs_recv_thread (arg=0x55612f3a2280) at gcs/src/gcs.cpp:1398 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7fd2fdcf0700 (LWP 31806)): #0 0x00007fd30ed4ed12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35a6b in safe_cond_timedwait (cond=0x55612e2545e0 , mp=0x55612e254520 , abstime=0x7fd2fdcefcb0, file=0x55612d09f880 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x000055612ca8d109 in inline_mysql_cond_timedwait (that=0x55612e2545e0 , mutex=0x55612e254520 , abstime=0x7fd2fdcefcb0, src_file=0x55612d09f8c8 "/home/stepan/mariadb/10.4/github/storage/maria/ma_servicethread.c", src_line=116) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x000055612ca8d64d in my_service_thread_sleep (control=0x55612d806780 , sleep_time=30000000000) at /home/stepan/mariadb/10.4/github/storage/maria/ma_servicethread.c:115 #4 0x000055612ca8150a in ma_checkpoint_background (arg=0x1e) at /home/stepan/mariadb/10.4/github/storage/maria/ma_checkpoint.c:707 #5 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f4922d0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #6 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7fd305e31700 (LWP 31801)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007fd306ca9dd7 in gu::Lock::wait (cond=..., this=0x7fd305e30bf0) at galerautils/src/gu_lock.hpp:45 #2 0x00007fd306ca9e69 in galera::ServiceThd::thd_func (arg=0x55612f39a6b8) at galera/src/galera_service_thd.cpp:30 #3 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7fd30432e700 (LWP 31804)): #0 0x00007fd30ed4e965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x000055612cc35769 in safe_cond_wait (cond=0x7fd2f4015a68, mp=0x7fd2f40159b8, file=0x55612cdb22c0 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:492 #2 0x000055612c2bd1b4 in inline_mysql_cond_wait (that=0x7fd2f4015a68, mutex=0x7fd2f40159b8, src_file=0x55612cdb24e0 "/home/stepan/mariadb/10.4/github/sql/wsrep_thd.h", src_line=69) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1178 #3 0x000055612c2c0416 in Wsrep_thd_queue::pop_front (this=0x7fd2f4015960) at /home/stepan/mariadb/10.4/github/sql/wsrep_thd.h:69 #4 0x000055612c2bee23 in wsrep_rollback_process (rollbacker=0x7fd2f4001d50, arg=0x55612f3e20b0) at /home/stepan/mariadb/10.4/github/sql/wsrep_thd.cc:266 #5 0x000055612c2ade70 in start_wsrep_THD (arg=0x55612f3e20b0) at /home/stepan/mariadb/10.4/github/sql/wsrep_mysqld.cc:2708 #6 0x000055612cbbe4a5 in pfs_spawn_thread (arg=0x55612f3eb960) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #7 0x00007fd30ed4add5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007fd30d107ead in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7fd30f15b880 (LWP 31799)): #0 0x00007fd30ed4f9d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x000055612cc300f6 in my_write_core (sig=6) at /home/stepan/mariadb/10.4/github/mysys/stacktrace.c:481 #2 0x000055612c39c964 in handle_fatal_signal (sig=6) at /home/stepan/mariadb/10.4/github/sql/signal_handler.cc:343 #3 #4 0x00007fd30d0fd20d in poll () from /lib64/libc.so.6 #5 0x000055612bf012db in handle_connections_sockets () at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:6455 #6 0x000055612bf00576 in mysqld_main (argc=162, argv=0x55612f220130) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:5891 #7 0x000055612bef4095 in main (argc=23, argv=0x7ffe548a0e98) at /home/stepan/mariadb/10.4/github/sql/main.cc:25 Compressed file /home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_gcs_fc_limit-innodb/mysqld.1/data/core.31799 - found 'core.1656' (1/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace from coredump /home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_gcs_fc_limit-innodb/mysqld.2/data/core.1656 Core generated by '/home/stepan/mariadb/10.4/github/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 1656] [New LWP 1861] [New LWP 1657] [New LWP 1658] [New LWP 1660] [New LWP 1659] [New LWP 1661] [New LWP 1864] [New LWP 1662] [New LWP 1874] [New LWP 1872] [New LWP 1867] [New LWP 1875] [New LWP 1883] [New LWP 1868] [New LWP 1877] [New LWP 1885] [New LWP 1876] [New LWP 1881] [New LWP 1880] [New LWP 1862] [New LWP 1884] [New LWP 1886] [New LWP 1866] [New LWP 2094] [New LWP 1882] [New LWP 1865] [New LWP 1879] [New LWP 1871] [New LWP 1878] [New LWP 1863] [New LWP 1873] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/home/stepan/mariadb/10.4/github/sql/mysqld --defaults-group-suffix=.2 --defaul'. Program terminated with signal 6, Aborted. #0 0x00007ff9375809d1 in pthread_kill () from /lib64/libpthread.so.0 #0 0x00007ff9375809d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x00005650e56980f6 in my_write_core (sig=6) at /home/stepan/mariadb/10.4/github/mysys/stacktrace.c:481 #2 0x00005650e4e04964 in handle_fatal_signal (sig=6) at /home/stepan/mariadb/10.4/github/sql/signal_handler.cc:343 #3 #4 0x00007ff93592e20d in poll () from /lib64/libc.so.6 #5 0x00005650e49692db in handle_connections_sockets () at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:6455 #6 0x00005650e4968576 in mysqld_main (argc=163, argv=0x5650e6dcf130) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:5891 #7 0x00005650e495c095 in main (argc=23, argv=0x7ffe5325cce8) at /home/stepan/mariadb/10.4/github/sql/main.cc:25 Thread 32 (Thread 0x7ff8f2ffd700 (LWP 1873)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5152d00 in os_event::timed_wait (this=0x5650e6fd7fd0, abstime=0x7ff8f2ffcd30) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x00005650e5152fa9 in os_event::wait_time_low (this=0x5650e6fd7fd0, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x00005650e5153149 in os_event_wait_time_low (event=0x5650e6fd7fd0, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x00005650e526d20f in srv_monitor_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:1687 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 31 (Thread 0x7ff8ff7fe700 (LWP 1863)): #0 0x00007ff93714c644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00005650e5144dad in LinuxAIOHandler::collect (this=0x7ff8ff7fdb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x00005650e51453f6 in LinuxAIOHandler::poll (this=0x7ff8ff7fdb00, m1=0x7ff8ff7fdba8, m2=0x7ff8ff7fdbb0, request=0x7ff8ff7fdbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x00005650e5145559 in os_aio_linux_handler (global_segment=1, m1=0x7ff8ff7fdba8, m2=0x7ff8ff7fdbb0, request=0x7ff8ff7fdbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x00005650e514a2c0 in os_aio_handler (segment=1, m1=0x7ff8ff7fdba8, m2=0x7ff8ff7fdbb0, request=0x7ff8ff7fdbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x00005650e54004f9 in fil_aio_wait (segment=1) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x00005650e5271e42 in io_handler_thread (arg=0x5650e64ab988 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 30 (Thread 0x7ff8dbfff700 (LWP 1878)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5153377 in os_event::wait (this=0x5650e6fd7dd0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x00005650e5152e6b in os_event::wait_low (this=0x5650e6fd7dd0, reset_sig_count=460) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x00005650e515316e in os_event_wait_low (event=0x5650e6fd7dd0, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x00005650e526b328 in srv_resume_thread (slot=0x5650e6267d10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x00005650e526f5e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7ff8f37fe700 (LWP 1871)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5152d00 in os_event::timed_wait (this=0x5650e72d5120, abstime=0x7ff8f37fdd50) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x00005650e5152fa9 in os_event::wait_time_low (this=0x5650e72d5120, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x00005650e5153149 in os_event_wait_time_low (event=0x5650e72d5120, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x00005650e5115527 in lock_wait_timeout_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/lock/lock0wait.cc:522 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7ff8db7fe700 (LWP 1879)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5153377 in os_event::wait (this=0x5650e6fd7ed0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x00005650e5152e6b in os_event::wait_low (this=0x5650e6fd7ed0, reset_sig_count=353) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x00005650e515316e in os_event_wait_low (event=0x5650e6fd7ed0, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x00005650e526b328 in srv_resume_thread (slot=0x5650e6267f10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x00005650e526f5e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7ff8fe7fc700 (LWP 1865)): #0 0x00007ff93714c644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00005650e5144dad in LinuxAIOHandler::collect (this=0x7ff8fe7fbb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x00005650e51453f6 in LinuxAIOHandler::poll (this=0x7ff8fe7fbb00, m1=0x7ff8fe7fbba8, m2=0x7ff8fe7fbbb0, request=0x7ff8fe7fbbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x00005650e5145559 in os_aio_linux_handler (global_segment=3, m1=0x7ff8fe7fbba8, m2=0x7ff8fe7fbbb0, request=0x7ff8fe7fbbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x00005650e514a2c0 in os_aio_handler (segment=3, m1=0x7ff8fe7fbba8, m2=0x7ff8fe7fbbb0, request=0x7ff8fe7fbbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x00005650e54004f9 in fil_aio_wait (segment=3) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x00005650e5271e42 in io_handler_thread (arg=0x5650e64ab998 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7ff8d9ffb700 (LWP 1882)): #0 0x00007ff937582e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x00005650e51536b4 in os_thread_sleep (tm=1000000) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0thread.cc:225 #2 0x00005650e5345dd3 in btr_defragment_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/btr/btr0defragment.cc:705 #3 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7ff91c8f2700 (LWP 2094)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e569da6b in safe_cond_timedwait (cond=0x5650e642d060 , mp=0x5650e642fa20 , abstime=0x7ff91c8f1c60, file=0x5650e575bd68 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x00005650e495ca15 in inline_mysql_cond_timedwait (that=0x5650e642d060 , mutex=0x5650e642fa20 , abstime=0x7ff91c8f1c60, src_file=0x5650e575ddb8 "/home/stepan/mariadb/10.4/github/sql/mysqld.cc", src_line=2686) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x00005650e4961b2c in cache_thread (thd=0x7ff8b0000b00) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:2685 #4 0x00005650e4961dd2 in one_thread_per_connection_end (thd=0x7ff8b0000b00, put_in_cache=true) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:2771 #5 0x00005650e4c26dcf in do_handle_one_connection (connect=0x5650e740f250) at /home/stepan/mariadb/10.4/github/sql/sql_connect.cc:1423 #6 0x00005650e4c26a80 in handle_one_connection (arg=0x5650e740f250) at /home/stepan/mariadb/10.4/github/sql/sql_connect.cc:1316 #7 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e6ed2ee0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #8 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #9 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 24 (Thread 0x7ff8fdffb700 (LWP 1866)): #0 0x00007ff93714c644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00005650e5144dad in LinuxAIOHandler::collect (this=0x7ff8fdffab00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x00005650e51453f6 in LinuxAIOHandler::poll (this=0x7ff8fdffab00, m1=0x7ff8fdffaba8, m2=0x7ff8fdffabb0, request=0x7ff8fdffabe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x00005650e5145559 in os_aio_linux_handler (global_segment=4, m1=0x7ff8fdffaba8, m2=0x7ff8fdffabb0, request=0x7ff8fdffabe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x00005650e514a2c0 in os_aio_handler (segment=4, m1=0x7ff8fdffaba8, m2=0x7ff8fdffabb0, request=0x7ff8fdffabe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x00005650e54004f9 in fil_aio_wait (segment=4) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x00005650e5271e42 in io_handler_thread (arg=0x5650e64ab9a0 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7ff91c988700 (LWP 1886)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e569d769 in safe_cond_wait (cond=0x5650e642d0a0 , mp=0x5650e6430320 , file=0x5650e576c0c0 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1167) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:492 #2 0x00005650e498f08e in inline_mysql_cond_wait (that=0x5650e642d0a0 , mutex=0x5650e6430320 , src_file=0x5650e576cc20 "/home/stepan/mariadb/10.4/github/sql/slave.cc", src_line=519) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1167 #3 0x00005650e49909cb in handle_slave_background (arg=0x0) at /home/stepan/mariadb/10.4/github/sql/slave.cc:519 #4 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e7481570) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 22 (Thread 0x7ff8d8ff9700 (LWP 1884)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e569d769 in safe_cond_wait (cond=0x7ff8d8ff8c70, mp=0x7ff8d8ff8cb0, file=0x5650e598ace0 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:492 #2 0x00005650e504615d in inline_mysql_cond_wait (that=0x7ff8d8ff8c70, mutex=0x7ff8d8ff8cb0, src_file=0x5650e598c7f8 "/home/stepan/mariadb/10.4/github/storage/innobase/handler/ha_innodb.cc", src_line=288) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1178 #3 0x00005650e504ba55 in thd_destructor_proxy () at /home/stepan/mariadb/10.4/github/storage/innobase/handler/ha_innodb.cc:288 #4 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e734c320) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7ff8fffff700 (LWP 1862)): #0 0x00007ff93714c644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00005650e5144dad in LinuxAIOHandler::collect (this=0x7ff8ffffeb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x00005650e51453f6 in LinuxAIOHandler::poll (this=0x7ff8ffffeb00, m1=0x7ff8ffffeba8, m2=0x7ff8ffffebb0, request=0x7ff8ffffebe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x00005650e5145559 in os_aio_linux_handler (global_segment=0, m1=0x7ff8ffffeba8, m2=0x7ff8ffffebb0, request=0x7ff8ffffebe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x00005650e514a2c0 in os_aio_handler (segment=0, m1=0x7ff8ffffeba8, m2=0x7ff8ffffebb0, request=0x7ff8ffffebe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x00005650e54004f9 in fil_aio_wait (segment=0) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x00005650e5271e42 in io_handler_thread (arg=0x5650e64ab980 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7ff8daffd700 (LWP 1880)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5153377 in os_event::wait (this=0x5650e6fd7e50) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x00005650e5152e6b in os_event::wait_low (this=0x5650e6fd7e50, reset_sig_count=378) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x00005650e515316e in os_event_wait_low (event=0x5650e6fd7e50, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x00005650e526b328 in srv_resume_thread (slot=0x5650e6267e10 , sig_count=0, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x00005650e526f5e9 in srv_worker_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2517 #6 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7ff8da7fc700 (LWP 1881)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5153377 in os_event::wait (this=0x5650e6fd8050) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x00005650e5152e6b in os_event::wait_low (this=0x5650e6fd8050, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x00005650e515316e in os_event_wait_low (event=0x5650e6fd8050, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x00005650e5370e86 in buf_dump_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0dump.cc:833 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7ff8f17fa700 (LWP 1876)): #0 0x00007ff937582e3d in nanosleep () from /lib64/libpthread.so.0 #1 0x00005650e51536b4 in os_thread_sleep (tm=1000000) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0thread.cc:225 #2 0x00005650e526ed65 in srv_master_sleep () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2334 #3 0x00005650e526eecd in srv_master_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2375 #4 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7ff91c9d3700 (LWP 1885)): #0 0x00007ff93587240a in sigwaitinfo () from /lib64/libc.so.6 #1 0x00005650e495c114 in my_sigwait (set=0x7ff91c9d2cf0, sig=0x7ff91c9d2cb4, code=0x7ff91c9d2cb8) at /home/stepan/mariadb/10.4/github/include/my_pthread.h:198 #2 0x00005650e4962a8d in signal_hand (arg=0x0) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:3243 #3 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e6fe0680) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #4 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #5 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7ff8f0ff9700 (LWP 1877)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5153377 in os_event::wait (this=0x5650e6fd7d50) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x00005650e5152e6b in os_event::wait_low (this=0x5650e6fd7d50, reset_sig_count=127) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x00005650e515316e in os_event_wait_low (event=0x5650e6fd7d50, reset_sig_count=127) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x00005650e526b328 in srv_resume_thread (slot=0x5650e6267c10 , sig_count=127, wait=true, timeout_usec=0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:909 #5 0x00005650e526faa0 in srv_purge_coordinator_suspend (slot=0x5650e6267c10 , rseg_history_len=12) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2657 #6 0x00005650e526fdd1 in srv_purge_coordinator_thread (arg=0x0) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:2738 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7ff8fcff9700 (LWP 1868)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5152d00 in os_event::timed_wait (this=0x5650e6fd80d0, abstime=0x7ff8fcff8b00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x00005650e5152fa9 in os_event::wait_time_low (this=0x5650e6fd80d0, time_in_usec=613000, reset_sig_count=2) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x00005650e5153149 in os_event_wait_time_low (event=0x5650e6fd80d0, time_in_usec=613000, reset_sig_count=2) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x00005650e537d358 in pc_sleep_if_needed (next_loop_time=959894955, sig_count=2, cur_time=959894342) at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:2643 #5 0x00005650e537e4fc in buf_flush_page_cleaner_coordinator () at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0flu.cc:3102 #6 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7ff8d97fa700 (LWP 1883)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5153377 in os_event::wait (this=0x5650e6fd8150) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:158 #2 0x00005650e5152e6b in os_event::wait_low (this=0x5650e6fd8150, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:325 #3 0x00005650e515316e in os_event_wait_low (event=0x5650e6fd8150, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:502 #4 0x00005650e53567d9 in buf_resize_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/buf/buf0buf.cc:3199 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7ff8f1ffb700 (LWP 1875)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5152d00 in os_event::timed_wait (this=0x5650e7239e70, abstime=0x7ff8f1ffab70) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x00005650e5152fa9 in os_event::wait_time_low (this=0x5650e7239e70, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x00005650e5153149 in os_event_wait_time_low (event=0x5650e7239e70, time_in_usec=5000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x00005650e52db563 in ib_wqueue_timedwait (wq=0x5650e723f990, wait_in_usecs=5000000) at /home/stepan/mariadb/10.4/github/storage/innobase/ut/ut0wqueue.cc:163 #5 0x00005650e544e23d in fts_optimize_thread (arg=0x5650e723f990) at /home/stepan/mariadb/10.4/github/storage/innobase/fts/fts0opt.cc:2865 #6 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7ff8fd7fa700 (LWP 1867)): #0 0x00007ff93714c644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00005650e5144dad in LinuxAIOHandler::collect (this=0x7ff8fd7f9b00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x00005650e51453f6 in LinuxAIOHandler::poll (this=0x7ff8fd7f9b00, m1=0x7ff8fd7f9ba8, m2=0x7ff8fd7f9bb0, request=0x7ff8fd7f9be0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x00005650e5145559 in os_aio_linux_handler (global_segment=5, m1=0x7ff8fd7f9ba8, m2=0x7ff8fd7f9bb0, request=0x7ff8fd7f9be0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x00005650e514a2c0 in os_aio_handler (segment=5, m1=0x7ff8fd7f9ba8, m2=0x7ff8fd7f9bb0, request=0x7ff8fd7f9be0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x00005650e54004f9 in fil_aio_wait (segment=5) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x00005650e5271e42 in io_handler_thread (arg=0x5650e64ab9a8 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7ff8f3fff700 (LWP 1872)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5152d00 in os_event::timed_wait (this=0x5650e6fd7f50, abstime=0x7ff8f3ffebb0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x00005650e5152fa9 in os_event::wait_time_low (this=0x5650e6fd7f50, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x00005650e5153149 in os_event_wait_time_low (event=0x5650e6fd7f50, time_in_usec=1000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x00005650e526d789 in srv_error_monitor_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0srv.cc:1851 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7ff8f27fc700 (LWP 1874)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e5152d00 in os_event::timed_wait (this=0x5650e73071e0, abstime=0x7ff8f27fbd70) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:275 #2 0x00005650e5152fa9 in os_event::wait_time_low (this=0x5650e73071e0, time_in_usec=10000000, reset_sig_count=1) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:385 #3 0x00005650e5153149 in os_event_wait_time_low (event=0x5650e73071e0, time_in_usec=10000000, reset_sig_count=0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0event.cc:485 #4 0x00005650e53e76ca in dict_stats_thread () at /home/stepan/mariadb/10.4/github/storage/innobase/dict/dict0stats_bg.cc:476 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7ff92c393700 (LWP 1662)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007ff92f3cce42 in fifo_lock_get (q=q@entry=0x7ff92f165010) at galerautils/src/gu_fifo.c:262 #2 0x00007ff92f3cd4b1 in gu_fifo_get_head (q=0x7ff92f165010, err=0x7ff92c39201c) at galerautils/src/gu_fifo.c:330 #3 0x00007ff92f4b0d1c in gcs_recv (conn=0x5650e6f60b70, action=0x7ff92c392060) at gcs/src/gcs.cpp:2024 #4 0x00007ff92f4ddf6c in galera::GcsActionSource::process (this=0x5650e6f603e0, recv_ctx=0x7ff92c392480, exit_loop=@0x7ff92c39224e: false) at galera/src/gcs_action_source.cpp:161 #5 0x00007ff92f4fcf24 in galera::ReplicatorSMM::async_recv (this=0x5650e6f48f90, recv_ctx=0x7ff92c392480) at galera/src/replicator_smm.cpp:391 #6 0x00005650e5753e7a in wsrep::wsrep_provider_v26::run_applier (this=0x5650e6ec1920, applier_ctx=0x7ff92c392480) at /home/stepan/mariadb/10.4/github/wsrep-lib/src/wsrep_provider_v26.cpp:727 #7 0x00005650e4d25787 in wsrep_replication_process (thd=0x7ff908000b00, arg=0x5650e6fa5b50) at /home/stepan/mariadb/10.4/github/sql/wsrep_thd.cc:62 #8 0x00005650e4d15e70 in start_wsrep_THD (arg=0x5650e6fa5b50) at /home/stepan/mariadb/10.4/github/sql/wsrep_mysqld.cc:2708 #9 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e6f79790) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #10 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #11 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7ff8feffd700 (LWP 1864)): #0 0x00007ff93714c644 in __io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00005650e5144dad in LinuxAIOHandler::collect (this=0x7ff8feffcb00) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1809 #2 0x00005650e51453f6 in LinuxAIOHandler::poll (this=0x7ff8feffcb00, m1=0x7ff8feffcba8, m2=0x7ff8feffcbb0, request=0x7ff8feffcbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:1968 #3 0x00005650e5145559 in os_aio_linux_handler (global_segment=2, m1=0x7ff8feffcba8, m2=0x7ff8feffcbb0, request=0x7ff8feffcbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:2022 #4 0x00005650e514a2c0 in os_aio_handler (segment=2, m1=0x7ff8feffcba8, m2=0x7ff8feffcbb0, request=0x7ff8feffcbe0) at /home/stepan/mariadb/10.4/github/storage/innobase/os/os0file.cc:5561 #5 0x00005650e54004f9 in fil_aio_wait (segment=2) at /home/stepan/mariadb/10.4/github/storage/innobase/fil/fil0fil.cc:4357 #6 0x00005650e5271e42 in io_handler_thread (arg=0x5650e64ab990 ) at /home/stepan/mariadb/10.4/github/storage/innobase/srv/srv0start.cc:324 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7ff92c3de700 (LWP 1661)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e569d769 in safe_cond_wait (cond=0x7ff914014758, mp=0x7ff9140146a8, file=0x5650e581a2c0 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1178) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:492 #2 0x00005650e4d251b4 in inline_mysql_cond_wait (that=0x7ff914014758, mutex=0x7ff9140146a8, src_file=0x5650e581a4e0 "/home/stepan/mariadb/10.4/github/sql/wsrep_thd.h", src_line=69) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1178 #3 0x00005650e4d28416 in Wsrep_thd_queue::pop_front (this=0x7ff914014650) at /home/stepan/mariadb/10.4/github/sql/wsrep_thd.h:69 #4 0x00005650e4d26e23 in wsrep_rollback_process (rollbacker=0x7ff914001d50, arg=0x5650e6faf290) at /home/stepan/mariadb/10.4/github/sql/wsrep_thd.cc:266 #5 0x00005650e4d15e70 in start_wsrep_THD (arg=0x5650e6faf290) at /home/stepan/mariadb/10.4/github/sql/wsrep_mysqld.cc:2708 #6 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e6f792a0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7ff92cbdf700 (LWP 1659)): #0 0x00007ff935939483 in epoll_wait () from /lib64/libc.so.6 #1 0x00007ff92f47b548 in asio::detail::epoll_reactor::run (this=0x5650e6f8de40, block=block@entry=true, ops=...) at asio/asio/detail/impl/epoll_reactor.ipp:391 #2 0x00007ff92f47fc40 in asio::detail::task_io_service::do_run_one (this=this@entry=0x5650e6f8dce0, lock=..., this_thread=..., ec=...) at asio/asio/detail/impl/task_io_service.ipp:355 #3 0x00007ff92f47fef2 in asio::detail::task_io_service::run (this=0x5650e6f8dce0, ec=...) at asio/asio/detail/impl/task_io_service.ipp:148 #4 0x00007ff92f49797d in asio::io_service::run (this=0x5650e6f8ca40) at asio/asio/impl/io_service.ipp:58 #5 0x00007ff92f4980d1 in gcomm::AsioProtonet::event_loop (this=0x5650e6f8c9a0, period=...) at gcomm/src/asio_protonet.cpp:116 #6 0x00007ff92f4b6f3d in GCommConn::run (this=0x5650e6f8d4c0) at gcs/src/gcs_gcomm.cpp:498 #7 0x00007ff92f4b9fe9 in GCommConn::run_fn (arg=) at gcs/src/gcs_gcomm.cpp:209 #8 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #9 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7ff91fffe700 (LWP 1660)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007ff92f4b54b7 in gu::Lock::wait (cond=..., this=0x7ff91fffd850) at galerautils/src/gu_lock.hpp:45 #2 0x00007ff92f4b7ffb in RecvBuf::front (this=this@entry=0x5650e6f8d6a8, timeout=...) at gcs/src/gcs_gcomm.cpp:124 #3 0x00007ff92f4b5b96 in gcomm_recv (backend=, msg=0x5650e6f60e50, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:673 #4 0x00007ff92f4a9b2c in core_msg_recv (backend=backend@entry=0x5650e6f60fb0, recv_msg=recv_msg@entry=0x5650e6f60e50, timeout=timeout@entry=9223372035999999999) at gcs/src/gcs_core.cpp:480 #5 0x00007ff92f4acda4 in gcs_core_recv (conn=0x5650e6f60df0, recv_act=recv_act@entry=0x7ff91fffdd70, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1184 #6 0x00007ff92f4b4028 in gcs_recv_thread (arg=0x5650e6f60b70) at gcs/src/gcs.cpp:1398 #7 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #8 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7ff92f063700 (LWP 1658)): #0 0x00007ff93757f965 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007ff92f4dadd7 in gu::Lock::wait (cond=..., this=0x7ff92f062bf0) at galerautils/src/gu_lock.hpp:45 #2 0x00007ff92f4dae69 in galera::ServiceThd::thd_func (arg=0x5650e6f496a8) at galera/src/galera_service_thd.cpp:30 #3 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7ff930014700 (LWP 1657)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e569da6b in safe_cond_timedwait (cond=0x5650e6ccece0 , mp=0x5650e6ccec20 , abstime=0x7ff930013d70, file=0x5650e5b50450 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x00005650e569eaae in inline_mysql_cond_timedwait (that=0x5650e6ccece0 , mutex=0x5650e6ccec20 , abstime=0x7ff930013d70, src_file=0x5650e5b50498 "/home/stepan/mariadb/10.4/github/mysys/thr_timer.c", src_line=292) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x00005650e569f671 in timer_handler (arg=0x0) at /home/stepan/mariadb/10.4/github/mysys/thr_timer.c:292 #4 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e6f15250) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #5 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #6 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7ff91dfd5700 (LWP 1861)): #0 0x00007ff93757fd12 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00005650e569da6b in safe_cond_timedwait (cond=0x5650e6cbc5e0 , mp=0x5650e6cbc520 , abstime=0x7ff91dfd4cb0, file=0x5650e5b07880 "/home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h", line=1204) at /home/stepan/mariadb/10.4/github/mysys/thr_mutex.c:546 #2 0x00005650e54f5109 in inline_mysql_cond_timedwait (that=0x5650e6cbc5e0 , mutex=0x5650e6cbc520 , abstime=0x7ff91dfd4cb0, src_file=0x5650e5b078c8 "/home/stepan/mariadb/10.4/github/storage/maria/ma_servicethread.c", src_line=116) at /home/stepan/mariadb/10.4/github/include/mysql/psi/mysql_thread.h:1204 #3 0x00005650e54f564d in my_service_thread_sleep (control=0x5650e626e780 , sleep_time=30000000000) at /home/stepan/mariadb/10.4/github/storage/maria/ma_servicethread.c:115 #4 0x00005650e54e950a in ma_checkpoint_background (arg=0x1e) at /home/stepan/mariadb/10.4/github/storage/maria/ma_checkpoint.c:707 #5 0x00005650e56264a5 in pfs_spawn_thread (arg=0x5650e6fcb4e0) at /home/stepan/mariadb/10.4/github/storage/perfschema/pfs.cc:1862 #6 0x00007ff93757bdd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007ff935938ead in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7ff93798c880 (LWP 1656)): #0 0x00007ff9375809d1 in pthread_kill () from /lib64/libpthread.so.0 #1 0x00005650e56980f6 in my_write_core (sig=6) at /home/stepan/mariadb/10.4/github/mysys/stacktrace.c:481 #2 0x00005650e4e04964 in handle_fatal_signal (sig=6) at /home/stepan/mariadb/10.4/github/sql/signal_handler.cc:343 #3 #4 0x00007ff93592e20d in poll () from /lib64/libc.so.6 #5 0x00005650e49692db in handle_connections_sockets () at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:6455 #6 0x00005650e4968576 in mysqld_main (argc=163, argv=0x5650e6dcf130) at /home/stepan/mariadb/10.4/github/sql/mysqld.cc:5891 #7 0x00005650e495c095 in main (argc=23, argv=0x7ffe5325cce8) at /home/stepan/mariadb/10.4/github/sql/main.cc:25 Compressed file /home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_gcs_fc_limit-innodb/mysqld.2/data/core.1656 - saving '/home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_gcs_fc_limit-innodb/' to '/home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_gcs_fc_limit-innodb/' ***Warnings generated in error logs during shutdown after running tests: galera.galera_fk_multitable galera.galera_toi_ddl_fk_insert galera.galera_fk_selfreferential galera.galera_create_function galera.galera_commit_empty galera.MW-286 galera.galera_events galera.galera_change_user galera.galera_alter_engine_innodb galera.galera_bf_abort_lock_table galera.galera_FK_duplicate_client_insert galera.MW-258 galera.galera_ftwrl_drain galera.galera_rsu_add_pk galera.MW-357 galera.galera_toi_ddl_fk_update galera.MDEV-16509 galera.galera_fk_cascade_update galera.galera_ddl_multiline galera.galera_bf_abort galera.galera_bf_abort_ftwrl galera.galera_create_table_as_select galera.GCF-939 galera.MW-369 galera.galera_fk_setnull galera.galera_alter_table_force galera.galera_fk_cascade_delete galera.GAL-480 galera.galera_bf_abort_sleep galera.galera_create_table_like galera.MDEV-20225 galera.galera_toi_ddl_error galera.MW-328D galera.galera_var_load_data_splitting galera.MW-416 galera.galera_wsrep_desync_wsrep_on galera.galera_events2 galera.basic galera.MW-309 galera.galera_bf_abort_at_after_statement galera.galera_enum galera.GAL-382 galera.MW-402 galera.galera_fk_multicolumn galera.MW-285 galera.enforce_storage_engine galera.binlog_checksum galera.galera_binlog_cache_size galera.create galera.galera_admin galera.GAL-401 galera.galera_create_procedure galera.MW-292 galera.galera_alter_engine_myisam galera.galera_create_trigger galera.galera_rsu_drop_pk galera.galera_bf_abort_flush_for_export galera.galera_ftwrl galera.MW-328E galera.galera_delete_limit galera.galera_drop_database galera.galera_desync_overlapped galera.galera_bf_abort_for_update galera.galera_applier_ftwrl_table galera.galera_drop_multi galera.galera_defaults galera.fk galera.MW-252 galera.galera_bf_abort_get_lock galera.galera_gcs_fc_limit galera.galera_fk_mismatch galera.galera_fk_no_pk galera.ev51914 galera.galera_binlog_row_image galera.galera#505 galera.MW-259 galera.galera_bf_abort_shutdown galera.galera_fk_conflict galera.galera#500 Attempting backtrace. You can use the following information to find out Attempting backtrace. You can use the following information to find out 191028 1:53:02 +32.844 galera.galera_gra_log 'innodb' [ pass ] 3765 191028 1:53:09 +7.015 galera.galera_insert_ignore 'innodb' [ pass ] 6691 191028 1:53:20 +11.543 galera.galera_insert_multi 'innodb' [ pass ] 11103 191028 1:53:34 +13.938 galera.galera_kill_applier 'innodb' [ pass ] 3574 191028 1:54:02 +27.898 galera.galera_kill_ddl 'innodb' [ pass ] 20327 191028 1:54:23 +20.559 galera.galera_kill_smallchanges 'innodb' [ pass ] 20255 191028 1:54:27 +4.312 galera.galera_last_committed_id 'innodb' [ pass ] 4029 191028 1:54:34 +7.137 galera.galera_lock_table 'innodb' [ pass ] 6827 191028 1:55:41 +66.876 galera.galera_many_indexes 'innodb' [ pass ] 66204 191028 1:55:49 +8.225 galera.galera_mdev_10812 'innodb' [ pass ] 7937 191028 1:55:56 +7.177 galera.galera_mdl_race 'innodb' [ pass ] 6762 191028 1:56:04 +8.043 galera.galera_multi_database 'innodb' [ pass ] 7603 191028 1:56:09 +4.826 galera.galera_myisam_autocommit 'innodb' [ pass ] 4382 191028 1:56:15 +5.582 galera.galera_myisam_transactions 'innodb' [ pass ] 5252 191028 1:56:22 +7.456 galera.galera_nopk_bit 'innodb' [ pass ] 7048 191028 1:56:29 +6.394 galera.galera_nopk_blob 'innodb' [ pass ] 5976 191028 1:56:35 +6.807 galera.galera_nopk_large_varchar 'innodb' [ pass ] 6185 191028 1:56:40 +4.789 galera.galera_nopk_unicode 'innodb' [ pass ] 4456 191028 1:56:48 +7.889 galera.galera_parallel_apply_lock_table 'innodb' [ pass ] 7475 191028 1:56:57 +9.161 galera.galera_parallel_simple 'innodb' [ pass ] 8526 191028 1:57:21 +24.216 galera.galera_pc_ignore_sb 'innodb' [ pass ] 23906 191028 1:57:28 +6.110 galera.galera_performance_schema 'innodb' [ pass ] 5801 191028 1:57:31 +3.959 galera.galera_pk_bigint_signed 'innodb' [ pass ] 3661 191028 1:57:35 +3.869 galera.galera_pk_bigint_unsigned 'innodb' [ pass ] 3536 191028 1:57:45 +9.543 galera.galera_prepared_statement 'innodb' [ pass ] 9283 191028 1:57:50 +4.969 galera.galera_read_only 'innodb' [ pass ] 4667 191028 1:57:57 +6.663 galera.galera_repl_key_format_flat16 'innodb' [ pass ] 6337 191028 1:58:01 +4.168 galera.galera_repl_max_ws_size 'innodb' [ pass ] 3792 191028 1:58:13 +12.516 galera.galera_restart_nochanges 'innodb' [ pass ] 12239 191028 1:59:37 +83.446 galera.galera_restart_on_unknown_option 'innodb' [ pass ] 83066 191028 1:59:56 +19.738 galera.galera_roles 'innodb' [ pass ] 19212 191028 2:00:02 +5.171 galera.galera_rsu_error 'innodb' [ pass ] 4836 191028 2:00:06 +4.490 galera.galera_rsu_simple 'innodb' [ pass ] 4099 191028 2:00:14 +7.813 galera.galera_rsu_wsrep_desync 'innodb' [ pass ] 7408 191028 2:00:19 +4.654 galera.galera_sbr 'innodb' [ pass ] 4316 191028 2:00:21 +2.296 galera.galera_schema_dirty_reads 'innodb' [ pass ] 2021 191028 2:00:29 +8.460 galera.galera_serializable 'innodb' [ pass ] 8004 191028 2:00:33 +3.453 galera.galera_server 'innodb' [ pass ] 3092 191028 2:00:54 +21.133 galera.galera_shutdown_nonprim 'innodb' [ pass ] 20743 191028 2:01:10 +15.823 galera.galera_sp_bf_abort 'innodb' [ pass ] 15452 191028 2:06:03 +293.041 galera.galera_sp_insert_parallel 'innodb' [ pass ] 290773 191028 2:06:07 +4.493 galera.galera_sql_log_bin_zero 'innodb' [ pass ] 4100 191028 2:06:10 +2.377 galera.galera_status_cluster 'innodb' [ pass ] 2022 191028 2:06:14 +4.218 galera.galera_status_local_index 'innodb' [ pass ] 3885 191028 2:06:16 +2.421 galera.galera_status_local_state 'innodb' [ pass ] 2146 191028 2:06:44 +28.235 galera.galera_suspend_slave 'innodb' [ pass ] 27745 191028 2:06:54 +9.273 galera.galera_sync_wait_show 'innodb' [ pass ] 8891 191028 2:07:05 +11.716 galera.galera_toi_alter_auto_increment 'innodb' [ pass ] 11368 191028 2:07:15 +9.648 galera.galera_toi_ddl_locking 'innodb' [ pass ] 9249 191028 2:07:21 +6.308 galera.galera_toi_ddl_nonconflicting 'innodb' [ pass ] 5380 191028 2:07:28 +6.764 galera.galera_toi_ddl_sequential 'innodb' [ pass ] 5590 191028 2:10:16 +168.222 galera.galera_toi_drop_database 'innodb' [ pass ] 166988 191028 2:10:22 +5.485 galera.galera_toi_ftwrl 'innodb' [ pass ] 5219 191028 2:10:28 +6.407 galera.galera_toi_lock_exclusive 'innodb' [ pass ] 5899 191028 2:10:33 +4.979 galera.galera_toi_lock_shared 'innodb' [ pass ] 4576 191028 2:11:12 +39.066 galera.galera_toi_truncate 'innodb' [ pass ] 38687 191028 2:11:16 +3.937 galera.galera_transaction_read_only 'innodb' [ pass ] 3493 191028 2:11:28 +11.361 galera.galera_transaction_replay 'innodb' [ pass ] 10975 191028 2:11:43 +15.759 galera.galera_truncate 'innodb' [ pass ] 15221 191028 2:11:53 +10.067 galera.galera_truncate_temporary 'innodb' [ pass ] 9694 191028 2:12:04 +10.565 galera.galera_unicode_identifiers 'innodb' [ pass ] 10222 191028 2:12:08 +4.403 galera.galera_unicode_pk 'innodb' [ pass ] 4062 191028 2:12:16 +7.503 galera.galera_update_limit 'innodb' [ pass ] 6904 191028 2:12:20 +4.221 galera.galera_var_OSU_method 'innodb' [ pass ] 3893 191028 2:12:25 +4.611 galera.galera_var_OSU_method2 'innodb' [ pass ] 4295 191028 2:12:31 +6.107 galera.galera_var_auto_inc_control_off 'innodb' [ pass ] 5808 191028 2:12:37 +6.613 galera.galera_var_certify_nonPK_off 'innodb' [ pass ] 6253 191028 2:12:46 +8.349 galera.galera_var_cluster_address 'innodb' [ pass ] 8056 191028 2:12:54 +7.933 galera.galera_var_desync_on 'innodb' [ pass ] 7512 191028 2:13:02 +7.845 galera.galera_var_dirty_reads 'innodb' [ pass ] 7529 191028 2:13:08 +6.891 galera.galera_var_fkchecks 'innodb' [ pass ] 6585 191028 2:13:42 +33.451 galera.galera_var_ignore_apply_errors 'innodb' [ pass ] 33104 191028 2:14:34 +51.687 galera.galera_var_max_ws_rows 'innodb' [ pass ] 51193 191028 2:14:38 +4.301 galera.galera_var_max_ws_size 'innodb' [ pass ] 3736 191028 2:14:42 +4.279 galera.galera_var_mysql_replication_bundle 'innodb' [ pass ] 3910 191028 2:14:45 +3.150 galera.galera_var_replicate_myisam_off 'innodb' [ pass ] 2724 191028 2:15:02 +16.240 galera.galera_var_replicate_myisam_on 'innodb' [ pass ] 15765 191028 2:15:09 +7.536 galera.galera_var_sync_wait 'innodb' [ pass ] 7176 191028 2:15:13 +4.319 galera.galera_var_wsrep_on_off 'innodb' [ pass ] 3985 191028 2:15:16 +2.292 galera.galera_wsrep_provider_options_syntax 'innodb' [ pass ] 2029 191028 2:15:25 +9.136 galera.galera_wsrep_provider_unset_set 'innodb' [ pass ] 8863 191028 2:15:32 +7.088 galera.galera_zero_length_column 'innodb' [ pass ] 6656 191028 2:15:35 +3.454 galera.grant 'innodb' [ pass ] 3182 191028 2:15:40 +4.778 galera.lp1276424 'innodb' [ pass ] 4444 191028 2:15:46 +5.326 galera.lp1347768 'innodb' [ pass ] 3977 191028 2:15:53 +7.865 galera.lp1376747 'innodb' [ pass ] 7406 191028 2:15:59 +6.045 galera.lp1376747-2 'innodb' [ pass ] 5634 191028 2:16:05 +5.577 galera.lp1376747-3 'innodb' [ pass ] 5241 191028 2:16:17 +11.642 galera.lp1376747-4 'innodb' [ pass ] 11042 191028 2:16:26 +8.931 galera.lp1438990 'innodb' [ pass ] 8388 191028 2:16:33 +7.030 galera.lp959512 'innodb' [ pass ] 6630 191028 2:16:37 +3.943 galera.mdev_9290 'innodb' [ pass ] 3600 191028 2:16:45 +8.228 galera.mysql-wsrep#110 'innodb' [ pass ] 7870 191028 2:16:51 +5.895 galera.mysql-wsrep#237 'innodb' [ pass ] 5498 191028 2:16:55 +4.080 galera.mysql-wsrep#247 'innodb' [ pass ] 3783 191028 2:17:08 +13.279 galera.mysql-wsrep#31 'innodb' [ pass ] 12947 191028 2:17:48 +40.145 galera.mysql-wsrep#332 'innodb' [ pass ] 30283 191028 2:17:56 +7.824 galera.mysql-wsrep#90 'innodb' [ pass ] 7134 191028 2:18:06 +9.854 galera.pxc-421 'innodb' [ pass ] 9298 191028 2:18:12 +6.275 galera.rename 'innodb' [ pass ] 5678 191028 2:18:19 +7.337 galera.sql_log_bin 'innodb' [ pass ] 6859 191028 2:18:25 +5.878 galera.unique_key 'innodb' [ pass ] 5358 191028 2:18:34 +8.294 galera.view 'innodb' [ pass ] 7736 191028 2:18:36 +2.501 galera.wsrep_trx_fragment_size_sr 'innodb' [ pass ] 2040 191028 2:19:08 +31.359 galera.MDEV-15443 'innodb' [ pass ] 13054 191028 2:19:34 +26.616 galera.MDEV-18832 'innodb' [ pass ] 7132 191028 2:22:22 +167.976 galera.MDEV-6860 'innodb' [ pass ] 149100 191028 2:22:46 +24.121 galera.MW-284 'innodb' [ pass ] 12885 191028 2:23:13 +27.038 galera.MW-336 'innodb' [ pass ] 6296 191028 2:23:32 +19.111 galera.galera#414 'innodb' [ pass ] 9403 191028 2:23:58 +25.591 galera.galera_as_master 'innodb' [ pass ] 8844 191028 2:24:15 +16.783 galera.galera_as_master_large 'innodb' [ pass ] 5528 191028 2:24:31 +16.182 galera.galera_as_slave 'innodb' [ pass ] 4709 191028 2:24:46 +15.478 galera.galera_as_slave_autoinc 'innodb' [ pass ] 4573 191028 2:25:01 +14.883 galera.galera_as_slave_gtid 'innodb' [ pass ] 4469 191028 2:25:20 +18.999 galera.galera_as_slave_gtid_replicate_do_db 'innodb' [ pass ] 8742 191028 2:25:54 +34.084 galera.galera_as_slave_nonprim 'innodb' [ pass ] 20999 191028 2:26:54 +59.852 galera.galera_autoinc_sst_mariabackup 'innodb' [ pass ] 37241 191028 2:27:41 +46.340 galera.galera_gcache_recover 'innodb' [ pass ] 29321 191028 2:28:54 +73.876 galera.galera_gcache_recover_full_gcache 'innodb' [ pass ] 54472 191028 2:29:19 +24.434 galera.galera_gcs_fragment 'innodb' [ pass ] 7866 191028 2:29:48 +28.781 galera.galera_gcs_max_packet_size 'innodb' [ pass ] 12477 191028 2:30:12 +23.867 galera.galera_gtid_slave 'innodb' [ pass ] 6303 191028 2:30:45 +33.869 galera.galera_gtid_slave_sst_rsync 'innodb' [ pass ] 22703 191028 2:31:52 +67.069 galera.galera_ist_mariabackup_innodb_flush_logs 'debug,innodb' [ pass ] 45819 191028 2:34:03 +130.962 galera.galera_ist_mysqldump 'debug,innodb' [ pass ] 113357 191028 2:34:37 +33.094 galera.galera_ist_recv_bind 'innodb' [ pass ] 15366 191028 2:35:17 +40.186 galera.galera_ist_restart_joiner 'innodb' [ pass ] 24537 191028 2:36:38 +80.902 galera.galera_ist_rsync 'debug,innodb' [ pass ] 61100 191028 2:37:20 +42.680 galera.galera_load_data 'innodb' [ pass ] 25019 191028 2:43:48 +387.802 galera.galera_many_rows 'innodb' [ pass ] 367211 191028 2:44:37 +48.953 galera.galera_mdev_15611 'innodb' [ pass ] 30399 191028 2:44:59 +22.040 galera.galera_ssl 'innodb' [ pass ] 3757 191028 2:45:21 +21.837 galera.galera_ssl_compression 'innodb' [ pass ] 4918 191028 2:47:14 +113.183 galera.galera_sst_mariabackup 'debug,innodb' [ pass ] 86252 191028 2:49:03 +108.430 galera.galera_sst_mariabackup_data_dir 'debug,innodb' [ pass ] 82452 191028 2:49:31 +28.623 galera.galera_sst_mariabackup_encrypt_with_key 'innodb' [ pass ] 2034 191028 2:51:09 +98.285 galera.galera_sst_mariabackup_table_options '16k,clear,innodb' [ pass ] 33282 191028 2:52:45 +95.983 galera.galera_sst_mariabackup_table_options '16k,crypt,innodb' [ pass ] 33718 191028 2:54:30 +104.267 galera.galera_sst_mariabackup_table_options '4k,clear,innodb' [ pass ] 41907 191028 2:56:08 +97.851 galera.galera_sst_mariabackup_table_options '4k,crypt,innodb' [ pass ] 35817 191028 2:57:43 +95.444 galera.galera_sst_mariabackup_table_options '8k,clear,innodb' [ pass ] 33492 191028 2:59:18 +95.343 galera.galera_sst_mariabackup_table_options '8k,crypt,innodb' [ pass ] 32319 191028 3:02:32 +194.164 galera.galera_sst_mysqldump 'debug,innodb' [ pass ] 177160 191028 3:03:21 +48.345 galera.galera_sst_mysqldump_with_key 'debug,innodb' [ fail ] Test ended at 2019-10-28 03:03:21 CURRENT_TEST: galera.galera_sst_mysqldump_with_key mysqltest: In included file "./suite/galera/include/galera_st_kill_slave_ddl.inc": included from /home/stepan/mariadb/10.4/github/mysql-test/suite/galera/t/galera_sst_mysqldump_with_key.test at line 28: At line 23: query 'COMMIT' failed: 1213: Deadlock found when trying to get lock; try restarting transaction The result from queries just before the failure was: < snip > SET AUTOCOMMIT=ON; Performing State Transfer on a server that has been killed and restarted while a DDL was in progress on it connection node_1; CREATE TABLE t1 (f1 CHAR(255)) ENGINE=InnoDB; SET AUTOCOMMIT=OFF; START TRANSACTION; INSERT INTO t1 VALUES ('node1_committed_before'); INSERT INTO t1 VALUES ('node1_committed_before'); INSERT INTO t1 VALUES ('node1_committed_before'); INSERT INTO t1 VALUES ('node1_committed_before'); INSERT INTO t1 VALUES ('node1_committed_before'); connection node_2; START TRANSACTION; INSERT INTO t1 VALUES ('node2_committed_before'); INSERT INTO t1 VALUES ('node2_committed_before'); INSERT INTO t1 VALUES ('node2_committed_before'); INSERT INTO t1 VALUES ('node2_committed_before'); INSERT INTO t1 VALUES ('node2_committed_before'); COMMIT; More results from queries before failure can be found in /home/stepan/mariadb/10.4/github/mysql-test/var/log/galera_sst_mysqldump_with_key,debug.log - saving '/home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_sst_mysqldump_with_key-debug,innodb/' to '/home/stepan/mariadb/10.4/github/mysql-test/var/log/galera.galera_sst_mysqldump_with_key-debug,innodb/' 191028 3:04:33 +71.859 galera.galera_sst_rsync 'debug,innodb' [ pass ] 62919 191028 3:05:53 +80.624 galera.galera_sst_rsync2 'debug,innodb' [ pass ] 64692 191028 3:07:12 +78.973 galera.galera_sst_rsync_data_dir 'debug,innodb' [ pass ] 62765 191028 3:07:33 +21.027 galera.galera_var_gtid_domain_id 'innodb' [ pass ] 5228 191028 3:07:57 +23.341 galera.galera_var_log_bin 'innodb' [ pass ] 3867 191028 3:08:42 +45.827 galera.galera_var_slave_threads 'innodb' [ pass ] 16547 191028 3:09:58 +75.477 galera.galera_wan_restart_ist 'innodb' [ pass ] 61920 191028 3:11:15 +76.848 galera.galera_wan_restart_sst 'innodb' [ pass ] 60607 191028 3:11:30 +15.502 galera.mdev_10518 'innodb' [ pass ] 4811 191028 3:12:42 +72.102 galera.mysql-wsrep#33 'innodb' [ pass ] 52465 191028 3:13:03 +20.449 galera.rpl_row_annotate 'innodb' [ pass ] 4255 191028 3:13:31 +28.096 galera.versioning_trx_id 'innodb' [ pass ] 4333 -------------------------------------------------------------------------- The servers were restarted 83 times Spent 7972.671 of 15341 seconds executing testcases Completed: Failed 2/275 tests, 99.27% were successful. Failing test(s): galera.galera_gcs_fc_limit galera.galera_sst_mysqldump_with_key The log files in var/log may give you some hint of what went wrong. If you want to report this error, please read first the documentation at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html Errors/warnings were found in logfiles during server shutdown after running the following sequence(s) of tests: galera.galera_fk_multitable galera.galera_toi_ddl_fk_insert galera.galera_fk_selfreferential galera.galera_create_function galera.galera_commit_empty galera.MW-286 galera.galera_events galera.galera_change_user galera.galera_alter_engine_innodb galera.galera_bf_abort_lock_table galera.galera_FK_duplicate_client_insert galera.MW-258 galera.galera_ftwrl_drain galera.galera_rsu_add_pk galera.MW-357 galera.galera_toi_ddl_fk_update galera.MDEV-16509 galera.galera_fk_cascade_update galera.galera_ddl_multiline galera.galera_bf_abort galera.galera_bf_abort_ftwrl galera.galera_create_table_as_select galera.GCF-939 galera.MW-369 galera.galera_fk_setnull galera.galera_alter_table_force galera.galera_fk_cascade_delete galera.GAL-480 galera.galera_bf_abort_sleep galera.galera_create_table_like galera.MDEV-20225 galera.galera_toi_ddl_error galera.MW-328D galera.galera_var_load_data_splitting galera.MW-416 galera.galera_wsrep_desync_wsrep_on galera.galera_events2 galera.basic galera.MW-309 galera.galera_bf_abort_at_after_statement galera.galera_enum galera.GAL-382 galera.MW-402 galera.galera_fk_multicolumn galera.MW-285 galera.enforce_storage_engine galera.binlog_checksum galera.galera_binlog_cache_size galera.create galera.galera_admin galera.GAL-401 galera.galera_create_procedure galera.MW-292 galera.galera_alter_engine_myisam galera.galera_create_trigger galera.galera_rsu_drop_pk galera.galera_bf_abort_flush_for_export galera.galera_ftwrl galera.MW-328E galera.galera_delete_limit galera.galera_drop_database galera.galera_desync_overlapped galera.galera_bf_abort_for_update galera.galera_applier_ftwrl_table galera.galera_drop_multi galera.galera_defaults galera.fk galera.MW-252 galera.galera_bf_abort_get_lock galera.galera_gcs_fc_limit galera.galera_fk_mismatch galera.galera_fk_no_pk galera.ev51914 galera.galera_binlog_row_image galera.galera#505 galera.MW-259 galera.galera_bf_abort_shutdown galera.galera_fk_conflict galera.galera#500 191028 3:13:34 +2.694 mysql-test-run: *** ERROR: there were failing test cases