[MDEV-26308] Galera test failure on galera.galera_split_brain - Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Critical
Resolution: Fixed
Affects Version/s: 10.4(EOL), 10.5, 10.6
Fix Version/s: 10.4.22, 10.5.13, 10.6.5
Component/s: Galera, Tests
Labels:
None

Description

galera.galera_split_brain 'innodb'       [ fail ]  timeout after 900 seconds

        Test ended at 2021-08-05 12:24:06

Test case timeout after 900 seconds

== /home/jan/mysql/10.6e-galera/mysql-test/var/log/galera_split_brain.log ==

connection node_2;

connection node_1;

connection node_1;

connection node_2;

call mtr.add_suppression("WSREP: TO isolation failed for: ");

connection node_1;

connection node_2;

Killing server ...

connection node_1;

CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB;

 == /home/jan/mysql/10.6e-galera/mysql-test/var/tmp/analyze-timeout-mysqld.2.err ==

mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local server through socket '/home/jan/mysql/10.6e-galera/mysql-test/var/tmp/mysqld.2.sock' (111)

 == /home/jan/mysql/10.6e-galera/mysql-test/var/tmp/analyze-timeout-mysqld.1.err ==

mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local server through socket '/home/jan/mysql/10.6e-galera/mysql-test/var/tmp/mysqld.1.sock' (111)

 - found 'core' (0/5)

Trying 'dbx' to get a backtrace

Trying 'gdb' to get a backtrace from coredump /home/jan/mysql/10.6e-galera/mysql-test/var/log/galera.galera_split_brain-innodb/mysqld.1/data/core

Core generated by '/home/jan/mysql/10.6e-galera/sql/mariadbd'

Output from gdb follows. The first stack trace is from the failing thread.

The following stack traces are from all threads (so the failing one is

duplicated).

--------------------------

warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing

warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing

warning: Can't open file /home/jan/mysql/10.6e-galera/mysql-test/var/mysqld.1/data/galera.cache during file-backed mapping note processing

warning: Can't open file /home/jan/mysql/10.6e-galera/mysql-test/var/mysqld.1/data/tc.log during file-backed mapping note processing

[New LWP 3287487]

[New LWP 3287491]

[New LWP 3287490]

[New LWP 3287494]

[New LWP 3287495]

[New LWP 3287498]

[New LWP 3287509]

[New LWP 3287499]

[New LWP 3287488]

[New LWP 3287492]

[New LWP 3287842]

[New LWP 3287504]

[New LWP 3287502]

[New LWP 3287493]

[New LWP 3287489]

[New LWP 3287508]

Warning: couldn't activate thread debugging using libthread_db: Cannot find new threads: generic error

warning: File "/usr/lib/x86_64-linux-gnu/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".

To enable execution of this file add

	add-auto-load-safe-path /usr/lib/x86_64-linux-gnu/libthread_db-1.0.so

line to your configuration file "/home/jan/.gdbinit".

To completely disable this security protection add

	set auto-load safe-path /

line to your configuration file "/home/jan/.gdbinit".

For more information about this security protection see the

"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:

	info "(gdb)Auto-loading safe path"

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Core was generated by `/home/jan/mysql/10.6e-galera/sql/mariadbd --defaults-group-suffix=.1 --defaults'.

Program terminated with signal SIGABRT, Aborted.

#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:54

54	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.

[Current thread is 1 (Thread 0x7f9517eb6080 (LWP 3287487))]

#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:54

#1  0x000055fcbe860326 in my_write_core (sig=6) at /home/jan/mysql/10.6e-galera/mysys/stacktrace.c:424

#2  0x000055fcbdf5de70 in handle_fatal_signal (sig=6) at /home/jan/mysql/10.6e-galera/sql/signal_handler.cc:344

#3  <signal handler called>

#4  0x00007f951829722f in __GI___poll (fds=0x55fcc18981e8, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29

#5  0x000055fcbda721e4 in handle_connections_sockets () at /home/jan/mysql/10.6e-galera/sql/mysqld.cc:6140

#6  0x000055fcbda717ca in mysqld_main (argc=170, argv=0x55fcc10af100) at /home/jan/mysql/10.6e-galera/sql/mysqld.cc:5845

#7  0x000055fcbda65bad in main (argc=23, argv=0x7ffd77ffd838) at /home/jan/mysql/10.6e-galera/sql/main.cc:34

Thread 16 (Thread 0x7f95100b5640 (LWP 3287508)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55fcbf5273e8 <COND_manager+40>) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcbf5273e8 <COND_manager+40>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711540 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55fcbf527328 <LOCK_manager+40>, cond=0x55fcbf5273c0 <COND_manager>) at pthread_cond_wait.c:504

#3  __pthread_cond_wait (cond=0x55fcbf5273c0 <COND_manager>, mutex=0x55fcbf527328 <LOCK_manager+40>) at pthread_cond_wait.c:628

#4  0x000055fcbe865531 in safe_cond_wait (cond=0x55fcbf5273c0 <COND_manager>, mp=0x55fcbf527300 <LOCK_manager>, file=0x55fcbedb8170 "/home/jan/mysql/10.6e-galera/mysys/my_thr_init.c", line=596) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:492

#5  0x000055fcbe85e7b4 in psi_cond_wait (that=0x55fcbf5273c0 <COND_manager>, mutex=0x55fcbf527300 <LOCK_manager>, file=0x55fcbe970fb8 "/home/jan/mysql/10.6e-galera/sql/sql_manager.cc", line=103) at /home/jan/mysql/10.6e-galera/mysys/my_thr_init.c:596

#6  0x000055fcbdbb66f0 in inline_mysql_cond_wait (that=0x55fcbf5273c0 <COND_manager>, mutex=0x55fcbf527300 <LOCK_manager>, src_file=0x55fcbe970fb8 "/home/jan/mysql/10.6e-galera/sql/sql_manager.cc", src_line=103) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1070

#7  0x000055fcbdbb6b32 in handle_manager (arg=0x0) at /home/jan/mysql/10.6e-galera/sql/sql_manager.cc:103

#8  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc187cea8) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#9  0x00007f951870b450 in start_thread (arg=0x7f95100b5640) at pthread_create.c:473

#10 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 15 (Thread 0x7f9513b96640 (LWP 3287489)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=411800320, abstime=0x7f9513b95cb0, clockid=0, expected=0, futex_word=0x55fcbfdacc6c <wsrep_cert_alert_cond+44>) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcbfdacc6c <wsrep_cert_alert_cond+44>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7f9513b95cb0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f951871199e in __pthread_cond_wait_common (abstime=0x7f9513b95cb0, clockid=0, mutex=0x55fcbfdacba8 <wsrep_cert_alert_mutex+40>, cond=0x55fcbfdacc40 <wsrep_cert_alert_cond>) at pthread_cond_wait.c:504

#3  __pthread_cond_timedwait (cond=0x55fcbfdacc40 <wsrep_cert_alert_cond>, mutex=0x55fcbfdacba8 <wsrep_cert_alert_mutex+40>, abstime=0x7f9513b95cb0) at pthread_cond_wait.c:646

#4  0x000055fcbe8658f3 in safe_cond_timedwait (cond=0x55fcbfdacc40 <wsrep_cert_alert_cond>, mp=0x55fcbfdacb80 <wsrep_cert_alert_mutex>, abstime=0x7f9513b95cb0, file=0x55fcbedb8170 "/home/jan/mysql/10.6e-galera/mysys/my_thr_init.c", line=609) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:546

#5  0x000055fcbe85e867 in psi_cond_timedwait (that=0x55fcbfdacc40 <wsrep_cert_alert_cond>, mutex=0x55fcbfdacb80 <wsrep_cert_alert_mutex>, abstime=0x7f9513b95cb0, file=0x55fcbec59328 "/home/jan/mysql/10.6e-galera/sql/wsrep_tls_service.cc", line=692) at /home/jan/mysql/10.6e-galera/mysys/my_thr_init.c:609

#6  0x000055fcbe363cfc in inline_mysql_cond_timedwait (that=0x55fcbfdacc40 <wsrep_cert_alert_cond>, mutex=0x55fcbfdacb80 <wsrep_cert_alert_mutex>, abstime=0x7f9513b95cb0, src_file=0x55fcbec59328 "/home/jan/mysql/10.6e-galera/sql/wsrep_tls_service.cc", src_line=692) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1086

#7  0x000055fcbe3650f5 in wsrep_cert_alert_run () at /home/jan/mysql/10.6e-galera/sql/wsrep_tls_service.cc:692

#8  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc11e8c08) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#9  0x00007f951870b450 in start_thread (arg=0x7f9513b96640) at pthread_create.c:473

#10 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 14 (Thread 0x7f9510679640 (LWP 3287493)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7f94f4017c70) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7f94f4017c70, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711540 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7f94f4017bc0, cond=0x7f94f4017c48) at pthread_cond_wait.c:504

#3  __pthread_cond_wait (cond=0x7f94f4017c48, mutex=0x7f94f4017bc0) at pthread_cond_wait.c:628

#4  0x000055fcbe865531 in safe_cond_wait (cond=0x7f94f4017c48, mp=0x7f94f4017b98, file=0x55fcbec5beb0 "/home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h", line=1072) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:492

#5  0x000055fcbe36c353 in inline_mysql_cond_wait (that=0x7f94f4017c48, mutex=0x7f94f4017b98, src_file=0x55fcbec5c848 "/home/jan/mysql/10.6e-galera/sql/wsrep_thd.h", src_line=69) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1072

#6  0x000055fcbe36f5d3 in Wsrep_thd_queue::pop_front (this=0x7f94f4017b40) at /home/jan/mysql/10.6e-galera/sql/wsrep_thd.h:69

#7  0x000055fcbe36d70c in wsrep_rollback_process (rollbacker=0x7f94f40022b8, arg=0x55fcc12526b0) at /home/jan/mysql/10.6e-galera/sql/wsrep_thd.cc:236

#8  0x000055fcbe359691 in start_wsrep_THD (arg=0x55fcc12526b0) at /home/jan/mysql/10.6e-galera/sql/wsrep_mysqld.cc:3863

#9  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc122c128) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#10 0x00007f951870b450 in start_thread (arg=0x7f9510679640) at pthread_create.c:473

#11 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 13 (Thread 0x7f94e37fe640 (LWP 3287502)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=32661, abstime=0x7f94e37fdb00, clockid=1, expected=0, futex_word=0x55fcc132008c) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcc132008c, expected=expected@entry=0, clockid=clockid@entry=1, abstime=abstime@entry=0x7f94e37fdb00, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711de8 in __pthread_cond_wait_common (abstime=0x7f94e37fdb00, clockid=1, mutex=0x55fcc13105d0, cond=0x55fcc1320060) at pthread_cond_wait.c:504

#3  __pthread_cond_clockwait (abstime=0x7f94e37fdb00, clockid=1, mutex=0x55fcc13105d0, cond=0x55fcc1320060) at pthread_cond_wait.c:681

#4  __pthread_cond_clockwait (cond=0x55fcc1320060, mutex=0x55fcc13105d0, clockid=1, abstime=0x7f94e37fdb00) at pthread_cond_wait.c:669

#5  0x000055fcbe7bba12 in std::condition_variable::__wait_until_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x55fcc1320060, __lock=..., __atime=...) at /usr/include/c++/10/condition_variable:209

#6  0x000055fcbe7bad35 in std::condition_variable::wait_until<std::chrono::duration<long, std::ratio<1l, 1000000000l> > > (this=0x55fcc1320060, __lock=..., __atime=...) at /usr/include/c++/10/condition_variable:119

#7  0x000055fcbe7ba1c6 in std::condition_variable::wait_for<long, std::ratio<1l, 1000l> > (this=0x55fcc1320060, __lock=..., __rtime=...) at /usr/include/c++/10/condition_variable:172

#8  0x000055fcbe7b7845 in tpool::thread_pool_generic::wait_for_tasks (this=0x55fcc13104c0, lk=..., thread_data=0x55fcc1320060) at /home/jan/mysql/10.6e-galera/tpool/tpool_generic.cc:450

#9  0x000055fcbe7b7a55 in tpool::thread_pool_generic::get_task (this=0x55fcc13104c0, thread_var=0x55fcc1320060, t=0x7f94e37fdc38) at /home/jan/mysql/10.6e-galera/tpool/tpool_generic.cc:503

#10 0x000055fcbe7b7ceb in tpool::thread_pool_generic::worker_main (this=0x55fcc13104c0, thread_var=0x55fcc1320060) at /home/jan/mysql/10.6e-galera/tpool/tpool_generic.cc:548

#11 0x000055fcbe7beb18 in std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__f=@0x55fcc1437fc8: (void (tpool::thread_pool_generic::*)(tpool::thread_pool_generic * const, tpool::worker_data *)) 0x55fcbe7b7c86 <tpool::thread_pool_generic::worker_main(tpool::worker_data*)>, __t=@0x55fcc1437fc0: 0x55fcc13104c0) at /usr/include/c++/10/bits/invoke.h:73

#12 0x000055fcbe7be9e8 in std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> (__fn=@0x55fcc1437fc8: (void (tpool::thread_pool_generic::*)(tpool::thread_pool_generic * const, tpool::worker_data *)) 0x55fcbe7b7c86 <tpool::thread_pool_generic::worker_main(tpool::worker_data*)>) at /usr/include/c++/10/bits/invoke.h:95

#13 0x000055fcbe7be907 in std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul> (this=0x55fcc1437fb8) at /usr/include/c++/10/thread:264

#14 0x000055fcbe7be8a0 in std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator() (this=0x55fcc1437fb8) at /usr/include/c++/10/thread:271

#15 0x000055fcbe7be880 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run (this=0x55fcc1437fb0) at /usr/include/c++/10/thread:215

#16 0x00007f95185bc694 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

#17 0x00007f951870b450 in start_thread (arg=0x7f94e37fe640) at pthread_create.c:473

#18 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 12 (Thread 0x7f9510100640 (LWP 3287504)):

#0  0x00007f95181ccdb2 in __GI___sigtimedwait (set=0x7f95100ffc70, info=0x7f95100ffb90, timeout=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:54

#1  0x000055fcbda65c49 in my_sigwait (set=0x7f95100ffc70, sig=0x7f95100ffc34, code=0x7f95100ffc38) at /home/jan/mysql/10.6e-galera/include/my_pthread.h:195

#2  0x000055fcbda6bec6 in signal_hand (arg=0x0) at /home/jan/mysql/10.6e-galera/sql/mysqld.cc:3121

#3  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc12f9628) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#4  0x00007f951870b450 in start_thread (arg=0x7f9510100640) at pthread_create.c:473

#5  0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 11 (Thread 0x7f94f1c5e640 (LWP 3287842)):

#0  0x00007f95182635df in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=req@entry=0x7f94f1c5c2f0, rem=rem@entry=0x0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:73

#1  0x00007f9518268ac7 in __GI___nanosleep (req=req@entry=0x7f94f1c5c2f0, rem=rem@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:25

#2  0x00007f951829a3c9 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:32

#3  0x000055fcbe8faf1b in wsrep::client_state::poll_enter_toi (this=0x7f94bc007558, lock=..., keys=std::vector of length 2, capacity 2 = {...}, buffer=..., meta=..., flags=3, wait_until=..., timed_out=@0x7f94f1c5c5c7: false) at /home/jan/mysql/10.6e-galera/wsrep-lib/src/client_state.cpp:404

#4  0x000055fcbe8fb224 in wsrep::client_state::enter_toi_local (this=0x7f94bc007558, keys=std::vector of length 2, capacity 2 = {...}, buffer=..., wait_until=...) at /home/jan/mysql/10.6e-galera/wsrep-lib/src/client_state.cpp:438

#5  0x000055fcbe353ee7 in wsrep_TOI_begin (thd=0x7f94bc000db8, db=0x7f94bc014a60 "test", table=0x7f94bc014330 "t1", table_list=0x7f94bc014368, alter_info=0x0, fk_tables=0x0, create_info=0x7f94f1c5c900) at /home/jan/mysql/10.6e-galera/sql/wsrep_mysqld.cc:2659

#6  0x000055fcbe35621d in wsrep_to_isolation_begin (thd=0x7f94bc000db8, db_=0x7f94bc014a60 "test", table_=0x7f94bc014330 "t1", table_list=0x7f94bc014368, alter_info=0x0, fk_tables=0x0, create_info=0x7f94f1c5c900) at /home/jan/mysql/10.6e-galera/sql/wsrep_mysqld.cc:3168

#7  0x000055fcbdcd5185 in Sql_cmd_create_table_like::execute (this=0x7f94bc014308, thd=0x7f94bc000db8) at /home/jan/mysql/10.6e-galera/sql/sql_table.cc:11798

#8  0x000055fcbdbcac79 in mysql_execute_command (thd=0x7f94bc000db8, is_called_from_prepared_stmt=false) at /home/jan/mysql/10.6e-galera/sql/sql_parse.cc:6003

#9  0x000055fcbdbd10f8 in mysql_parse (thd=0x7f94bc000db8, rawbuf=0x7f94bc014250 "CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB", length=42, parser_state=0x7f94f1c5d390) at /home/jan/mysql/10.6e-galera/sql/sql_parse.cc:8057

#10 0x000055fcbdbd0685 in wsrep_mysql_parse (thd=0x7f94bc000db8, rawbuf=0x7f94bc014250 "CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB", length=42, parser_state=0x7f94f1c5d390) at /home/jan/mysql/10.6e-galera/sql/sql_parse.cc:7871

#11 0x000055fcbdbbd006 in dispatch_command (command=COM_QUERY, thd=0x7f94bc000db8, packet=0x7f94bc00b8b9 "CREATE TABLE t1 (f1 INTEGER) ENGINE=InnoDB", packet_length=42, blocking=true) at /home/jan/mysql/10.6e-galera/sql/sql_parse.cc:1890

#12 0x000055fcbdbbba67 in do_command (thd=0x7f94bc000db8, blocking=true) at /home/jan/mysql/10.6e-galera/sql/sql_parse.cc:1411

#13 0x000055fcbdd7a780 in do_handle_one_connection (connect=0x55fcc18a8258, put_in_cache=true) at /home/jan/mysql/10.6e-galera/sql/sql_connect.cc:1418

#14 0x000055fcbdd7a413 in handle_one_connection (arg=0x55fcc18a8258) at /home/jan/mysql/10.6e-galera/sql/sql_connect.cc:1312

#15 0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc122e098) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#16 0x00007f951870b450 in start_thread (arg=0x7f94f1c5e640) at pthread_create.c:473

#17 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 10 (Thread 0x7f9510e7a640 (LWP 3287492)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55fcc1244134) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcc1244134, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711540 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55fcc1244650, cond=0x55fcc1244108) at pthread_cond_wait.c:504

#3  __pthread_cond_wait (cond=0x55fcc1244108, mutex=0x55fcc1244650) at pthread_cond_wait.c:628

#4  0x000055fcbe865531 in safe_cond_wait (cond=0x55fcc1244108, mp=0x55fcc1244628, file=0x55fcbedb8170 "/home/jan/mysql/10.6e-galera/mysys/my_thr_init.c", line=596) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:492

#5  0x000055fcbe85e7b4 in psi_cond_wait (that=0x55fcc1244108, mutex=0x55fcc1244628, file=0x55fcbec63988 "/home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc", line=495) at /home/jan/mysql/10.6e-galera/mysys/my_thr_init.c:596

#6  0x000055fcbe384568 in inline_mysql_cond_wait (that=0x55fcc1244108, mutex=0x55fcc1244628, src_file=0x55fcbec63988 "/home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc", src_line=495) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1070

#7  0x000055fcbe3856f2 in Wsrep_thread_service::wait (this=0x55fcc10e93d0, cond=0x55fcc1244108, mutex=0x55fcc1244628) at /home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc:495

#8  0x000055fcbe926313 in wsrep_thread_service_v1::cond_wait_cb (cond=0x55fcc1244108, mutex=0x55fcc1244628) at /home/jan/mysql/10.6e-galera/wsrep-lib/src/thread_service_v1.cpp:186

#9  0x00007f9513213fc7 in gu_cond_wait_SYS (mutex=0x55fcc12440c8, cond=0x55fcc1244108) at /home/jan/mysql/es-galera/galerautils/src/gu_threads.h:249

#10 gu::Lock::wait (this=<synthetic pointer>, this=<synthetic pointer>, cond=...) at /home/jan/mysql/es-galera/galerautils/src/gu_lock.hpp:46

#11 RecvBuf::front (timeout=..., this=0x55fcc12440c8) at /home/jan/mysql/es-galera/gcs/src/gcs_gcomm.cpp:116

#12 gcomm_recv (backend=<optimized out>, msg=0x55fcc1213488, timeout=9223372035999999999) at /home/jan/mysql/es-galera/gcs/src/gcs_gcomm.cpp:613

#13 0x00007f951320196c in core_msg_recv (timeout=9223372035999999999, recv_msg=0x55fcc1213488, backend=0x55fcc1213630) at /home/jan/mysql/es-galera/gcs/src/gcs_core.cpp:484

#14 gcs_core_recv (conn=0x55fcc1213410, recv_act=recv_act@entry=0x7f9510e79b80, timeout=<optimized out>) at /home/jan/mysql/es-galera/gcs/src/gcs_core.cpp:1205

#15 0x00007f95131fc3be in gcs_recv_thread (arg=<optimized out>) at /home/jan/mysql/es-galera/gcs/src/gcs.cpp:1418

#16 0x000055fcbe383c6e in thread_start_fn (args_ptr=0x55fcc1253130) at /home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc:186

#17 0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc122b6b8) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#18 0x00007f951870b450 in start_thread (arg=0x7f9510e7a640) at pthread_create.c:473

#19 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 9 (Thread 0x7f9518cf1640 (LWP 3287488)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x7f9518cf0cf0, clockid=0, expected=0, futex_word=0x55fcbfe060a8 <COND_timer+40>) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcbfe060a8 <COND_timer+40>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7f9518cf0cf0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f951871199e in __pthread_cond_wait_common (abstime=0x7f9518cf0cf0, clockid=0, mutex=0x55fcbfe05fe8 <LOCK_timer+40>, cond=0x55fcbfe06080 <COND_timer>) at pthread_cond_wait.c:504

#3  __pthread_cond_timedwait (cond=0x55fcbfe06080 <COND_timer>, mutex=0x55fcbfe05fe8 <LOCK_timer+40>, abstime=0x7f9518cf0cf0) at pthread_cond_wait.c:646

#4  0x000055fcbe8658f3 in safe_cond_timedwait (cond=0x55fcbfe06080 <COND_timer>, mp=0x55fcbfe05fc0 <LOCK_timer>, abstime=0x7f9518cf0cf0, file=0x55fcbedb8170 "/home/jan/mysql/10.6e-galera/mysys/my_thr_init.c", line=609) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:546

#5  0x000055fcbe85e867 in psi_cond_timedwait (that=0x55fcbfe06080 <COND_timer>, mutex=0x55fcbfe05fc0 <LOCK_timer>, abstime=0x7f9518cf0cf0, file=0x55fcbedb9260 "/home/jan/mysql/10.6e-galera/mysys/thr_timer.c", line=321) at /home/jan/mysql/10.6e-galera/mysys/my_thr_init.c:609

#6  0x000055fcbe8668b5 in inline_mysql_cond_timedwait (that=0x55fcbfe06080 <COND_timer>, mutex=0x55fcbfe05fc0 <LOCK_timer>, abstime=0x7f9518cf0cf0, src_file=0x55fcbedb9260 "/home/jan/mysql/10.6e-galera/mysys/thr_timer.c", src_line=321) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1086

#7  0x000055fcbe86757c in timer_handler (arg=0x0) at /home/jan/mysql/10.6e-galera/mysys/thr_timer.c:321

#8  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc11dbd28) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#9  0x00007f951870b450 in start_thread (arg=0x7f9518cf1640) at pthread_create.c:473

#10 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 8 (Thread 0x7f94f1a5e640 (LWP 3287499)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55fcbf35e0e8 <buf_pool+17576>) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcbf35e0e8 <buf_pool+17576>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711540 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55fcbf35dfe8 <buf_pool+17320>, cond=0x55fcbf35e0c0 <buf_pool+17536>) at pthread_cond_wait.c:504

#3  __pthread_cond_wait (cond=0x55fcbf35e0c0 <buf_pool+17536>, mutex=0x55fcbf35dfe8 <buf_pool+17320>) at pthread_cond_wait.c:628

#4  0x000055fcbe865531 in safe_cond_wait (cond=0x55fcbf35e0c0 <buf_pool+17536>, mp=0x55fcbf35dfc0 <buf_pool+17280>, file=0x55fcbed4d390 "/home/jan/mysql/10.6e-galera/storage/innobase/buf/buf0flu.cc", line=2215) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:492

#5  0x000055fcbe6be530 in buf_flush_page_cleaner () at /home/jan/mysql/10.6e-galera/storage/innobase/buf/buf0flu.cc:2215

#6  0x000055fcbe6c1eed in std::__invoke_impl<void, void (*)()> (__f=@0x55fcc12533b8: 0x55fcbe6be1e9 <buf_flush_page_cleaner()>) at /usr/include/c++/10/bits/invoke.h:60

#7  0x000055fcbe6c1e87 in std::__invoke<void (*)()> (__fn=@0x55fcc12533b8: 0x55fcbe6be1e9 <buf_flush_page_cleaner()>) at /usr/include/c++/10/bits/invoke.h:95

#8  0x000055fcbe6c1e28 in std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul> (this=0x55fcc12533b8) at /usr/include/c++/10/thread:264

#9  0x000055fcbe6c1df8 in std::thread::_Invoker<std::tuple<void (*)()> >::operator() (this=0x55fcc12533b8) at /usr/include/c++/10/thread:271

#10 0x000055fcbe6c1dd8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run (this=0x55fcc12533b0) at /usr/include/c++/10/thread:215

#11 0x00007f95185bc694 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

#12 0x00007f951870b450 in start_thread (arg=0x7f94f1a5e640) at pthread_create.c:473

#13 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 7 (Thread 0x7f951006a640 (LWP 3287509)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=-1092913163, abstime=0x7f9510069c50, clockid=32661, expected=0, futex_word=0x55fcbf51d8c8 <thread_cache+40>) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcbf51d8c8 <thread_cache+40>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7f9510069c50, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f951871199e in __pthread_cond_wait_common (abstime=0x7f9510069c50, clockid=0, mutex=0x55fcbf51d938 <thread_cache+152>, cond=0x55fcbf51d8a0 <thread_cache>) at pthread_cond_wait.c:504

#3  __pthread_cond_timedwait (cond=0x55fcbf51d8a0 <thread_cache>, mutex=0x55fcbf51d938 <thread_cache+152>, abstime=0x7f9510069c50) at pthread_cond_wait.c:646

#4  0x000055fcbe8658f3 in safe_cond_timedwait (cond=0x55fcbf51d8a0 <thread_cache>, mp=0x55fcbf51d910 <thread_cache+112>, abstime=0x7f9510069c50, file=0x55fcbedb8170 "/home/jan/mysql/10.6e-galera/mysys/my_thr_init.c", line=609) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:546

#5  0x000055fcbe85e867 in psi_cond_timedwait (that=0x55fcbf51d8a0 <thread_cache>, mutex=0x55fcbf51d910 <thread_cache+112>, abstime=0x7f9510069c50, file=0x55fcbe9bf250 "/home/jan/mysql/10.6e-galera/sql/thread_cache.h", line=176) at /home/jan/mysql/10.6e-galera/mysys/my_thr_init.c:609

#6  0x000055fcbdd7afc1 in inline_mysql_cond_timedwait (that=0x55fcbf51d8a0 <thread_cache>, mutex=0x55fcbf51d910 <thread_cache+112>, abstime=0x7f9510069c50, src_file=0x55fcbe9bf250 "/home/jan/mysql/10.6e-galera/sql/thread_cache.h", src_line=176) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1086

#7  0x000055fcbdd7b1e8 in Thread_cache::park (this=0x55fcbf51d8a0 <thread_cache>) at /home/jan/mysql/10.6e-galera/sql/thread_cache.h:176

#8  0x000055fcbdd7a80c in do_handle_one_connection (connect=0x55fcc18a8698, put_in_cache=true) at /home/jan/mysql/10.6e-galera/sql/sql_connect.cc:1431

#9  0x000055fcbdd7a413 in handle_one_connection (arg=0x55fcc1845e48) at /home/jan/mysql/10.6e-galera/sql/sql_connect.cc:1312

#10 0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc189de88) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#11 0x00007f951870b450 in start_thread (arg=0x7f951006a640) at pthread_create.c:473

#12 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 6 (Thread 0x7f94f2c5f640 (LWP 3287498)):

#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38

#1  0x00007f9518c05d14 in ?? () from /lib/x86_64-linux-gnu/liburing.so.1

#2  0x00007f9518c058e7 in __io_uring_get_cqe () from /lib/x86_64-linux-gnu/liburing.so.1

#3  0x000055fcbe7bf873 in io_uring_wait_cqe_nr (ring=0x55fcc11ab3d8, cqe_ptr=0x7f94f2c5ec50, wait_nr=1) at /usr/include/liburing.h:494

#4  0x000055fcbe7bf89d in io_uring_wait_cqe (ring=0x55fcc11ab3d8, cqe_ptr=0x7f94f2c5ec50) at /usr/include/liburing.h:514

#5  0x000055fcbe7bffe9 in (anonymous namespace)::aio_uring::thread_routine (aio=0x55fcc11ab3d0) at /home/jan/mysql/10.6e-galera/tpool/aio_liburing.cc:137

#6  0x000055fcbe7c06cc in std::__invoke_impl<void, void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> (__f=@0x55fcc1243040: 0x55fcbe7bffc2 <(anonymous namespace)::aio_uring::thread_routine((anonymous namespace)::aio_uring*)>) at /usr/include/c++/10/bits/invoke.h:60

#7  0x000055fcbe7c0633 in std::__invoke<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> (__fn=@0x55fcc1243040: 0x55fcbe7bffc2 <(anonymous namespace)::aio_uring::thread_routine((anonymous namespace)::aio_uring*)>) at /usr/include/c++/10/bits/invoke.h:95

#8  0x000055fcbe7c05a3 in std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::_M_invoke<0, 1> (this=0x55fcc1243038) at /usr/include/c++/10/thread:264

#9  0x000055fcbe7c055c in std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::operator() (this=0x55fcc1243038) at /usr/include/c++/10/thread:271

#10 0x000055fcbe7c0540 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> > >::_M_run (this=0x55fcc1243030) at /usr/include/c++/10/thread:215

#11 0x00007f95185bc694 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

#12 0x00007f951870b450 in start_thread (arg=0x7f94f2c5f640) at pthread_create.c:473

#13 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 5 (Thread 0x7f94f3861640 (LWP 3287495)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=-1092913440, abstime=0x7f94f3860c40, clockid=22012, expected=0, futex_word=0x55fcbfd5f7a8 <COND_checkpoint+40>) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcbfd5f7a8 <COND_checkpoint+40>, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x7f94f3860c40, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f951871199e in __pthread_cond_wait_common (abstime=0x7f94f3860c40, clockid=0, mutex=0x55fcbfd5f6e8 <LOCK_checkpoint+40>, cond=0x55fcbfd5f780 <COND_checkpoint>) at pthread_cond_wait.c:504

#3  __pthread_cond_timedwait (cond=0x55fcbfd5f780 <COND_checkpoint>, mutex=0x55fcbfd5f6e8 <LOCK_checkpoint+40>, abstime=0x7f94f3860c40) at pthread_cond_wait.c:646

#4  0x000055fcbe8658f3 in safe_cond_timedwait (cond=0x55fcbfd5f780 <COND_checkpoint>, mp=0x55fcbfd5f6c0 <LOCK_checkpoint>, abstime=0x7f94f3860c40, file=0x55fcbebd7940 "/home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h", line=1088) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:546

#5  0x000055fcbe200ce7 in inline_mysql_cond_timedwait (that=0x55fcbfd5f780 <COND_checkpoint>, mutex=0x55fcbfd5f6c0 <LOCK_checkpoint>, abstime=0x7f94f3860c40, src_file=0x55fcbebd7980 "/home/jan/mysql/10.6e-galera/storage/maria/ma_servicethread.c", src_line=115) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1088

#6  0x000055fcbe2011d9 in my_service_thread_sleep (control=0x55fcbf3439a0 <checkpoint_control>, sleep_time=30000000000) at /home/jan/mysql/10.6e-galera/storage/maria/ma_servicethread.c:115

#7  0x000055fcbe1f4e60 in ma_checkpoint_background (arg=0x1e) at /home/jan/mysql/10.6e-galera/storage/maria/ma_checkpoint.c:707

#8  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc12f64a8) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#9  0x00007f951870b450 in start_thread (arg=0x7f94f3861640) at pthread_create.c:473

#10 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 (Thread 0x7f951062e640 (LWP 3287494)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x7f95127010fc) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7f95127010fc, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711540 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7f9512701090, cond=0x7f95127010d0) at pthread_cond_wait.c:504

#3  __pthread_cond_wait (cond=cond@entry=0x7f95127010d0, mutex=mutex@entry=0x7f9512701090) at pthread_cond_wait.c:628

#4  0x00007f9513320369 in gu_cond_wait_SYS (mutex=0x7f9512701090, cond=0x7f95127010d0) at /home/jan/mysql/es-galera/galerautils/src/gu_threads.h:251

#5  fifo_lock_get (q=0x7f9512701010) at /home/jan/mysql/es-galera/galerautils/src/gu_fifo.c:262

#6  gu_fifo_get_head (q=0x7f9512701010, err=err@entry=0x7f951062d584) at /home/jan/mysql/es-galera/galerautils/src/gu_fifo.c:330

#7  0x00007f95131f6407 in gcs_recv (conn=0x55fcc1213130, action=action@entry=0x7f951062d5f0) at /home/jan/mysql/es-galera/gcs/src/gcs.cpp:2064

#8  0x00007f95131dfc3a in galera::Gcs::recv (act=..., this=<optimized out>) at /home/jan/mysql/es-galera/galera/src/galera_gcs.hpp:127

#9  galera::GcsActionSource::process (this=0x55fcc1239a90, recv_ctx=0x7f951062dc20, exit_loop=@0x7f951062d7ff: false) at /home/jan/mysql/es-galera/galera/src/gcs_action_source.cpp:165

#10 0x00007f95131b1f60 in galera::ReplicatorSMM::async_recv (this=0x55fcc1209440, recv_ctx=0x7f951062dc20) at /home/jan/mysql/es-galera/galera/src/replicator_smm.cpp:394

#11 0x00007f951318af01 in galera_recv (gh=<optimized out>, recv_ctx=<optimized out>) at /home/jan/mysql/es-galera/galera/src/wsrep_provider.cpp:261

#12 0x000055fcbe91f8a8 in wsrep::wsrep_provider_v26::run_applier (this=0x55fcc10de010, applier_ctx=0x7f951062dc20) at /home/jan/mysql/10.6e-galera/wsrep-lib/src/wsrep_provider_v26.cpp:796

#13 0x000055fcbe36c8e7 in wsrep_replication_process (thd=0x7f94f8000ff8, arg=0x55fcc1252500) at /home/jan/mysql/10.6e-galera/sql/wsrep_thd.cc:57

#14 0x000055fcbe359691 in start_wsrep_THD (arg=0x55fcc1252500) at /home/jan/mysql/10.6e-galera/sql/wsrep_mysqld.cc:3863

#15 0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc122e098) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#16 0x00007f951870b450 in start_thread (arg=0x7f951062e640) at pthread_create.c:473

#17 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 3 (Thread 0x7f95125ff640 (LWP 3287490)):

#0  __futex_abstimed_wait_common64 (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0x55fcc1209d6c) at ../sysdeps/nptl/futex-internal.c:74

#1  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x55fcc1209d6c, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ../sysdeps/nptl/futex-internal.c:123

#2  0x00007f9518711540 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x55fcc123a1f0, cond=0x55fcc1209d40) at pthread_cond_wait.c:504

#3  __pthread_cond_wait (cond=0x55fcc1209d40, mutex=0x55fcc123a1f0) at pthread_cond_wait.c:628

#4  0x000055fcbe865531 in safe_cond_wait (cond=0x55fcc1209d40, mp=0x55fcc123a1c8, file=0x55fcbedb8170 "/home/jan/mysql/10.6e-galera/mysys/my_thr_init.c", line=596) at /home/jan/mysql/10.6e-galera/mysys/thr_mutex.c:492

#5  0x000055fcbe85e7b4 in psi_cond_wait (that=0x55fcc1209d40, mutex=0x55fcc123a1c8, file=0x55fcbec63988 "/home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc", line=495) at /home/jan/mysql/10.6e-galera/mysys/my_thr_init.c:596

#6  0x000055fcbe384568 in inline_mysql_cond_wait (that=0x55fcc1209d40, mutex=0x55fcc123a1c8, src_file=0x55fcbec63988 "/home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc", src_line=495) at /home/jan/mysql/10.6e-galera/include/mysql/psi/mysql_thread.h:1070

#7  0x000055fcbe3856f2 in Wsrep_thread_service::wait (this=0x55fcc10e93d0, cond=0x55fcc1209d40, mutex=0x55fcc123a1c8) at /home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc:495

#8  0x000055fcbe926313 in wsrep_thread_service_v1::cond_wait_cb (cond=0x55fcc1209d40, mutex=0x55fcc123a1c8) at /home/jan/mysql/10.6e-galera/wsrep-lib/src/thread_service_v1.cpp:186

#9  0x00007f95131d9bf2 in gu_cond_wait_SYS (mutex=0x55fcc1209d00, cond=0x55fcc1209d40) at /home/jan/mysql/es-galera/galerautils/src/gu_threads.h:249

#10 gu::Lock::wait (this=<synthetic pointer>, this=<synthetic pointer>, cond=...) at /home/jan/mysql/es-galera/galerautils/src/gu_lock.hpp:46

#11 galera::ServiceThd::thd_func (arg=0x55fcc1209ce0) at /home/jan/mysql/es-galera/galera/src/galera_service_thd.cpp:31

#12 0x000055fcbe383c6e in thread_start_fn (args_ptr=0x55fcc1212570) at /home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc:186

#13 0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc123a318) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#14 0x00007f951870b450 in start_thread (arg=0x7f95125ff640) at pthread_create.c:473

#15 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7f951167b640 (LWP 3287491)):

#0  0x00007f95182a309e in epoll_wait (epfd=6, events=events@entry=0x7f951167a2a0, maxevents=maxevents@entry=128, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30

#1  0x00007f95132f6a93 in asio::detail::epoll_reactor::run (this=0x55fcc1200040, block=block@entry=true, ops=...) at /home/jan/mysql/es-galera/asio/asio/detail/impl/epoll_reactor.ipp:391

#2  0x00007f95132f7541 in asio::detail::task_io_service::do_run_one (this=this@entry=0x55fcc1240d20, lock=..., this_thread=..., ec=...) at /home/jan/mysql/es-galera/asio/asio/detail/impl/task_io_service.ipp:355

#3  0x00007f95132f1081 in asio::detail::task_io_service::run (ec=..., this=0x55fcc1240d20) at /home/jan/mysql/es-galera/asio/asio/detail/impl/task_io_service.ipp:148

#4  asio::io_service::run (this=<optimized out>, this=<optimized out>) at /home/jan/mysql/es-galera/asio/asio/impl/io_service.ipp:58

#5  gu::AsioIoService::run (this=<optimized out>) at /home/jan/mysql/es-galera/galerautils/src/gu_asio.cpp:713

#6  0x00007f9513218a3e in GCommConn::run (this=0x55fcc1243eb0) at /home/jan/mysql/es-galera/gcs/src/gcs_gcomm.cpp:438

#7  0x00007f951321905a in run_fn (arg=<optimized out>) at /home/jan/mysql/es-galera/gcs/src/gcs_gcomm.cpp:306

#8  0x000055fcbe383c6e in thread_start_fn (args_ptr=0x55fcc1244780) at /home/jan/mysql/10.6e-galera/sql/wsrep_thread_service.cc:186

#9  0x000055fcbe2b1c44 in pfs_spawn_thread (arg=0x55fcc1244828) at /home/jan/mysql/10.6e-galera/storage/perfschema/pfs.cc:2201

#10 0x00007f951870b450 in start_thread (arg=0x7f951167b640) at pthread_create.c:473

#11 0x00007f95182a2d53 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f9517eb6080 (LWP 3287487)):

#0  __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:54

#1  0x000055fcbe860326 in my_write_core (sig=6) at /home/jan/mysql/10.6e-galera/mysys/stacktrace.c:424

#2  0x000055fcbdf5de70 in handle_fatal_signal (sig=6) at /home/jan/mysql/10.6e-galera/sql/signal_handler.cc:344

#3  <signal handler called>

#4  0x00007f951829722f in __GI___poll (fds=0x55fcc18981e8, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29

#5  0x000055fcbda721e4 in handle_connections_sockets () at /home/jan/mysql/10.6e-galera/sql/mysqld.cc:6140

#6  0x000055fcbda717ca in mysqld_main (argc=170, argv=0x55fcc10af100) at /home/jan/mysql/10.6e-galera/sql/mysqld.cc:5845

#7  0x000055fcbda65bad in main (argc=23, argv=0x7ffd77ffd838) at /home/jan/mysql/10.6e-galera/sql/main.cc:34

 - saving '/home/jan/mysql/10.6e-galera/mysql-test/var/log/galera.galera_split_brain-innodb/' to '/home/jan/mysql/10.6e-galera/mysql-test/var/log/galera.galera_split_brain-innodb/'

--------------------------------------------------------------------------

Attachments

Activity

People

Assignee:: Jan Lindström (Inactive)

Reporter:: Jan Lindström (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 2021-08-05 09:42

Updated:: 6 days ago 12:49

Resolved:: 2021-08-18 11:19

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.