Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Cannot Reproduce
-
10.4.18
-
None
-
kvm-deb-buster-amd64
Description
galera.galera_bf_kill failed on BB 10.4 CS: Crash with SIGSEGV.
It seems to be a sporadic issue.
|
10.4.18, 542d769ea1a22a7a6a87c9fe76ff911a162ade44, kvm-deb-buster-amd64 |
galera.galera_bf_kill 'innodb' w1 [ fail ]
|
Test ended at 2021-01-28 04:25:56
|
|
|
CURRENT_TEST: galera.galera_bf_kill
|
|
|
|
|
The server [mysqld.2 - pid: 13514, winpid: 13514, exit: 256] crashed while running 'check testcase after test'
|
Server log from this test:
|
----------SERVER LOG START-----------
|
2021-01-28 4:25:48 0 [Warning] WSREP: option --wsrep-causal-reads is deprecated
|
2021-01-28 4:25:48 0 [Note] /usr/sbin/mysqld (mysqld 10.4.18-MariaDB-1:10.4.18+maria~buster-log) starting as process 13515 ...
|
2021-01-28 4:25:48 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 32186)
|
2021-01-28 4:25:48 0 [Warning] Changed limits: max_open_files: 1024 max_connections: 151 (was 151) table_cache: 421 (was 2000)
|
2021-01-28 4:25:48 0 [Note] WSREP: Server initial position: 00000000-0000-0000-0000-000000000000:-1
|
2021-01-28 4:25:48 0 [Note] WSREP: Loading provider /usr/lib/galera/libgalera_smm.so initial position: 00000000-0000-0000-0000-000000000000:-1
|
2021-01-28 4:25:48 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
|
2021-01-28 4:25:48 0 [Note] WSREP: wsrep_load(): Galera 4.7(ree4f10fc) by Codership Oy <info@codership.com> loaded successfully.
|
2021-01-28 4:25:48 0 [Note] WSREP: CRC-32C: using "slicing-by-8" algorithm.
|
2021-01-28 4:25:48 0 [Warning] WSREP: Could not open state file for reading: '/dev/shm/var/1/mysqld.2/data//grastate.dat'
|
2021-01-28 4:25:48 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 1
|
2021-01-28 4:25:48 0 [Note] WSREP: GCache DEBUG: opened preamble:
|
Version: 0
|
UUID: 00000000-0000-0000-0000-000000000000
|
Seqno: -1 - -1
|
Offset: -1
|
Synced: 0
|
2021-01-28 4:25:48 0 [Note] WSREP: Skipped GCache ring buffer recovery: could not determine history UUID.
|
2021-01-28 4:25:48 0 [Note] WSREP: Passing config to GCS: base_dir = /dev/shm/var/1/mysqld.2/data/; base_host = 127.0.0.1; base_port = 16005; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT30S; evs.install_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT10S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /dev/shm/var/1/mysqld.2/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast
|
|
|
[...]
|
|
|
2021-01-28 4:25:55 1 [Note] WSREP: abort transaction: BF: drop table t1 victim: NULL victim conf: executing
|
2021-01-28 4:25:55 1 [Note] WSREP: cluster conflict due to high priority abort for threads:
|
2021-01-28 4:25:55 1 [Note] WSREP: Winning thread:
|
THD: 1, mode: toi, state: exec, conflict: executing, seqno: 10
|
SQL: drop table t1
|
2021-01-28 4:25:55 1 [Note] WSREP: Victim thread:
|
THD: 22, mode: local, state: idle, conflict: executing, seqno: -1
|
SQL: NULL
|
2021-01-28 4:25:55 1 [Note] WSREP: context: /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/handler/ha_innodb.cc:18602
|
2021-01-28 4:25:55 1 [Note] WSREP: Aborter BF trx_id: 18446744073709551615 thread: 1 seqno: 10 client_state: exec client_mode: toi transaction_mode: executing query: drop table t1
|
2021-01-28 4:25:55 1 [Note] WSREP: Victim normal trx_id: 85 thread: 22 seqno: -1 client_state: idle client_mode: local transaction_mode: executing query: NULL
|
2021-01-28 4:25:55 1 [Note] WSREP: wsrep_thd_set_wsrep_aborter called
|
2021-01-28 4:25:55 1 [Note] WSREP: wsrep_bf_abort BF aborter before
|
thd: 1 thd_ptr: 0x7fc78c000c08 client_mode: toi client_state: exec trx_state: executing
|
next_trx_id: 73 trx_id: -1 seqno: 10
|
is_streaming: 0 fragments: 0
|
sql_errno: 0 message:
|
command: 9 query: drop table t1
|
2021-01-28 4:25:55 1 [Note] WSREP: wsrep_bf_abort victim before
|
thd: 22 thd_ptr: 0x7fc734000c08 client_mode: local client_state: idle trx_state: executing
|
next_trx_id: 66 trx_id: 66 seqno: -1
|
is_streaming: 0 fragments: 0
|
sql_errno: 0 message:
|
command: 4 query: (null)
|
2021-01-28 4:25:55 1 [Note] WSREP: enqueuing trx abort for (22)
|
2021-01-28 4:25:55 22 [Note] WSREP: Rollbacker aborting local thd (22 140495497606912)
|
2021-01-28 4:25:55 0 [Note] WSREP: rollbacker aborted thd: (22 140495497606912)
|
2021-01-28 4:25:55 1 [Note] WSREP: Set WSREPXid for InnoDB: ce756f28-614a-11eb-ba77-caebd29b854c:10
|
2021-01-28 4:25:55 22 [Note] WSREP: ha_rollback_trans(22, TRUE) rolled back: (null): Got an error reading communication packets; is_real 1
|
2021-01-28 4:25:55 22 [Note] WSREP: ha_rollback_trans(22, TRUE) rolled back: (null): Got an error reading communication packets; is_real 1
|
2021-01-28 4:25:55 15 [Note] WSREP: assigned new next trx id: 74
|
2021-01-28 4:25:56 23 [Note] WSREP: assigned new next trx id: 75
|
2021-01-28 4:25:56 23 [Note] WSREP: wsrep_thd_binlog_reset
|
2021-01-28 4:25:56 23 [Note] WSREP: assigned new next trx id: 76
|
210128 4:25:56 [ERROR] mysqld got signal 11 ;
|
This could be because you hit a bug. It is also possible that this binary
|
or one of the libraries it was linked against is corrupt, improperly built,
|
or misconfigured. This error can also be caused by malfunctioning hardware.
|
|
|
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
|
|
|
We will try our best to scrape up some info that will hopefully help
|
diagnose the problem, but since we have already crashed,
|
something is definitely wrong and this may fail.
|
|
|
Server version: 10.4.18-MariaDB-1:10.4.18+maria~buster-log
|
key_buffer_size=1048576
|
read_buffer_size=131072
|
max_used_connections=3
|
max_threads=153
|
thread_count=11
|
It is possible that mysqld could use up to
|
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63536 K bytes of memory
|
Hope that's ok; if not, decrease some variables in the equation.
|
|
|
Thread pointer: 0x7fc734000c08
|
Attempting backtrace. You can use the following information to find out
|
where mysqld died. If you see no messages after this, something went
|
terribly wrong...
|
stack_bottom = 0x7fc79883ada8 thread_stack 0x49000
|
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5596299e849e]
|
/usr/sbin/mysqld(handle_fatal_signal+0x54d)[0x5596294dc08d]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7fc7b1086730]
|
/usr/sbin/mysqld(_Z16thd_clear_errorsP3THD+0x1e)[0x55962928168e]
|
/usr/sbin/mysqld(_ZN3THD11change_userEv+0x3b)[0x5596292860bb]
|
/usr/sbin/mysqld(_ZN3THD15reset_for_reuseEv+0x19)[0x559629286289]
|
/usr/sbin/mysqld(_ZN7CONNECT10create_thdEP3THD+0x24)[0x5596293bc614]
|
/usr/sbin/mysqld(_Z29one_thread_per_connection_endP3THDb+0x2fd)[0x5596292100bd]
|
/usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x1bb)[0x5596293bc9bb]
|
/usr/sbin/mysqld(handle_one_connection+0x3d)[0x5596293bcb2d]
|
/usr/sbin/mysqld(+0xd75f8b)[0x559629998f8b]
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3)[0x7fc7b107bfa3]
|
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fc7b05904cf]
|
|
|
Trying to get some variables.
|
Some pointers may be invalid and cause the dump to abort.
|
Query (0x0): (null)
|
Connection ID (thread ID): 22
|
Status: NOT_KILLED
|
|
|
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on
|
|
|
The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
|
information that should help you find out what is causing the crash.
|
|
|
We think the query pointer is invalid, but we will try to print it anyway.
|
Query:
|
|
|
Writing a core file...
|
Working directory at /dev/shm/var/1/mysqld.2/data
|
Resource Limits:
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size unlimited unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 23734 23734 processes
|
Max open files 1024 1024 files
|
Max locked memory 67108864 67108864 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 23734 23734 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Core pattern: core
|
|
|
----------SERVER LOG END-------------
|
mysqltest failed but provided no output
|
The result from queries just before the failure was:
|
< snip >
|
begin;
|
update t1 set a =5, b=2;
|
connection node_2;
|
ALTER TABLE t1 ADD UNIQUE KEY b3(b);
|
select * from t1;
|
a b
|
2 1
|
disconnect node_2a;
|
connect node_2a, 127.0.0.1, root, , test, $NODE_MYPORT_2;
|
connection node_2a;
|
SET SESSION wsrep_on=OFF;
|
begin;
|
update t1 set a =5, b=2;
|
connection node_2;
|
select * from t1;
|
a b
|
2 1
|
disconnect node_2a;
|
connection node_1;
|
drop table t1;
|
|
|
|
|
|
|
- found 'core' (0/0)
|
|
|
Trying 'dbx' to get a backtrace
|
|
|
Trying 'gdb' to get a backtrace from coredump /dev/shm/var/1/log/galera.galera_bf_kill-innodb/mysqld.2/data/core
|
Core generated by '/usr/sbin/mysqld'
|
Output from gdb follows. The first stack trace is from the failing thread.
|
The following stack traces are from all threads (so the failing one is
|
duplicated).
|
--------------------------
|
[New LWP 13916]
|
[New LWP 13889]
|
[New LWP 13515]
|
[New LWP 13881]
|
[New LWP 13886]
|
[New LWP 13900]
|
[New LWP 13883]
|
[New LWP 13880]
|
[New LWP 13901]
|
[New LWP 13879]
|
[New LWP 13876]
|
[New LWP 13642]
|
[New LWP 13644]
|
[New LWP 13895]
|
[New LWP 13643]
|
[New LWP 13894]
|
[New LWP 13518]
|
[New LWP 13891]
|
[New LWP 13887]
|
[New LWP 13899]
|
[New LWP 13882]
|
[New LWP 13915]
|
[New LWP 13893]
|
[New LWP 13892]
|
[New LWP 13904]
|
[New LWP 13898]
|
[New LWP 13897]
|
[New LWP 13896]
|
[New LWP 13890]
|
[New LWP 13888]
|
[New LWP 13878]
|
[New LWP 13877]
|
[New LWP 13520]
|
[New LWP 13519]
|
[Thread debugging using libthread_db enabled]
|
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
|
Core was generated by `/usr/sbin/mysqld --defaults-group-suffix=.2 --defaults-file=/dev/shm/var/1/my.c'.
|
Program terminated with signal SIGSEGV, Segmentation fault.
|
#0 __pthread_kill (threadid=<optimized out>, signo=signo@entry=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
|
56 ../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
|
[Current thread is 1 (Thread 0x7fc79883b700 (LWP 13916))]
|
#0 __pthread_kill (threadid=<optimized out>, signo=signo@entry=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
|
#1 0x00005596299e8297 in my_write_core (sig=sig@entry=11) at ./mysys/stacktrace.c:386
|
#2 0x00005596294dc0a8 in handle_fatal_signal (sig=11) at ./sql/signal_handler.cc:343
|
#3 <signal handler called>
|
#4 0x000055962928168e in thd_clear_errors (thd=thd@entry=0x7fc734000c08) at ./sql/sql_class.cc:311
|
#5 0x00005596292860bb in THD::change_user (this=this@entry=0x7fc734000c08) at ./sql/sql_class.cc:1422
|
#6 0x0000559629286289 in THD::reset_for_reuse (this=this@entry=0x7fc734000c08) at ./sql/sql_class.cc:1671
|
#7 0x00005596293bc614 in CONNECT::create_thd (this=this@entry=0x55962b50bd48, thd=thd@entry=0x7fc734000c08) at ./sql/sql_connect.cc:1500
|
#8 0x00005596292100bd in cache_thread (thd=0x7fc734000c08) at ./sql/mysqld.cc:2717
|
#9 one_thread_per_connection_end (thd=0x7fc734000c08, put_in_cache=<optimized out>) at ./sql/mysqld.cc:2780
|
#10 0x00005596293bc9bb in do_handle_one_connection (connect=connect@entry=0x55962b5a2d78) at ./sql/sql_connect.cc:1423
|
#11 0x00005596293bcb2d in handle_one_connection (arg=arg@entry=0x55962b5a2d78) at ./sql/sql_connect.cc:1316
|
#12 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b5a2fd8) at ./storage/perfschema/pfs.cc:1869
|
#13 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#14 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 34 (Thread 0x7fc7ab3a9700 (LWP 13519)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b17e0e0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b17e090, cond=0x55962b17e0b8) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b17e0b8, mutex=mutex@entry=0x55962b17e090) at pthread_cond_wait.c:655
|
#3 0x00007fc7ab557bf0 in gu::Lock::wait (this=<synthetic pointer>, cond=...) at ./galerautils/src/gu_mutex.hpp:86
|
#4 galera::ServiceThd::thd_func (arg=0x55962b17e078) at ./galera/src/galera_service_thd.cpp:30
|
#5 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#6 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 33 (Thread 0x7fc7a93a5700 (LWP 13520)):
|
#0 0x00007fc7b05907ef in epoll_wait (epfd=6, events=events@entry=0x7fc7a93a4430, maxevents=maxevents@entry=128, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
|
#1 0x00007fc7ab682c7f in asio::detail::epoll_reactor::run (this=0x55962b177180, block=block@entry=true, ops=...) at ./asio/asio/detail/impl/epoll_reactor.ipp:391
|
#2 0x00007fc7ab67ce64 in asio::detail::task_io_service::do_run_one (ec=..., this_thread=..., lock=..., this=0x55962b1d2950) at ./asio/asio/detail/impl/task_io_service.ipp:355
|
#3 asio::detail::task_io_service::run (ec=..., this=0x55962b1d2950) at ./asio/asio/detail/impl/task_io_service.ipp:148
|
#4 asio::io_service::run (this=<optimized out>) at ./asio/asio/impl/io_service.ipp:58
|
#5 gu::AsioIoService::run (this=this@entry=0x55962b180078) at ./galerautils/src/gu_asio.cpp:634
|
#6 0x00007fc7ab5b07e8 in gcomm::AsioProtonet::event_loop (this=0x55962b17ffc0, period=...) at ./gcomm/src/asio_protonet.cpp:103
|
#7 0x00007fc7ab595114 in GCommConn::run (this=0x55962b151eb0) at ./galerautils/src/gu_datetime.hpp:61
|
#8 0x00007fc7ab59bbf9 in GCommConn::run_fn (arg=<optimized out>) at ./gcs/src/gcs_gcomm.cpp:174
|
#9 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#10 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 32 (Thread 0x7fc773fff700 (LWP 13877)):
|
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
|
#1 0x00007fc7b105127a in ?? () from /usr/lib/x86_64-linux-gnu/libaio.so.1
|
#2 0x00005596296e8c31 in LinuxAIOHandler::collect (this=0x7fc773ffebe0) at ./storage/innobase/os/os0file.cc:1819
|
#3 0x00005596296ed145 in LinuxAIOHandler::poll (this=0x7fc773ffebe0, m1=0x7fc773ffec80, m2=0x7fc773ffec88, request=0x7fc773ffec90) at ./storage/innobase/os/os0file.cc:1980
|
#4 0x00005596296eecb0 in os_aio_linux_handler (request=0x7fc773ffec90, m2=0x7fc773ffec88, m1=0x7fc773ffec80, global_segment=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:805
|
#5 os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7fc773ffec80, m2=m2@entry=0x7fc773ffec88, request=request@entry=0x7fc773ffec90) at ./storage/innobase/os/os0file.cc:5607
|
#6 0x00005596298390a8 in fil_aio_wait (segment=segment@entry=0) at ./storage/innobase/fil/fil0fil.cc:4316
|
#7 0x000055962976fc90 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 31 (Thread 0x7fc7737fe700 (LWP 13878)):
|
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
|
#1 0x00007fc7b105127a in ?? () from /usr/lib/x86_64-linux-gnu/libaio.so.1
|
#2 0x00005596296e8c31 in LinuxAIOHandler::collect (this=0x7fc7737fdbe0) at ./storage/innobase/os/os0file.cc:1819
|
#3 0x00005596296ed145 in LinuxAIOHandler::poll (this=0x7fc7737fdbe0, m1=0x7fc7737fdc80, m2=0x7fc7737fdc88, request=0x7fc7737fdc90) at ./storage/innobase/os/os0file.cc:1980
|
#4 0x00005596296eecb0 in os_aio_linux_handler (request=0x7fc7737fdc90, m2=0x7fc7737fdc88, m1=0x7fc7737fdc80, global_segment=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:805
|
#5 os_aio_handler (segment=segment@entry=1, m1=m1@entry=0x7fc7737fdc80, m2=m2@entry=0x7fc7737fdc88, request=request@entry=0x7fc7737fdc90) at ./storage/innobase/os/os0file.cc:5607
|
#6 0x00005596298390a8 in fil_aio_wait (segment=segment@entry=1) at ./storage/innobase/fil/fil0fil.cc:4316
|
#7 0x000055962976fc90 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 30 (Thread 0x7fc76a7fc700 (LWP 13888)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc76a7fbd80, expected=0, futex_word=0x55962b2d0330) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc76a7fbd80, mutex=0x55962b2d02e0, cond=0x55962b2d0308) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=0x55962b2d0308, mutex=0x55962b2d02e0, abstime=abstime@entry=0x7fc76a7fbd80) at pthread_cond_wait.c:667
|
#3 0x00005596296efde2 in os_event::timed_wait (this=<optimized out>, abstime=0x7fc76a7fbd80) at ./storage/innobase/include/sync0types.h:463
|
#4 0x00005596296eff41 in os_event::wait_time_low (this=0x55962b2d02d0, time_in_usec=<optimized out>, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
|
#5 0x000055962976f050 in srv_monitor_thread () at ./storage/innobase/srv/srv0srv.cc:1690
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 29 (Thread 0x7fc76bfff700 (LWP 13890)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc76bffeb30, expected=0, futex_word=0x55962b42e930) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc76bffeb30, mutex=0x55962b42e8e0, cond=0x55962b42e908) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=0x55962b42e908, mutex=0x55962b42e8e0, abstime=abstime@entry=0x7fc76bffeb30) at pthread_cond_wait.c:667
|
#3 0x00005596296efde2 in os_event::timed_wait (this=<optimized out>, abstime=0x7fc76bffeb30) at ./storage/innobase/include/sync0types.h:463
|
#4 0x00005596296eff41 in os_event::wait_time_low (this=0x55962b42e8d0, time_in_usec=<optimized out>, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
|
#5 0x00005596297a5680 in ib_wqueue_timedwait (wq=wq@entry=0x55962b4b57e0, wait_in_usecs=wait_in_usecs@entry=5000000) at ./storage/innobase/ut/ut0wqueue.cc:154
|
#6 0x000055962986d782 in fts_optimize_thread (arg=0x55962b4b57e0) at ./storage/innobase/fts/fts0opt.cc:2829
|
#7 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#8 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 28 (Thread 0x7fc7627fc700 (LWP 13896)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b2d03a0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b2d0350, cond=0x55962b2d0378) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b2d0378, mutex=mutex@entry=0x55962b2d0350) at pthread_cond_wait.c:655
|
#3 0x00005596296efea1 in os_event::wait (this=0x55962b2d0340) at ./storage/innobase/include/sync0types.h:463
|
#4 os_event::wait_low (this=0x55962b2d0340, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:325
|
#5 0x00005596297ea67b in buf_dump_thread () at ./storage/innobase/buf/buf0dump.cc:832
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 27 (Thread 0x7fc761ffb700 (LWP 13897)):
|
#0 0x00007fc7b1085bf0 in __GI___nanosleep (requested_time=requested_time@entry=0x7fc761ffa750, remaining=remaining@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
|
#1 0x00005596296f03b0 in os_thread_sleep (tm=tm@entry=1000000) at ./storage/innobase/os/os0thread.cc:231
|
#2 0x00005596297d6d8a in btr_defragment_thread () at ./storage/innobase/btr/btr0defragment.cc:712
|
#3 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#4 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 26 (Thread 0x7fc7617fa700 (LWP 13898)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b2d0480) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b2d0430, cond=0x55962b2d0458) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b2d0458, mutex=mutex@entry=0x55962b2d0430) at pthread_cond_wait.c:655
|
#3 0x00005596296efea1 in os_event::wait (this=0x55962b2d0420) at ./storage/innobase/include/sync0types.h:463
|
#4 os_event::wait_low (this=0x55962b2d0420, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:325
|
#5 0x00005596297e43c8 in buf_resize_thread () at ./storage/innobase/buf/buf0buf.cc:3142
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 25 (Thread 0x7fc7988d1700 (LWP 13904)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc7988d0d00, expected=0, futex_word=0x55962a1e3248 <COND_thread_cache+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc7988d0d00, mutex=0x55962a1e09a0 <LOCK_thread_cache>, cond=0x55962a1e3220 <COND_thread_cache>) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=cond@entry=0x55962a1e3220 <COND_thread_cache>, mutex=mutex@entry=0x55962a1e09a0 <LOCK_thread_cache>, abstime=abstime@entry=0x7fc7988d0d00) at pthread_cond_wait.c:667
|
#3 0x000055962920ff9f in inline_mysql_cond_timedwait (that=0x55962a1e3220 <COND_thread_cache>, mutex=0x55962a1e09a0 <LOCK_thread_cache>, src_file=0x559629a7d008 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695, abstime=0x7fc7988d0d00) at ./include/mysql/psi/mysql_thread.h:1211
|
#4 cache_thread (thd=0x7fc774002078) at ./sql/mysqld.cc:2694
|
#5 one_thread_per_connection_end (thd=0x7fc774002078, put_in_cache=<optimized out>) at ./sql/mysqld.cc:2780
|
#6 0x00005596293bc9bb in do_handle_one_connection (connect=connect@entry=0x55962b57ade8) at ./sql/sql_connect.cc:1423
|
#7 0x00005596293bcb2d in handle_one_connection (arg=arg@entry=0x55962b57ade8) at ./sql/sql_connect.cc:1316
|
#8 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b5a02a8) at ./storage/perfschema/pfs.cc:1869
|
#9 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#10 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 24 (Thread 0x7fc768ff9700 (LWP 13892)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b2d0100) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b2d00b0, cond=0x55962b2d00d8) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b2d00d8, mutex=mutex@entry=0x55962b2d00b0) at pthread_cond_wait.c:655
|
#3 0x00005596296efea1 in os_event::wait (this=0x55962b2d00a0) at ./storage/innobase/include/sync0types.h:463
|
#4 os_event::wait_low (this=0x55962b2d00a0, reset_sig_count=15) at ./storage/innobase/os/os0event.cc:325
|
#5 0x000055962976e326 in srv_resume_thread (timeout_usec=0, wait=true, sig_count=15, slot=0x55962a15eb40 <srv_sys+192>) at ./storage/innobase/srv/srv0srv.cc:2655
|
#6 srv_purge_coordinator_suspend (rseg_history_len=<optimized out>, slot=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2655
|
#7 srv_purge_coordinator_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2732
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 23 (Thread 0x7fc763fff700 (LWP 13893)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b2d0254) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b2d0200, cond=0x55962b2d0228) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b2d0228, mutex=mutex@entry=0x55962b2d0200) at pthread_cond_wait.c:655
|
#3 0x00005596296efea1 in os_event::wait (this=0x55962b2d01f0) at ./storage/innobase/include/sync0types.h:463
|
#4 os_event::wait_low (this=0x55962b2d01f0, reset_sig_count=40) at ./storage/innobase/os/os0event.cc:325
|
#5 0x000055962976d08c in srv_resume_thread (sig_count=0, wait=true, timeout_usec=0, slot=0x55962a15ebe8 <srv_sys+360>) at ./storage/innobase/srv/srv0srv.cc:2517
|
#6 srv_worker_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2517
|
#7 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#8 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 22 (Thread 0x7fc798886700 (LWP 13915)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc798885d00, expected=0, futex_word=0x55962a1e3248 <COND_thread_cache+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc798885d00, mutex=0x55962a1e09a0 <LOCK_thread_cache>, cond=0x55962a1e3220 <COND_thread_cache>) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=cond@entry=0x55962a1e3220 <COND_thread_cache>, mutex=mutex@entry=0x55962a1e09a0 <LOCK_thread_cache>, abstime=abstime@entry=0x7fc798885d00) at pthread_cond_wait.c:667
|
#3 0x000055962920ff9f in inline_mysql_cond_timedwait (that=0x55962a1e3220 <COND_thread_cache>, mutex=0x55962a1e09a0 <LOCK_thread_cache>, src_file=0x559629a7d008 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695, abstime=0x7fc798885d00) at ./include/mysql/psi/mysql_thread.h:1211
|
#4 cache_thread (thd=0x7fc780002408) at ./sql/mysqld.cc:2694
|
#5 one_thread_per_connection_end (thd=0x7fc780002408, put_in_cache=<optimized out>) at ./sql/mysqld.cc:2780
|
#6 0x00005596293bc9bb in do_handle_one_connection (connect=connect@entry=0x55962b5a1378) at ./sql/sql_connect.cc:1423
|
#7 0x00005596293bcb2d in handle_one_connection (arg=arg@entry=0x55962b5a1378) at ./sql/sql_connect.cc:1316
|
#8 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b5a27c8) at ./storage/perfschema/pfs.cc:1869
|
#9 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#10 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 21 (Thread 0x7fc7717fa700 (LWP 13882)):
|
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
|
#1 0x00007fc7b105127a in ?? () from /usr/lib/x86_64-linux-gnu/libaio.so.1
|
#2 0x00005596296e8c31 in LinuxAIOHandler::collect (this=0x7fc7717f9be0) at ./storage/innobase/os/os0file.cc:1819
|
#3 0x00005596296ed145 in LinuxAIOHandler::poll (this=0x7fc7717f9be0, m1=0x7fc7717f9c80, m2=0x7fc7717f9c88, request=0x7fc7717f9c90) at ./storage/innobase/os/os0file.cc:1980
|
#4 0x00005596296eecb0 in os_aio_linux_handler (request=0x7fc7717f9c90, m2=0x7fc7717f9c88, m1=0x7fc7717f9c80, global_segment=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:805
|
#5 os_aio_handler (segment=segment@entry=5, m1=m1@entry=0x7fc7717f9c80, m2=m2@entry=0x7fc7717f9c88, request=request@entry=0x7fc7717f9c90) at ./storage/innobase/os/os0file.cc:5607
|
#6 0x00005596298390a8 in fil_aio_wait (segment=segment@entry=5) at ./storage/innobase/fil/fil0fil.cc:4316
|
#7 0x000055962976fc90 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 20 (Thread 0x7fc760ff9700 (LWP 13899)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x7fc760ff8df8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x7fc760ff8da0, cond=0x7fc760ff8dd0) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x7fc760ff8dd0, mutex=mutex@entry=0x7fc760ff8da0) at pthread_cond_wait.c:655
|
#3 0x00005596296707dc in inline_mysql_cond_wait (src_file=0x559629bcd088 "/home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/handler/ha_innodb.cc", src_line=311, mutex=0x7fc760ff8da0, that=0x7fc760ff8dd0) at ./include/mysql/psi/mysql_thread.h:1185
|
#4 thd_destructor_proxy () at ./storage/innobase/handler/ha_innodb.cc:311
|
#5 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b4412a8) at ./storage/perfschema/pfs.cc:1869
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 19 (Thread 0x7fc76affd700 (LWP 13887)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc76affcc40, expected=0, futex_word=0x55962b2d02c0) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc76affcc40, mutex=0x55962b2d0270, cond=0x55962b2d0298) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=0x55962b2d0298, mutex=0x55962b2d0270, abstime=abstime@entry=0x7fc76affcc40) at pthread_cond_wait.c:667
|
#3 0x00005596296efde2 in os_event::timed_wait (this=<optimized out>, abstime=0x7fc76affcc40) at ./storage/innobase/include/sync0types.h:463
|
#4 0x00005596296eff41 in os_event::wait_time_low (this=0x55962b2d0260, time_in_usec=<optimized out>, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
|
#5 0x000055962976b9ef in srv_error_monitor_thread () at ./storage/innobase/srv/srv0srv.cc:1854
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 18 (Thread 0x7fc7697fa700 (LWP 13891)):
|
#0 0x00007fc7b1085bf0 in __GI___nanosleep (requested_time=requested_time@entry=0x7fc7697f9d60, remaining=remaining@entry=0x0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
|
#1 0x00005596296f03b0 in os_thread_sleep (tm=tm@entry=1000000) at ./storage/innobase/os/os0thread.cc:231
|
#2 0x000055962976c288 in srv_master_sleep () at ./storage/innobase/srv/srv0srv.cc:2341
|
#3 srv_master_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2381
|
#4 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#5 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 17 (Thread 0x7fc7abf11700 (LWP 13518)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc7abf10dc0, expected=0, futex_word=0x55962aa75de8 <COND_timer+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc7abf10dc0, mutex=0x55962aa75e00 <LOCK_timer>, cond=0x55962aa75dc0 <COND_timer>) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=cond@entry=0x55962aa75dc0 <COND_timer>, mutex=mutex@entry=0x55962aa75e00 <LOCK_timer>, abstime=abstime@entry=0x7fc7abf10dc0) at pthread_cond_wait.c:667
|
#3 0x00005596299ec86e in inline_mysql_cond_timedwait (that=0x55962aa75dc0 <COND_timer>, mutex=0x55962aa75e00 <LOCK_timer>, src_file=0x559629c32278 "/home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_timer.c", src_line=292, abstime=0x7fc7abf10dc0) at ./include/mysql/psi/mysql_thread.h:1211
|
#4 timer_handler (arg=arg@entry=0x0) at ./mysys/thr_timer.c:292
|
#5 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b13dc38) at ./storage/perfschema/pfs.cc:1869
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 16 (Thread 0x7fc7637fe700 (LWP 13894)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b2d0174) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b2d0120, cond=0x55962b2d0148) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b2d0148, mutex=mutex@entry=0x55962b2d0120) at pthread_cond_wait.c:655
|
#3 0x00005596296efea1 in os_event::wait (this=0x55962b2d0110) at ./storage/innobase/include/sync0types.h:463
|
#4 os_event::wait_low (this=0x55962b2d0110, reset_sig_count=94) at ./storage/innobase/os/os0event.cc:325
|
#5 0x000055962976d08c in srv_resume_thread (sig_count=0, wait=true, timeout_usec=0, slot=0x55962a15eb78 <srv_sys+248>) at ./storage/innobase/srv/srv0srv.cc:2517
|
#6 srv_worker_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2517
|
#7 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#8 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 15 (Thread 0x7fc7a83a3700 (LWP 13643)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x7fc784012dc0) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x7fc784012d68, cond=0x7fc784012d98) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x7fc784012d98, mutex=mutex@entry=0x7fc784012d68) at pthread_cond_wait.c:655
|
#3 0x0000559629459d5b in inline_mysql_cond_wait (src_line=69, src_file=0x559629abc4b8 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/wsrep_thd.h", mutex=0x7fc784012d68, that=0x7fc784012d98) at ./include/mysql/psi/mysql_thread.h:1185
|
#4 Wsrep_thd_queue::pop_front (this=0x7fc784012d10) at ./sql/wsrep_thd.h:69
|
#5 wsrep_rollback_process (rollbacker=0x7fc784000c08, arg=<optimized out>) at ./sql/wsrep_thd.cc:241
|
#6 0x000055962944c278 in start_wsrep_THD (arg=arg@entry=0x55962b1d5670) at ./sql/wsrep_mysqld.h:423
|
#7 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b19f508) at ./storage/perfschema/pfs.cc:1869
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 14 (Thread 0x7fc762ffd700 (LWP 13895)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b2d01e4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b2d0190, cond=0x55962b2d01b8) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b2d01b8, mutex=mutex@entry=0x55962b2d0190) at pthread_cond_wait.c:655
|
#3 0x00005596296efea1 in os_event::wait (this=0x55962b2d0180) at ./storage/innobase/include/sync0types.h:463
|
#4 os_event::wait_low (this=0x55962b2d0180, reset_sig_count=90) at ./storage/innobase/os/os0event.cc:325
|
#5 0x000055962976d08c in srv_resume_thread (sig_count=0, wait=true, timeout_usec=0, slot=0x55962a15ebb0 <srv_sys+304>) at ./storage/innobase/srv/srv0srv.cc:2517
|
#6 srv_worker_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2517
|
#7 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#8 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 13 (Thread 0x7fc7a8358700 (LWP 13644)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x7fc7ab4ab0e4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x7fc7ab4ab090, cond=0x7fc7ab4ab0b8) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x7fc7ab4ab0b8, mutex=mutex@entry=0x7fc7ab4ab090) at pthread_cond_wait.c:655
|
#3 0x00007fc7ab69f12e in fifo_lock_get (q=0x7fc7ab4ab010) at ./galerautils/src/gu_fifo.c:262
|
#4 gu_fifo_get_head (q=0x7fc7ab4ab010, err=err@entry=0x7fc7a8357674) at ./galerautils/src/gu_fifo.c:330
|
#5 0x00007fc7ab575c65 in gcs_recv (conn=0x55962b186f70, action=action@entry=0x7fc7a83576f0) at ./gcs/src/gcs.cpp:2047
|
#6 0x00007fc7ab55efc6 in galera::Gcs::recv (act=..., this=<optimized out>) at ./galera/src/galera_gcs.hpp:120
|
#7 galera::GcsActionSource::process (this=0x55962b1cf800, recv_ctx=0x7fc7a8357cf0, exit_loop=@0x7fc7a83578ff: false) at ./galera/src/gcs_action_source.cpp:162
|
#8 0x00007fc7ab532cb0 in galera::ReplicatorSMM::async_recv (this=0x55962b17d920, recv_ctx=0x7fc7a8357cf0) at ./galera/src/replicator_smm.cpp:390
|
#9 0x00007fc7ab50e48b in galera_recv (gh=<optimized out>, recv_ctx=<optimized out>) at ./galera/src/wsrep_provider.cpp:236
|
#10 0x0000559629a7019e in wsrep::wsrep_provider_v26::run_applier (this=<optimized out>, applier_ctx=<optimized out>) at ./wsrep-lib/src/wsrep_provider_v26.cpp:743
|
#11 0x000055962945a4bf in wsrep_replication_process (thd=0x7fc78c000c08, arg=<optimized out>) at ./wsrep-lib/include/wsrep/server_state.hpp:308
|
#12 0x000055962944c278 in start_wsrep_THD (arg=arg@entry=0x55962b1d5690) at ./sql/wsrep_mysqld.h:423
|
#13 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b19f858) at ./storage/perfschema/pfs.cc:1869
|
#14 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#15 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 12 (Thread 0x7fc7a8ba4700 (LWP 13642)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962b1520f8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962b1520a8, cond=0x55962b1520d0) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=cond@entry=0x55962b1520d0, mutex=mutex@entry=0x55962b1520a8) at pthread_cond_wait.c:655
|
#3 0x00007fc7ab592219 in gu::Lock::wait (this=<synthetic pointer>, cond=...) at ./galerautils/src/gu_mutex.hpp:86
|
#4 RecvBuf::front (timeout=..., this=0x55962b1520a8) at ./gcs/src/gcs_gcomm.cpp:114
|
#5 gcomm_recv (backend=<optimized out>, msg=0x55962b187250, timeout=9223372035999999999) at ./gcs/src/gcs_gcomm.cpp:608
|
#6 0x00007fc7ab57ff88 in core_msg_recv (timeout=9223372035999999999, recv_msg=0x55962b187250, backend=0x55962b1873b0) at ./gcs/src/gcs_core.cpp:1201
|
#7 gcs_core_recv (conn=0x55962b1871f0, recv_act=recv_act@entry=0x7fc7a8ba3d60, timeout=9223372035999999999) at ./gcs/src/gcs_core.cpp:1201
|
#8 0x00007fc7ab57bc4e in gcs_recv_thread (arg=0x55962b186f70) at ./gcs/src/gcs.cpp:1413
|
#9 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#10 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 11 (Thread 0x7fc799e69700 (LWP 13876)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc799e68d30, expected=0, futex_word=0x55962aa65e08 <COND_checkpoint+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc799e68d30, mutex=0x55962aa65e20 <LOCK_checkpoint>, cond=0x55962aa65de0 <COND_checkpoint>) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=cond@entry=0x55962aa65de0 <COND_checkpoint>, mutex=mutex@entry=0x55962aa65e20 <LOCK_checkpoint>, abstime=abstime@entry=0x7fc799e68d30) at pthread_cond_wait.c:667
|
#3 0x00005596298d28ac in inline_mysql_cond_timedwait (src_file=0x559629c1c3a8 "/home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_servicethread.c", src_line=116, abstime=0x7fc799e68d30, mutex=0x55962aa65e20 <LOCK_checkpoint>, that=0x55962aa65de0 <COND_checkpoint>) at ./include/mysql/psi/mysql_thread.h:1211
|
#4 my_service_thread_sleep (control=control@entry=0x55962a161640 <checkpoint_control>, sleep_time=<optimized out>) at ./storage/maria/ma_servicethread.c:115
|
#5 0x00005596298cae09 in ma_checkpoint_background (arg=arg@entry=0x1e) at ./storage/maria/ma_checkpoint.c:707
|
#6 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b28c7f8) at ./storage/perfschema/pfs.cc:1869
|
#7 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#8 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 10 (Thread 0x7fc772ffd700 (LWP 13879)):
|
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
|
#1 0x00007fc7b105127a in ?? () from /usr/lib/x86_64-linux-gnu/libaio.so.1
|
#2 0x00005596296e8c31 in LinuxAIOHandler::collect (this=0x7fc772ffcbe0) at ./storage/innobase/os/os0file.cc:1819
|
#3 0x00005596296ed145 in LinuxAIOHandler::poll (this=0x7fc772ffcbe0, m1=0x7fc772ffcc80, m2=0x7fc772ffcc88, request=0x7fc772ffcc90) at ./storage/innobase/os/os0file.cc:1980
|
#4 0x00005596296eecb0 in os_aio_linux_handler (request=0x7fc772ffcc90, m2=0x7fc772ffcc88, m1=0x7fc772ffcc80, global_segment=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:805
|
#5 os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7fc772ffcc80, m2=m2@entry=0x7fc772ffcc88, request=request@entry=0x7fc772ffcc90) at ./storage/innobase/os/os0file.cc:5607
|
#6 0x00005596298390a8 in fil_aio_wait (segment=segment@entry=2) at ./storage/innobase/fil/fil0fil.cc:4316
|
#7 0x000055962976fc90 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 9 (Thread 0x7fc79891c700 (LWP 13901)):
|
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x55962a1e320c <COND_slave_background+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
|
#1 __pthread_cond_wait_common (abstime=0x0, mutex=0x55962a1e06a0 <LOCK_slave_background>, cond=0x55962a1e31e0 <COND_slave_background>) at pthread_cond_wait.c:502
|
#2 __pthread_cond_wait (cond=0x55962a1e31e0 <COND_slave_background>, mutex=0x55962a1e06a0 <LOCK_slave_background>) at pthread_cond_wait.c:655
|
#3 0x000055962922c4e8 in inline_mysql_cond_wait (that=<optimized out>, mutex=<optimized out>, src_file=0x559629a87230 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc", src_line=522) at ./include/mysql/psi/mysql_thread.h:1174
|
#4 handle_slave_background (arg=arg@entry=0x0) at ./sql/slave.cc:522
|
#5 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b4f9e28) at ./storage/perfschema/pfs.cc:1869
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 8 (Thread 0x7fc7727fc700 (LWP 13880)):
|
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
|
#1 0x00007fc7b105127a in ?? () from /usr/lib/x86_64-linux-gnu/libaio.so.1
|
#2 0x00005596296e8c31 in LinuxAIOHandler::collect (this=0x7fc7727fbbe0) at ./storage/innobase/os/os0file.cc:1819
|
#3 0x00005596296ed145 in LinuxAIOHandler::poll (this=0x7fc7727fbbe0, m1=0x7fc7727fbc80, m2=0x7fc7727fbc88, request=0x7fc7727fbc90) at ./storage/innobase/os/os0file.cc:1980
|
#4 0x00005596296eecb0 in os_aio_linux_handler (request=0x7fc7727fbc90, m2=0x7fc7727fbc88, m1=0x7fc7727fbc80, global_segment=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:805
|
#5 os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7fc7727fbc80, m2=m2@entry=0x7fc7727fbc88, request=request@entry=0x7fc7727fbc90) at ./storage/innobase/os/os0file.cc:5607
|
#6 0x00005596298390a8 in fil_aio_wait (segment=segment@entry=3) at ./storage/innobase/fil/fil0fil.cc:4316
|
#7 0x000055962976fc90 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 7 (Thread 0x7fc770ff9700 (LWP 13883)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc770ff8ba0, expected=0, futex_word=0x55962b2d0410) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc770ff8ba0, mutex=0x55962b2d03c0, cond=0x55962b2d03e8) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=0x55962b2d03e8, mutex=0x55962b2d03c0, abstime=abstime@entry=0x7fc770ff8ba0) at pthread_cond_wait.c:667
|
#3 0x00005596296efde2 in os_event::timed_wait (this=<optimized out>, abstime=0x7fc770ff8ba0) at ./storage/innobase/include/sync0types.h:463
|
#4 0x00005596296eff41 in os_event::wait_time_low (this=0x55962b2d03b0, time_in_usec=<optimized out>, reset_sig_count=4) at ./storage/innobase/os/os0event.cc:385
|
#5 0x00005596291da0af in pc_sleep_if_needed (cur_time=459592, sig_count=4, next_loop_time=460536) at ./storage/innobase/buf/buf0flu.cc:2627
|
#6 buf_flush_page_cleaner_coordinator () at ./storage/innobase/buf/buf0flu.cc:3086
|
#7 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#8 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 6 (Thread 0x7fc798967700 (LWP 13900)):
|
#0 0x00007fc7b04cf4cc in __GI___sigtimedwait (set=set@entry=0x7fc798966cf0, info=info@entry=0x7fc798966d70, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
|
#1 0x00007fc7b04cf537 in __GI___sigwaitinfo (set=set@entry=0x7fc798966cf0, info=info@entry=0x7fc798966d70) at ../sysdeps/unix/sysv/linux/sigwaitinfo.c:25
|
#2 0x000055962920e7cb in my_sigwait (code=<synthetic pointer>, sig=0x7fc798966ce8, set=0x7fc798966cf0) at ./include/my_pthread.h:196
|
#3 signal_hand (arg=arg@entry=0x0) at ./sql/mysqld.cc:3251
|
#4 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b2983d8) at ./storage/perfschema/pfs.cc:1869
|
#5 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#6 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 5 (Thread 0x7fc76b7fe700 (LWP 13886)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc76b7fdce0, expected=0, futex_word=0x55962b468820) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc76b7fdce0, mutex=0x55962b4687d0, cond=0x55962b4687f8) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=0x55962b4687f8, mutex=0x55962b4687d0, abstime=abstime@entry=0x7fc76b7fdce0) at pthread_cond_wait.c:667
|
#3 0x00005596296efde2 in os_event::timed_wait (this=<optimized out>, abstime=0x7fc76b7fdce0) at ./storage/innobase/include/sync0types.h:463
|
#4 0x00005596296eff41 in os_event::wait_time_low (this=0x55962b4687c0, time_in_usec=<optimized out>, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
|
#5 0x00005596296ced7c in lock_wait_timeout_thread () at ./storage/innobase/lock/lock0wait.cc:522
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 4 (Thread 0x7fc771ffb700 (LWP 13881)):
|
#0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
|
#1 0x00007fc7b105127a in ?? () from /usr/lib/x86_64-linux-gnu/libaio.so.1
|
#2 0x00005596296e8c31 in LinuxAIOHandler::collect (this=0x7fc771ffabe0) at ./storage/innobase/os/os0file.cc:1819
|
#3 0x00005596296ed145 in LinuxAIOHandler::poll (this=0x7fc771ffabe0, m1=0x7fc771ffac80, m2=0x7fc771ffac88, request=0x7fc771ffac90) at ./storage/innobase/os/os0file.cc:1980
|
#4 0x00005596296eecb0 in os_aio_linux_handler (request=0x7fc771ffac90, m2=0x7fc771ffac88, m1=0x7fc771ffac80, global_segment=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:805
|
#5 os_aio_handler (segment=segment@entry=4, m1=m1@entry=0x7fc771ffac80, m2=m2@entry=0x7fc771ffac88, request=request@entry=0x7fc771ffac90) at ./storage/innobase/os/os0file.cc:5607
|
#6 0x00005596298390a8 in fil_aio_wait (segment=segment@entry=4) at ./storage/innobase/fil/fil0fil.cc:4316
|
#7 0x000055962976fc90 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
|
#8 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#9 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 3 (Thread 0x7fc7b03037c0 (LWP 13515)):
|
#0 0x00007fc7b0585819 in __GI___poll (fds=fds@entry=0x7ffec4fb9770, nfds=nfds@entry=2, timeout=timeout@entry=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
|
#1 0x00007fc7b05a174e in __poll_chk (fds=fds@entry=0x7ffec4fb9770, nfds=nfds@entry=2, timeout=timeout@entry=-1, fdslen=fdslen@entry=24) at poll_chk.c:27
|
#2 0x00005596292176bf in poll (__timeout=-1, __nfds=2, __fds=0x7ffec4fb9770) at /usr/include/x86_64-linux-gnu/bits/poll2.h:41
|
#3 handle_connections_sockets () at ./sql/mysqld.cc:6481
|
#4 0x0000559629218758 in mysqld_main (argc=<optimized out>, argv=<optimized out>) at ./sql/mysqld.cc:5917
|
#5 0x00007fc7b04bb09b in __libc_start_main (main=0x5596291f35f0 <main(int, char**)>, argc=23, argv=0x7ffec4fb9bb8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffec4fb9ba8) at ../csu/libc-start.c:308
|
#6 0x000055962920bcfa in _start () at ./sql/mysqld.cc:4683
|
|
|
Thread 2 (Thread 0x7fc769ffb700 (LWP 13889)):
|
#0 futex_abstimed_wait_cancelable (private=0, abstime=0x7fc769ffacf0, expected=0, futex_word=0x55962b499c10) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
|
#1 __pthread_cond_wait_common (abstime=0x7fc769ffacf0, mutex=0x55962b499bc0, cond=0x55962b499be8) at pthread_cond_wait.c:539
|
#2 __pthread_cond_timedwait (cond=0x55962b499be8, mutex=0x55962b499bc0, abstime=abstime@entry=0x7fc769ffacf0) at pthread_cond_wait.c:667
|
#3 0x00005596296efde2 in os_event::timed_wait (this=<optimized out>, abstime=0x7fc769ffacf0) at ./storage/innobase/include/sync0types.h:463
|
#4 0x00005596296eff41 in os_event::wait_time_low (this=0x55962b499bb0, time_in_usec=<optimized out>, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
|
#5 0x000055962982f0a0 in dict_stats_thread () at ./storage/innobase/dict/dict0stats_bg.cc:475
|
#6 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#7 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
Thread 1 (Thread 0x7fc79883b700 (LWP 13916)):
|
#0 __pthread_kill (threadid=<optimized out>, signo=signo@entry=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
|
#1 0x00005596299e8297 in my_write_core (sig=sig@entry=11) at ./mysys/stacktrace.c:386
|
#2 0x00005596294dc0a8 in handle_fatal_signal (sig=11) at ./sql/signal_handler.cc:343
|
#3 <signal handler called>
|
#4 0x000055962928168e in thd_clear_errors (thd=thd@entry=0x7fc734000c08) at ./sql/sql_class.cc:311
|
#5 0x00005596292860bb in THD::change_user (this=this@entry=0x7fc734000c08) at ./sql/sql_class.cc:1422
|
#6 0x0000559629286289 in THD::reset_for_reuse (this=this@entry=0x7fc734000c08) at ./sql/sql_class.cc:1671
|
#7 0x00005596293bc614 in CONNECT::create_thd (this=this@entry=0x55962b50bd48, thd=thd@entry=0x7fc734000c08) at ./sql/sql_connect.cc:1500
|
#8 0x00005596292100bd in cache_thread (thd=0x7fc734000c08) at ./sql/mysqld.cc:2717
|
#9 one_thread_per_connection_end (thd=0x7fc734000c08, put_in_cache=<optimized out>) at ./sql/mysqld.cc:2780
|
#10 0x00005596293bc9bb in do_handle_one_connection (connect=connect@entry=0x55962b5a2d78) at ./sql/sql_connect.cc:1423
|
#11 0x00005596293bcb2d in handle_one_connection (arg=arg@entry=0x55962b5a2d78) at ./sql/sql_connect.cc:1316
|
#12 0x0000559629998f8b in pfs_spawn_thread (arg=0x55962b5a2fd8) at ./storage/perfschema/pfs.cc:1869
|
#13 0x00007fc7b107bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
|
#14 0x00007fc7b05904cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
|
|
|
- deleting it, already saved 0
|
- saving '/dev/shm/var/1/log/galera.galera_bf_kill-innodb/' to '/dev/shm/var/log/galera.galera_bf_kill-innodb/'
|
|
|
Retrying test galera.galera_bf_kill, attempt(2/3)...
|
|
|
worker[1] > Restart - not started
|
worker[1] > Restart - not started
|
***Warnings generated in error logs during shutdown after running tests: galera.galera_bf_kill
|
|
|
210128 4:25:56 [ERROR] mysqld got signal 11 ;
|
Attempting backtrace. You can use the following information to find out
|
|
|
galera.galera_bf_kill 'innodb' w1 [ retry-pass ] 2090
|
|
|
Retrying test galera.galera_bf_kill, attempt(3/3)...
|
|
|
galera.galera_bf_kill 'innodb' w1 [ retry-pass ] 2048
|
worker[1] > Restart [mysqld.1 - pid: 13944, winpid: 13944] - using different config file
|
worker[1] > Restart [mysqld.2 - pid: 13980, winpid: 13980] - using different config file
|