Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-21525

galera.galera_ftwrl MTR failed: Crash with signal SIGABRT

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.4.12
    • Fix Version/s: 10.4
    • Component/s: Galera, Tests
    • Labels:
    • Environment:
      kvm-deb-xenial-amd64

      Description

      galera.galera_ftwrl failed on bb: Crash with signal SIGABRT.

      stdio.log:

      10.4.12 2cff807d3f2149d17e776fbddb0fc9e23fa64e1b

      galera.galera_ftwrl 'innodb'             w1 [ fail ]  timeout after 900 seconds
              Test ended at 2019-12-31 07:28:33
       
      Test case timeout after 900 seconds
       
      == /dev/shm/var/1/log/galera_ftwrl.log == 
      connection node_2;
      connection node_1;
      CREATE TABLE t1 (id INT PRIMARY KEY) ENGINE=InnoDB;
      connection node_2;
      SET GLOBAL wsrep_provider_options = "repl.causal_read_timeout=PT1S";
      FLUSH TABLES WITH READ LOCK;
       
       == /dev/shm/var/1/tmp/analyze-timeout-mysqld.2.err ==
      mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/dev/shm/var/tmp/1/mysqld.2.sock' (111)
       
       == /dev/shm/var/1/tmp/analyze-timeout-mysqld.1.err ==
      mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/dev/shm/var/tmp/1/mysqld.1.sock' (111)
       
       
       - found 'core' (0/0)
       
      Trying 'dbx' to get a backtrace
       
      Trying 'gdb' to get a backtrace from coredump /dev/shm/var/1/log/galera.galera_ftwrl-innodb/mysqld.2/data/core
      Core generated by '/usr/sbin/mysqld'
      Output from gdb follows. The first stack trace is from the failing thread.
      The following stack traces are from all threads (so the failing one is
      duplicated).
      --------------------------
      [New LWP 6599]
      [New LWP 6638]
      [New LWP 6602]
      [New LWP 6838]
      [New LWP 6820]
      [New LWP 6832]
      [New LWP 6640]
      [New LWP 6847]
      [New LWP 6840]
      [New LWP 6821]
      [New LWP 6846]
      [New LWP 6822]
      [New LWP 6845]
      [New LWP 6639]
      [New LWP 6841]
      [New LWP 6826]
      [New LWP 6601]
      [New LWP 6833]
      [New LWP 6600]
      [New LWP 6844]
      [New LWP 6839]
      [New LWP 6834]
      [New LWP 6836]
      [New LWP 6837]
      [New LWP 6835]
      [New LWP 6842]
      [New LWP 6831]
      [New LWP 6843]
      [New LWP 6827]
      [New LWP 6823]
      [New LWP 6825]
      [New LWP 6824]
      [Thread debugging using libthread_db enabled]
      Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
      Core was generated by `/usr/sbin/mysqld --defaults-group-suffix=.2 --defaults-file=/dev/shm/var/1/my.c'.
      Program terminated with signal SIGABRT, Aborted.
      #0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
      62	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
      [Current thread is 1 (Thread 0x7ffbb22f18c0 (LWP 6599))]
      #0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
      #1  0x0000562e25944e97 in my_write_core (sig=sig@entry=6) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/stacktrace.c:481
      #2  0x0000562e253d721e in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/signal_handler.cc:343
      #3  <signal handler called>
      #4  0x00007ffbb04b874d in poll () at ../sysdeps/unix/syscall-template.S:84
      #5  0x00007ffbb04d60be in __poll_chk (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, fdslen=<optimized out>) at poll_chk.c:27
      #6  0x0000562e2510130f in poll (__timeout=-1, __nfds=2, __fds=0x7ffff807ae30) at /usr/include/x86_64-linux-gnu/bits/poll2.h:41
      #7  handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:6456
      #8  0x0000562e25102816 in mysqld_main (argc=163, argv=0x562e26db0848) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:5892
      #9  0x00007ffbb03dd830 in __libc_start_main (main=0x562e250db9e0 <main(int, char**)>, argc=23, argv=0x7ffff807b568, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffff807b558) at ../csu/libc-start.c:291
      #10 0x0000562e250f5199 in _start ()
       
      Thread 32 (Thread 0x7ffb7a5fc700 (LWP 6824)):
      #0  0x00007ffbb1d4964a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x0000562e256ae2f8 in LinuxAIOHandler::collect (this=this@entry=0x7ffb7a5fbb00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
      #2  0x0000562e256ae962 in LinuxAIOHandler::poll (this=this@entry=0x7ffb7a5fbb00, m1=m1@entry=0x7ffb7a5fbb90, m2=m2@entry=0x7ffb7a5fbb98, request=request@entry=0x7ffb7a5fbba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
      #3  0x0000562e256b2ae8 in os_aio_linux_handler (request=0x7ffb7a5fbba0, m2=0x7ffb7a5fbb98, m1=0x7ffb7a5fbb90, global_segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
      #4  os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7ffb7a5fbb90, m2=m2@entry=0x7ffb7a5fbb98, request=request@entry=0x7ffb7a5fbba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
      #5  0x0000562e25833105 in fil_aio_wait (segment=segment@entry=3) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4357
      #6  0x0000562e25745ed8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb7a5fc700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 31 (Thread 0x7ffb79dfb700 (LWP 6825)):
      #0  0x00007ffbb1d4964a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x0000562e256ae2f8 in LinuxAIOHandler::collect (this=this@entry=0x7ffb79dfab00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
      #2  0x0000562e256ae962 in LinuxAIOHandler::poll (this=this@entry=0x7ffb79dfab00, m1=m1@entry=0x7ffb79dfab90, m2=m2@entry=0x7ffb79dfab98, request=request@entry=0x7ffb79dfaba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
      #3  0x0000562e256b2ae8 in os_aio_linux_handler (request=0x7ffb79dfaba0, m2=0x7ffb79dfab98, m1=0x7ffb79dfab90, global_segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
      #4  os_aio_handler (segment=segment@entry=4, m1=m1@entry=0x7ffb79dfab90, m2=m2@entry=0x7ffb79dfab98, request=request@entry=0x7ffb79dfaba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
      #5  0x0000562e25833105 in fil_aio_wait (segment=segment@entry=4) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4357
      #6  0x0000562e25745ed8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb79dfb700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 30 (Thread 0x7ffb7adfd700 (LWP 6823)):
      #0  0x00007ffbb1d4964a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x0000562e256ae2f8 in LinuxAIOHandler::collect (this=this@entry=0x7ffb7adfcb00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
      #2  0x0000562e256ae962 in LinuxAIOHandler::poll (this=this@entry=0x7ffb7adfcb00, m1=m1@entry=0x7ffb7adfcb90, m2=m2@entry=0x7ffb7adfcb98, request=request@entry=0x7ffb7adfcba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
      #3  0x0000562e256b2ae8 in os_aio_linux_handler (request=0x7ffb7adfcba0, m2=0x7ffb7adfcb98, m1=0x7ffb7adfcb90, global_segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
      #4  os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7ffb7adfcb90, m2=m2@entry=0x7ffb7adfcb98, request=request@entry=0x7ffb7adfcba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
      #5  0x0000562e25833105 in fil_aio_wait (segment=segment@entry=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4357
      #6  0x0000562e25745ed8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb7adfd700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 29 (Thread 0x7ffb78df9700 (LWP 6827)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e256b7a12 in os_event::timed_wait (this=this@entry=0x562e26f9a530, abstime=abstime@entry=0x7ffb78df8980) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:275
      #2  0x0000562e256b7c61 in os_event::wait_time_low (this=0x562e26f9a530, time_in_usec=time_in_usec@entry=986000, reset_sig_count=reset_sig_count@entry=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:385
      #3  0x0000562e256b801a in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=986000, reset_sig_count=reset_sig_count@entry=2) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:485
      #4  0x0000562e257e231c in pc_sleep_if_needed (cur_time=1177265, sig_count=2, next_loop_time=1178251) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/buf/buf0flu.cc:2640
      #5  buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/buf/buf0flu.cc:3099
      #6  0x00007ffbb16e16ba in start_thread (arg=0x7ffb78df9700) at pthread_create.c:333
      #7  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 28 (Thread 0x7ffb5d7fa700 (LWP 6843)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e256b8066 in os_event::wait (this=0x562e26f9a5a0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=1, this=0x562e26f9a5a0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x562e26f9a5a0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
      #4  0x0000562e257cfbcc in buf_resize_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/buf/buf0buf.cc:3178
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5d7fa700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 27 (Thread 0x7ffb6f7fe700 (LWP 6831)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e256b7a12 in os_event::timed_wait (this=this@entry=0x562e271dd9d0, abstime=abstime@entry=0x7ffb6f7fdbf0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:275
      #2  0x0000562e256b7c61 in os_event::wait_time_low (this=this@entry=0x562e271dd9d0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:385
      #3  0x0000562e256b801a in os_event_wait_time_low (event=event@entry=0x562e271dd9d0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:485
      #4  0x0000562e2568d319 in lock_wait_timeout_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/lock/lock0wait.cc:522
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6f7fe700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 26 (Thread 0x7ffb5dffb700 (LWP 6842)):
      #0  0x00007ffbb16eac1d in nanosleep () at ../sysdeps/unix/syscall-template.S:84
      #1  0x0000562e256b82d0 in os_thread_sleep (tm=tm@entry=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0thread.cc:231
      #2  0x0000562e257c1a47 in btr_defragment_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/btr/btr0defragment.cc:717
      #3  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5dffb700) at pthread_create.c:333
      #4  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 25 (Thread 0x7ffb6dffb700 (LWP 6835)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e256b7a12 in os_event::timed_wait (this=this@entry=0x562e271ae360, abstime=abstime@entry=0x7ffb6dffaa30) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:275
      #2  0x0000562e256b7c61 in os_event::wait_time_low (this=0x562e271ae360, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=reset_sig_count@entry=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:385
      #3  0x0000562e256b801a in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=reset_sig_count@entry=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:485
      #4  0x0000562e2578db0a in ib_wqueue_timedwait (wq=wq@entry=0x562e27212d30, wait_in_usecs=wait_in_usecs@entry=5000000) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/ut/ut0wqueue.cc:156
      #5  0x0000562e2586d58a in fts_optimize_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fts/fts0opt.cc:2836
      #6  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6dffb700) at pthread_create.c:333
      #7  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 24 (Thread 0x7ffb6cff9700 (LWP 6837)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e256b8066 in os_event::wait (this=0x562e26f9a220) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=105, this=0x562e26f9a220) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x562e26f9a220, reset_sig_count=reset_sig_count@entry=105) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
      #4  0x0000562e257443a1 in srv_resume_thread (timeout_usec=0, wait=true, sig_count=105, slot=0x562e26302d40 <srv_sys+192>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:909
      #5  srv_purge_coordinator_suspend (rseg_history_len=<optimized out>, slot=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2657
      #6  srv_purge_coordinator_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2738
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6cff9700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 23 (Thread 0x7ffb6d7fa700 (LWP 6836)):
      #0  0x00007ffbb16eac1d in nanosleep () at ../sysdeps/unix/syscall-template.S:84
      #1  0x0000562e256b82d0 in os_thread_sleep (tm=tm@entry=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0thread.cc:231
      #2  0x0000562e2574262a in srv_master_sleep () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2334
      #3  srv_master_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2375
      #4  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6d7fa700) at pthread_create.c:333
      #5  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 22 (Thread 0x7ffb6e7fc700 (LWP 6834)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e256b7a12 in os_event::timed_wait (this=this@entry=0x562e2720ee30, abstime=abstime@entry=0x7ffb6e7fbbe0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:275
      #2  0x0000562e256b7c61 in os_event::wait_time_low (this=0x562e2720ee30, time_in_usec=time_in_usec@entry=10000000, reset_sig_count=1, reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:385
      #3  0x0000562e256b801a in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=10000000, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:485
      #4  0x0000562e25826b40 in dict_stats_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/dict/dict0stats_bg.cc:476
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6e7fc700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 21 (Thread 0x7ffb5f7fe700 (LWP 6839)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e256b8066 in os_event::wait (this=0x562e26f9a300) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=270, this=0x562e26f9a300) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x562e26f9a300, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
      #4  0x0000562e257430f7 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x562e26302db0 <srv_sys+304>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:909
      #5  srv_worker_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2517
      #6  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5f7fe700) at pthread_create.c:333
      #7  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 20 (Thread 0x7ffb5cff9700 (LWP 6844)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e25637f53 in inline_mysql_cond_wait (src_file=0x562e25b673a0 "/home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/handler/ha_innodb.cc", src_line=288, mutex=0x7ffb5cff8cc0, that=0x7ffb5cff8cf0) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1178
      #2  thd_destructor_proxy () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/handler/ha_innodb.cc:288
      #3  0x0000562e25587351 in pfs_spawn_thread (arg=0x562e271b8b58) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #4  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5cff9700) at pthread_create.c:333
      #5  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 19 (Thread 0x7ffbab181700 (LWP 6600)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e25949a1d in inline_mysql_cond_timedwait (that=0x562e26c16bc0 <COND_timer>, mutex=0x562e26c16c00 <LOCK_timer>, src_file=0x562e25bb75c8 "/home/buildbot/buildbot/build/mariadb-10.4.12/mysys/thr_timer.c", src_line=292, abstime=0x7ffbab180ce0) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1204
      #2  timer_handler (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/thr_timer.c:292
      #3  0x0000562e25587351 in pfs_spawn_thread (arg=0x562e26eca928) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #4  0x00007ffbb16e16ba in start_thread (arg=0x7ffbab181700) at pthread_create.c:333
      #5  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 18 (Thread 0x7ffb6ffff700 (LWP 6833)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e256b7a12 in os_event::timed_wait (this=this@entry=0x562e26f9a450, abstime=abstime@entry=0x7ffb6fffec90) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:275
      #2  0x0000562e256b7c61 in os_event::wait_time_low (this=0x562e26f9a450, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:385
      #3  0x0000562e256b801a in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:485
      #4  0x0000562e25740e62 in srv_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:1687
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6ffff700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 17 (Thread 0x7ffbaa3c8700 (LWP 6601)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00007ffbaa6c0680 in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #2  galera::ServiceThd::thd_func (arg=0x562e26f06868) at galera/src/galera_service_thd.cpp:30
      #3  0x00007ffbb16e16ba in start_thread (arg=0x7ffbaa3c8700) at pthread_create.c:333
      #4  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 16 (Thread 0x7ffb795fa700 (LWP 6826)):
      #0  0x00007ffbb1d4964a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x0000562e256ae2f8 in LinuxAIOHandler::collect (this=this@entry=0x7ffb795f9b00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
      #2  0x0000562e256ae962 in LinuxAIOHandler::poll (this=this@entry=0x7ffb795f9b00, m1=m1@entry=0x7ffb795f9b90, m2=m2@entry=0x7ffb795f9b98, request=request@entry=0x7ffb795f9ba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
      #3  0x0000562e256b2ae8 in os_aio_linux_handler (request=0x7ffb795f9ba0, m2=0x7ffb795f9b98, m1=0x7ffb795f9b90, global_segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
      #4  os_aio_handler (segment=segment@entry=5, m1=m1@entry=0x7ffb795f9b90, m2=m2@entry=0x7ffb795f9b98, request=request@entry=0x7ffb795f9ba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
      #5  0x0000562e25833105 in fil_aio_wait (segment=segment@entry=5) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4357
      #6  0x0000562e25745ed8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb795fa700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 15 (Thread 0x7ffb5e7fc700 (LWP 6841)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e256b8066 in os_event::wait (this=0x562e26f9a4c0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=1, this=0x562e26f9a4c0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x562e26f9a4c0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
      #4  0x0000562e257d76f7 in buf_dump_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/buf/buf0dump.cc:833
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5e7fc700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 14 (Thread 0x7ffba83c4700 (LWP 6639)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e25349983 in inline_mysql_cond_wait (src_line=69, src_file=0x562e25a46e90 "/home/buildbot/buildbot/build/mariadb-10.4.12/sql/wsrep_thd.h", mutex=0x7ffb90012ae8, that=0x7ffb90012b18) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1178
      #2  Wsrep_thd_queue::pop_front (this=0x7ffb90012a90) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/wsrep_thd.h:69
      #3  wsrep_rollback_process (rollbacker=0x7ffb900009a8, arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/wsrep_thd.cc:228
      #4  0x0000562e2533946e in start_wsrep_THD (arg=arg@entry=0x562e26f440c0) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/wsrep_mysqld.cc:2723
      #5  0x0000562e25587351 in pfs_spawn_thread (arg=0x562e26f32c28) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #6  0x00007ffbb16e16ba in start_thread (arg=0x7ffba83c4700) at pthread_create.c:333
      #7  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 13 (Thread 0x7ffba8147700 (LWP 6845)):
      #0  0x00007ffbb03f3b26 in __GI___sigwaitinfo (set=<optimized out>, set@entry=0x7ffba8146c10, info=info@entry=0x7ffba8146c90) at ../sysdeps/unix/sysv/linux/sigwaitinfo.c:56
      #1  0x0000562e250f7a13 in my_sigwait (code=<synthetic pointer>, sig=0x7ffba8146c08, set=0x7ffba8146c10) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/my_pthread.h:198
      #2  signal_hand (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:3245
      #3  0x0000562e25587351 in pfs_spawn_thread (arg=0x562e26f8fa68) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #4  0x00007ffbb16e16ba in start_thread (arg=0x7ffba8147700) at pthread_create.c:333
      #5  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 12 (Thread 0x7ffb7b5fe700 (LWP 6822)):
      #0  0x00007ffbb1d4964a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x0000562e256ae2f8 in LinuxAIOHandler::collect (this=this@entry=0x7ffb7b5fdb00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
      #2  0x0000562e256ae962 in LinuxAIOHandler::poll (this=this@entry=0x7ffb7b5fdb00, m1=m1@entry=0x7ffb7b5fdb90, m2=m2@entry=0x7ffb7b5fdb98, request=request@entry=0x7ffb7b5fdba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
      #3  0x0000562e256b2ae8 in os_aio_linux_handler (request=0x7ffb7b5fdba0, m2=0x7ffb7b5fdb98, m1=0x7ffb7b5fdb90, global_segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
      #4  os_aio_handler (segment=segment@entry=1, m1=m1@entry=0x7ffb7b5fdb90, m2=m2@entry=0x7ffb7b5fdb98, request=request@entry=0x7ffb7b5fdba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
      #5  0x0000562e25833105 in fil_aio_wait (segment=segment@entry=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4357
      #6  0x0000562e25745ed8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb7b5fe700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 11 (Thread 0x7ffba80fd700 (LWP 6846)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e251161c1 in inline_mysql_cond_wait (that=<optimized out>, mutex=<optimized out>, src_file=0x562e25a13c58 "/home/buildbot/buildbot/build/mariadb-10.4.12/sql/slave.cc", src_line=522) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1167
      #2  handle_slave_background (arg=arg@entry=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/slave.cc:522
      #3  0x0000562e25587351 in pfs_spawn_thread (arg=0x562e272befc8) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #4  0x00007ffbb16e16ba in start_thread (arg=0x7ffba80fd700) at pthread_create.c:333
      #5  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 10 (Thread 0x7ffb7bdff700 (LWP 6821)):
      #0  0x00007ffbb1d4964a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x0000562e256ae2f8 in LinuxAIOHandler::collect (this=this@entry=0x7ffb7bdfeb00) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1813
      #2  0x0000562e256ae962 in LinuxAIOHandler::poll (this=this@entry=0x7ffb7bdfeb00, m1=m1@entry=0x7ffb7bdfeb90, m2=m2@entry=0x7ffb7bdfeb98, request=request@entry=0x7ffb7bdfeba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:1974
      #3  0x0000562e256b2ae8 in os_aio_linux_handler (request=0x7ffb7bdfeba0, m2=0x7ffb7bdfeb98, m1=0x7ffb7bdfeb90, global_segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:2028
      #4  os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7ffb7bdfeb90, m2=m2@entry=0x7ffb7bdfeb98, request=request@entry=0x7ffb7bdfeba0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0file.cc:5598
      #5  0x0000562e25833105 in fil_aio_wait (segment=segment@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/fil/fil0fil.cc:4357
      #6  0x0000562e25745ed8 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0start.cc:324
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb7bdff700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 9 (Thread 0x7ffb5effd700 (LWP 6840)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e256b8066 in os_event::wait (this=0x562e26f9a370) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=276, this=0x562e26f9a370) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x562e26f9a370, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
      #4  0x0000562e257430f7 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x562e26302de8 <srv_sys+360>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:909
      #5  srv_worker_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2517
      #6  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5effd700) at pthread_create.c:333
      #7  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 8 (Thread 0x7ffba80b3700 (LWP 6847)):
      #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
      #1  0x00007ffbb16e3efe in __GI___pthread_mutex_lock (mutex=mutex@entry=0x7ffb8001a520) at ../nptl/pthread_mutex_lock.c:135
      #2  0x0000562e252f6bb4 in inline_mysql_mutex_lock (src_line=3366, src_file=0x562e25a094d0 "/home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_class.h", that=0x7ffb8001a520) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:697
      #3  THD::exit_cond (src_function=0x562e25a0e676 "<unknown>", src_file=0x562e25a40980 "/home/buildbot/buildbot/build/mariadb-10.4.12/sql/rpl_parallel.cc", src_line=489, stage=0x7ffba80af2f0, this=0x7ffb80009fe8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_class.h:3366
      #4  pool_mark_busy (pool=pool@entry=0x562e263aa4a0 <global_rpl_thread_pool>, thd=thd@entry=0x7ffb80009fe8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/rpl_parallel.cc:489
      #5  0x0000562e252f7be6 in rpl_pause_for_ftwrl (thd=thd@entry=0x7ffb80009fe8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/rpl_parallel.cc:563
      #6  0x0000562e251befb1 in mysql_execute_command (thd=thd@entry=0x7ffb80009fe8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_parse.cc:5453
      #7  0x0000562e251c6afa in mysql_parse (thd=thd@entry=0x7ffb80009fe8, rawbuf=rawbuf@entry=0x7ffb8001a750 "FLUSH TABLES WITH READ LOCK", length=length@entry=27, parser_state=parser_state@entry=0x7ffba80b1ff0, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_parse.cc:7901
      #8  0x0000562e251c7389 in wsrep_mysql_parse (thd=thd@entry=0x7ffb80009fe8, rawbuf=0x7ffb8001a750 "FLUSH TABLES WITH READ LOCK", length=27, parser_state=parser_state@entry=0x7ffba80b1ff0, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_parse.cc:7705
      #9  0x0000562e251ca3bb in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7ffb80009fe8, packet=packet@entry=0x7ffb80001649 "FLUSH TABLES WITH READ LOCK", packet_length=packet_length@entry=27, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_parse.cc:1827
      #10 0x0000562e251ca98c in do_command (thd=0x7ffb80009fe8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_parse.cc:1360
      #11 0x0000562e252a78ba in do_handle_one_connection (connect=connect@entry=0x562e273099d8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_connect.cc:1412
      #12 0x0000562e252a797d in handle_one_connection (arg=arg@entry=0x562e273099d8) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/sql_connect.cc:1316
      #13 0x0000562e25587351 in pfs_spawn_thread (arg=0x562e27316b48) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #14 0x00007ffbb16e16ba in start_thread (arg=0x7ffba80b3700) at pthread_create.c:333
      #15 0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 7 (Thread 0x7ffba837a700 (LWP 6640)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00007ffbaa52c2ee in fifo_lock_get (q=0x7ffbaa4ca010) at galerautils/src/gu_fifo.c:262
      #2  gu_fifo_get_head (q=0x7ffbaa4ca010, err=err@entry=0x7ffba8378d6c) at galerautils/src/gu_fifo.c:330
      #3  0x00007ffbaa67dfb5 in gcs_recv (conn=0x562e26f1a7e0, action=action@entry=0x7ffba8378df0) at gcs/src/gcs.cpp:2024
      #4  0x00007ffbaa6cbd16 in galera::Gcs::recv (act=..., this=<optimized out>) at galera/src/galera_gcs.hpp:120
      #5  galera::GcsActionSource::process (this=0x562e26f0f5b0, recv_ctx=0x7ffba83793f0, exit_loop=@0x7ffba8378fff: false) at galera/src/gcs_action_source.cpp:161
      #6  0x00007ffbaa6f44b3 in galera::ReplicatorSMM::async_recv (this=0x562e26f06110, recv_ctx=0x7ffba83793f0) at galera/src/replicator_smm.cpp:391
      #7  0x00007ffbaa712e4b in galera_recv (gh=<optimized out>, recv_ctx=<optimized out>) at galera/src/wsrep_provider.cpp:236
      #8  0x0000562e259f743e in wsrep::wsrep_provider_v26::run_applier (this=<optimized out>, applier_ctx=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/wsrep-lib/src/wsrep_provider_v26.cpp:731
      #9  0x0000562e2534ac92 in wsrep_replication_process (thd=0x7ffb840009a8, arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/wsrep_thd.cc:62
      #10 0x0000562e2533946e in start_wsrep_THD (arg=arg@entry=0x562e26f67d00) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/wsrep_mysqld.cc:2723
      #11 0x0000562e25587351 in pfs_spawn_thread (arg=0x562e26f32f88) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #12 0x00007ffbb16e16ba in start_thread (arg=0x7ffba837a700) at pthread_create.c:333
      #13 0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 6 (Thread 0x7ffb6effd700 (LWP 6832)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e256b7a12 in os_event::timed_wait (this=this@entry=0x562e26f9a3e0, abstime=abstime@entry=0x7ffb6effcb60) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:275
      #2  0x0000562e256b7c61 in os_event::wait_time_low (this=0x562e26f9a3e0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:385
      #3  0x0000562e256b801a in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:485
      #4  0x0000562e2574152b in srv_error_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:1851
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb6effd700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 5 (Thread 0x7ffb99668700 (LWP 6820)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x0000562e255d3e7c in inline_mysql_cond_timedwait (src_file=0x562e25b63810 "/home/buildbot/buildbot/build/mariadb-10.4.12/storage/maria/ma_servicethread.c", src_line=116, abstime=0x7ffb99667c60, mutex=0x562e26bbb440 <LOCK_checkpoint>, that=0x562e26bbb400 <COND_checkpoint>) at /home/buildbot/buildbot/build/mariadb-10.4.12/include/mysql/psi/mysql_thread.h:1204
      #2  my_service_thread_sleep (control=control@entry=0x562e262f6520 <checkpoint_control>, sleep_time=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/maria/ma_servicethread.c:115
      #3  0x0000562e255cb4c7 in ma_checkpoint_background (arg=arg@entry=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/maria/ma_checkpoint.c:707
      #4  0x0000562e25587351 in pfs_spawn_thread (arg=0x562e26f90cb8) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/perfschema/pfs.cc:1862
      #5  0x00007ffbb16e16ba in start_thread (arg=0x7ffb99668700) at pthread_create.c:333
      #6  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 4 (Thread 0x7ffb5ffff700 (LWP 6838)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x0000562e256b8066 in os_event::wait (this=0x562e26f9a290) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=464, this=0x562e26f9a290) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x562e26f9a290, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/os/os0event.cc:502
      #4  0x0000562e257430f7 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x562e26302d78 <srv_sys+248>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:909
      #5  srv_worker_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.12/storage/innobase/srv/srv0srv.cc:2517
      #6  0x00007ffbb16e16ba in start_thread (arg=0x7ffb5ffff700) at pthread_create.c:333
      #7  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 3 (Thread 0x7ffb9bffe700 (LWP 6602)):
      #0  0x00007ffbb04c4a13 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00007ffbaa64351d in asio::detail::epoll_reactor::run (this=0x562e26f47760, block=block@entry=true, ops=...) at asio/asio/detail/impl/epoll_reactor.ipp:391
      #2  0x00007ffbaa65f8d4 in asio::detail::task_io_service::do_run_one (ec=..., this_thread=..., lock=..., this=0x562e26f47680) at asio/asio/detail/impl/task_io_service.ipp:355
      #3  asio::detail::task_io_service::run (ec=..., this=0x562e26f47680) at asio/asio/detail/impl/task_io_service.ipp:148
      #4  asio::io_service::run (this=<optimized out>) at asio/asio/impl/io_service.ipp:58
      #5  gcomm::AsioProtonet::event_loop (this=<optimized out>, period=...) at gcomm/src/asio_protonet.cpp:116
      #6  0x00007ffbaa68bf88 in GCommConn::run (this=0x562e26f46cc0) at gcs/src/gcs_gcomm.cpp:498
      #7  0x00007ffbaa6938d9 in GCommConn::run_fn (arg=<optimized out>) at gcs/src/gcs_gcomm.cpp:209
      #8  0x00007ffbb16e16ba in start_thread (arg=0x7ffb9bffe700) at pthread_create.c:333
      #9  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 2 (Thread 0x7ffb9b7fd700 (LWP 6638)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00007ffbaa689bab in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #2  RecvBuf::front (timeout=..., this=0x562e26f46ef8) at gcs/src/gcs_gcomm.cpp:124
      #3  gcomm_recv (backend=<optimized out>, msg=0x562e26f1aac0, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:673
      #4  0x00007ffbaa679e98 in core_msg_recv (timeout=9223372035999999999, recv_msg=0x562e26f1aac0, backend=0x562e26f1ac20) at gcs/src/gcs_core.cpp:480
      #5  gcs_core_recv (conn=0x562e26f1aa60, recv_act=recv_act@entry=0x7ffb9b7fcc80, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1184
      #6  0x00007ffbaa68612f in gcs_recv_thread (arg=0x562e26f1a7e0) at gcs/src/gcs.cpp:1398
      #7  0x00007ffbb16e16ba in start_thread (arg=0x7ffb9b7fd700) at pthread_create.c:333
      #8  0x00007ffbb04c441d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 1 (Thread 0x7ffbb22f18c0 (LWP 6599)):
      #0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
      #1  0x0000562e25944e97 in my_write_core (sig=sig@entry=6) at /home/buildbot/buildbot/build/mariadb-10.4.12/mysys/stacktrace.c:481
      #2  0x0000562e253d721e in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/signal_handler.cc:343
      #3  <signal handler called>
      #4  0x00007ffbb04b874d in poll () at ../sysdeps/unix/syscall-template.S:84
      #5  0x00007ffbb04d60be in __poll_chk (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, fdslen=<optimized out>) at poll_chk.c:27
      #6  0x0000562e2510130f in poll (__timeout=-1, __nfds=2, __fds=0x7ffff807ae30) at /usr/include/x86_64-linux-gnu/bits/poll2.h:41
      #7  handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:6456
      #8  0x0000562e25102816 in mysqld_main (argc=163, argv=0x562e26db0848) at /home/buildbot/buildbot/build/mariadb-10.4.12/sql/mysqld.cc:5892
      #9  0x00007ffbb03dd830 in __libc_start_main (main=0x562e250db9e0 <main(int, char**)>, argc=23, argv=0x7ffff807b568, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffff807b558) at ../csu/libc-start.c:291
      #10 0x0000562e250f5199 in _start ()
       
      Compressed file /dev/shm/var/1/log/galera.galera_ftwrl-innodb/mysqld.2/data/core
       - saving '/dev/shm/var/1/log/galera.galera_ftwrl-innodb/' to '/dev/shm/var/log/galera.galera_ftwrl-innodb/'
       
      Retrying test galera.galera_ftwrl, attempt(2/3)...
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jplindst Jan Lindström
              Reporter:
              stepan.patryshev Stepan Patryshev
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated: