Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-22549

galera.galera_var_auto_inc_control_off MTR failed: Program terminated with signal SIGABRT

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 10.4.13
    • Fix Version/s: 10.4
    • Component/s: Galera, Tests
    • Labels:
      None
    • Environment:
      kvm-deb-xenial-x86

      Description

      galera.galera_var_auto_inc_control_off failed on bb 10.4 CS: "Program terminated with signal SIGABRT, Aborted".

      stdio.log:

      10.4.13 CS, 93475aff8de80a0ef53cbee924bcb70de6e86f2c, kvm-deb-xenial-x86

      galera.galera_var_auto_inc_control_off 'innodb' w1 [ fail ]  timeout after 900 seconds
              Test ended at 2020-04-24 08:37:52
       
      Test case timeout after 900 seconds
       
      == /dev/shm/var/1/log/galera_var_auto_inc_control_off.log == 
      SELECT f1 FROM t1;
      f1
      1
      connection node_2a;
      SELECT @@auto_increment_increment = 1;
      @@auto_increment_increment = 1
      1
      SELECT @@auto_increment_offset = 1;
      @@auto_increment_offset = 1
      1
      SET AUTOCOMMIT=OFF;
      START TRANSACTION;
      INSERT INTO t1 (node) VALUES ('node2');
      SELECT f1 FROM t1;
      f1
      1
      connection node_1a;
      COMMIT;
      connection node_2a;
      COMMIT;
       
       == /dev/shm/var/1/tmp/analyze-timeout-mysqld.1.err ==
      mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/dev/shm/var/tmp/1/mysqld.1.sock' (111)
       
       == /dev/shm/var/1/tmp/analyze-timeout-mysqld.2.err ==
      mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/dev/shm/var/tmp/1/mysqld.2.sock' (111)
       
       
       - found 'core' (0/0)
       
      Trying 'dbx' to get a backtrace
       
      Trying 'gdb' to get a backtrace from coredump /dev/shm/var/1/log/galera.galera_var_auto_inc_control_off-innodb/mysqld.1/data/core
      Core generated by '/usr/sbin/mysqld'
      Output from gdb follows. The first stack trace is from the failing thread.
      The following stack traces are from all threads (so the failing one is
      duplicated).
      --------------------------
      [New LWP 3645]
      [New LWP 3666]
      [New LWP 3661]
      [New LWP 3649]
      [New LWP 3659]
      [New LWP 3664]
      [New LWP 3674]
      [New LWP 3662]
      [New LWP 3667]
      [New LWP 3658]
      [New LWP 3663]
      [New LWP 3665]
      [New LWP 3653]
      [New LWP 3891]
      [New LWP 3657]
      [New LWP 3679]
      [New LWP 3654]
      [New LWP 3678]
      [New LWP 3656]
      [New LWP 3673]
      [New LWP 3892]
      [New LWP 3655]
      [New LWP 3652]
      [New LWP 3651]
      [New LWP 3650]
      [New LWP 3648]
      [New LWP 3647]
      [New LWP 3646]
      [New LWP 3672]
      [New LWP 3671]
      [New LWP 3670]
      [New LWP 3669]
      [New LWP 3668]
      [New LWP 3675]
      [Thread debugging using libthread_db enabled]
      Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
      Core was generated by `/usr/sbin/mysqld --defaults-group-suffix=.1 --defaults-file=/dev/shm/var/1/my.c'.
      Program terminated with signal SIGABRT, Aborted.
      #0  0xb77cfc31 in __kernel_vsyscall ()
      [Current thread is 1 (Thread 0xb6f0e7c0 (LWP 3645))]
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774fcc1 in __pthread_kill (threadid=3069241280, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:58
      #2  0x80be87cc in my_write_core (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/stacktrace.c:481
      #3  0x8064c846 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:343
      #4  <signal handler called>
      #5  0xb77cfc31 in __kernel_vsyscall ()
      #6  0xb719aecf in poll () at ../sysdeps/unix/syscall-template.S:84
      #7  0xb71b5630 in __poll_chk (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, fdslen=24) at poll_chk.c:27
      #8  0x803517c7 in poll (__timeout=-1, __nfds=2, __fds=0xbf97f370) at /usr/include/i386-linux-gnu/bits/poll2.h:41
      #9  handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:6473
      #10 0x80352d74 in mysqld_main (argc=162, argv=0xbf97f684) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5909
      #11 0x8032cc77 in main (argc=23, argv=0xbf97f684) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/main.cc:25
       
      Thread 34 (Thread 0xac0feb40 (LWP 3675)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb70eaeac in __GI___sigwaitinfo (set=<optimized out>, info=0xac0fe1b4) at ../sysdeps/unix/sysv/linux/sigwaitinfo.c:56
      #2  0x80347fbf in my_sigwait (code=<synthetic pointer>, sig=0xac0fe130, set=0xac0fe134) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/my_pthread.h:196
      #3  signal_hand (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:3248
      #4  0x80b9548b in pfs_spawn_thread (arg=0x820aaae0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #5  0xb7749295 in start_thread (arg=0xac0feb40) at pthread_create.c:333
      #6  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 33 (Thread 0xa4cebb40 (LWP 3668)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80865249 in os_event::wait (this=0x81f48f98) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
      #3  os_event::wait_low (reset_sig_count=393, this=0x81f48f98) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
      #4  os_event_wait_low (event=0x81f48f98, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
      #5  0x808f3e94 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x8131c90c <srv_sys+140>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:911
      #6  srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2519
      #7  0xb7749295 in start_thread (arg=0xa4cebb40) at pthread_create.c:333
      #8  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 32 (Thread 0xa44eab40 (LWP 3669)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80865249 in os_event::wait (this=0x81f49048) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
      #3  os_event::wait_low (reset_sig_count=414, this=0x81f49048) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
      #4  os_event_wait_low (event=0x81f49048, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
      #5  0x808f3e94 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x8131c944 <srv_sys+196>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:911
      #6  srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2519
      #7  0xb7749295 in start_thread (arg=0xa44eab40) at pthread_create.c:333
      #8  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 31 (Thread 0xa3ce9b40 (LWP 3670)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80865249 in os_event::wait (this=0x81f48ff0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
      #3  os_event::wait_low (reset_sig_count=324, this=0x81f48ff0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
      #4  os_event_wait_low (event=0x81f48ff0, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
      #5  0x808f3e94 in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x8131c928 <srv_sys+168>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:911
      #6  srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2519
      #7  0xb7749295 in start_thread (arg=0xa3ce9b40) at pthread_create.c:333
      #8  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 30 (Thread 0xa34e8b40 (LWP 3671)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80865249 in os_event::wait (this=0x81f49150) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
      #3  os_event::wait_low (reset_sig_count=1, this=0x81f49150) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
      #4  os_event_wait_low (event=0x81f49150, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
      #5  0x8098d0da in buf_dump_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0dump.cc:833
      #6  0xb7749295 in start_thread (arg=0xa34e8b40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 29 (Thread 0xa2ce7b40 (LWP 3672)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb775288a in nanosleep () at ../sysdeps/unix/syscall-template.S:84
      #2  0x808655f0 in os_thread_sleep (tm=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0thread.cc:231
      #3  0x8097640f in btr_defragment_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/btr/btr0defragment.cc:717
      #4  0xb7749295 in start_thread (arg=0xa2ce7b40) at pthread_create.c:333
      #5  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 28 (Thread 0xb2c73b40 (LWP 3646)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80bed42d in inline_mysql_cond_timedwait (that=0x81bf3360 <COND_timer>, mutex=0x81bf3394 <LOCK_timer>, src_file=0x80e5ac28 "/home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c", src_line=292, abstime=0xb2c73224) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
      #3  timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/thr_timer.c:292
      #4  0x80b9548b in pfs_spawn_thread (arg=0x81e274f8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #5  0xb7749295 in start_thread (arg=0xb2c73b40) at pthread_create.c:333
      #6  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 27 (Thread 0xb159cb40 (LWP 3647)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0xb2220a96 in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #3  galera::ServiceThd::thd_func (arg=0x81e4bebc) at galera/src/galera_service_thd.cpp:30
      #4  0xb7749295 in start_thread (arg=0xb159cb40) at pthread_create.c:333
      #5  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 26 (Thread 0xaf898b40 (LWP 3648)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb71a588a in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
      #2  0xb21889a6 in asio::detail::epoll_reactor::run (this=0x81e9b000, block=true, ops=...) at asio/asio/detail/impl/epoll_reactor.ipp:391
      #3  0xb21a888b in asio::detail::task_io_service::do_run_one (ec=..., this_thread=..., lock=..., this=0x81e9af70) at asio/asio/detail/impl/task_io_service.ipp:355
      #4  asio::detail::task_io_service::run (ec=..., this=0x81e9af70) at asio/asio/detail/impl/task_io_service.ipp:148
      #5  asio::io_service::run (this=0x81e9a5ac) at asio/asio/impl/io_service.ipp:58
      #6  gcomm::AsioProtonet::event_loop (this=0x81e9a540, period=...) at gcomm/src/asio_protonet.cpp:116
      #7  0xb21df412 in GCommConn::run (this=this@entry=0x81e9a718) at gcs/src/gcs_gcomm.cpp:433
      #8  0xb21e705b in GCommConn::run_fn (arg=0x81e9a718) at gcs/src/gcs_gcomm.cpp:174
      #9  0xb7749295 in start_thread (arg=0xaf898b40) at pthread_create.c:333
      #10 0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 25 (Thread 0xb2472b40 (LWP 3650)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x805aadbd in inline_mysql_cond_wait (src_line=69, src_file=0x80cea2a4 "/home/buildbot/buildbot/build/mariadb-10.4.13/sql/wsrep_thd.h", mutex=0xae61009c, that=0xae6100b8) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1185
      #3  Wsrep_thd_queue::pop_front (this=0xae610070) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/wsrep_thd.h:69
      #4  wsrep_rollback_process (rollbacker=0xae600510, arg=0x81e9b9b8) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/wsrep_thd.cc:241
      #5  0x8059bfbd in start_wsrep_THD (arg=0x81e9b9b8) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/wsrep_mysqld.cc:2792
      #6  0x80b9548b in pfs_spawn_thread (arg=0x81eaf6d0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #7  0xb7749295 in start_thread (arg=0xb2472b40) at pthread_create.c:333
      #8  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 24 (Thread 0xae896b40 (LWP 3651)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0xb203654d in fifo_lock_get (q=0x81e85b10) at galerautils/src/gu_fifo.c:262
      #3  gu_fifo_get_head (q=0x81e85b10, err=err@entry=0xae895d08) at galerautils/src/gu_fifo.c:330
      #4  0xb21ce71f in gcs_recv (conn=0x81e515f8, action=action@entry=0xae895da0) at gcs/src/gcs.cpp:2024
      #5  0xb222f33e in galera::Gcs::recv (act=..., this=<optimized out>) at galera/src/galera_gcs.hpp:120
      #6  galera::GcsActionSource::process (this=0x81e96c98, recv_ctx=0xae89617c, exit_loop=@0xae895ee3: false) at galera/src/gcs_action_source.cpp:162
      #7  0xb227182a in galera::ReplicatorSMM::async_recv (this=0x81e4b9c0, recv_ctx=0xae89617c) at galera/src/replicator_smm.cpp:390
      #8  0xb2297bac in galera_recv (gh=0x81e274f0, recv_ctx=0xae89617c) at galera/src/wsrep_provider.cpp:236
      #9  0x80c99e5f in wsrep::wsrep_provider_v26::run_applier (this=0x81d22230, applier_ctx=0xae89617c) at /home/buildbot/buildbot/build/mariadb-10.4.13/wsrep-lib/src/wsrep_provider_v26.cpp:731
      #10 0x805abe65 in wsrep_replication_process (thd=0xae300510, arg=0x81eb04c8) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/wsrep_thd.cc:58
      #11 0x8059bfbd in start_wsrep_THD (arg=0x81eb04c8) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/wsrep_mysqld.cc:2792
      #12 0x80b9548b in pfs_spawn_thread (arg=0x81eafa40) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #13 0xb7749295 in start_thread (arg=0xae896b40) at pthread_create.c:333
      #14 0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 23 (Thread 0xad81cb40 (LWP 3652)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80aa5bfd in inline_mysql_cond_timedwait (src_file=0x80e451fc "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c", src_line=116, abstime=0xad81c194, mutex=0x81be75d4 <LOCK_checkpoint>, that=0x81be75a0 <COND_checkpoint>) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
      #3  my_service_thread_sleep (control=0x8131e464 <checkpoint_control>, sleep_time=30000000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_servicethread.c:115
      #4  0x80a9c35e in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/maria/ma_checkpoint.c:707
      #5  0x80b9548b in pfs_spawn_thread (arg=0x81f1e8f0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #6  0xb7749295 in start_thread (arg=0xad81cb40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 22 (Thread 0xaaefcb40 (LWP 3655)):
      #0  0xb77ad570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
      #1  0xb77ad5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
      #2  0x8085a277 in LinuxAIOHandler::collect (this=0xaaefc114) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
      #3  0x8085a97b in LinuxAIOHandler::poll (this=0xaaefc114, m1=0xaaefc180, m2=0xaaefc184, request=0xaaefc188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
      #4  0x8085f337 in os_aio_linux_handler (request=0xaaefc188, m2=0xaaefc184, m1=0xaaefc180, global_segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
      #5  os_aio_handler (segment=2, m1=0xaaefc180, m2=0xaaefc184, request=0xaaefc188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
      #6  0x809ebdd0 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4350
      #7  0x808f6e41 in io_handler_thread (arg=0x813d98c8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
      #8  0xb7749295 in start_thread (arg=0xaaefcb40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 21 (Thread 0xabfd6b40 (LWP 3892)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80349b74 in inline_mysql_cond_timedwait (src_file=0x80cad008 "/home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc", src_line=2691, abstime=0xabfd6174, mutex=0x81382554 <LOCK_thread_cache>, that=0x81384500 <COND_thread_cache>) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
      #3  cache_thread (thd=0xae518c80) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:2690
      #4  one_thread_per_connection_end (thd=0xae518c80, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:2776
      #5  0x805033cf in do_handle_one_connection (connect=0x81ecfd98) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_connect.cc:1423
      #6  0x80503532 in handle_one_connection (arg=0x81ecfd98) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_connect.cc:1316
      #7  0x80b9548b in pfs_spawn_thread (arg=0x82160a80) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #8  0xb7749295 in start_thread (arg=0xabfd6b40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 20 (Thread 0xa24e6b40 (LWP 3673)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80865249 in os_event::wait (this=0x81f49200) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
      #3  os_event::wait_low (reset_sig_count=1, this=0x81f49200) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
      #4  os_event_wait_low (event=0x81f49200, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
      #5  0x80984efc in buf_resize_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0buf.cc:3159
      #6  0xb7749295 in start_thread (arg=0xa24e6b40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 19 (Thread 0xaa6fbb40 (LWP 3656)):
      #0  0xb77ad570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
      #1  0xb77ad5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
      #2  0x8085a277 in LinuxAIOHandler::collect (this=0xaa6fb114) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
      #3  0x8085a97b in LinuxAIOHandler::poll (this=0xaa6fb114, m1=0xaa6fb180, m2=0xaa6fb184, request=0xaa6fb188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
      #4  0x8085f337 in os_aio_linux_handler (request=0xaa6fb188, m2=0xaa6fb184, m1=0xaa6fb180, global_segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
      #5  os_aio_handler (segment=3, m1=0xaa6fb180, m2=0xaa6fb184, request=0xaa6fb188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
      #6  0x809ebdd0 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4350
      #7  0x808f6e41 in io_handler_thread (arg=0x813d98cc <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
      #8  0xb7749295 in start_thread (arg=0xaa6fbb40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 18 (Thread 0xac0b4b40 (LWP 3678)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80367766 in inline_mysql_cond_wait (that=<optimized out>, mutex=<optimized out>, src_file=0x80cb7680 "/home/buildbot/buildbot/build/mariadb-10.4.13/sql/slave.cc", src_line=522) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1174
      #3  handle_slave_background (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/slave.cc:522
      #4  0x80b9548b in pfs_spawn_thread (arg=0x8212eaf0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #5  0xb7749295 in start_thread (arg=0xac0b4b40) at pthread_create.c:333
      #6  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 17 (Thread 0xab6fdb40 (LWP 3654)):
      #0  0xb77ad570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
      #1  0xb77ad5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
      #2  0x8085a277 in LinuxAIOHandler::collect (this=0xab6fd114) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
      #3  0x8085a97b in LinuxAIOHandler::poll (this=0xab6fd114, m1=0xab6fd180, m2=0xab6fd184, request=0xab6fd188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
      #4  0x8085f337 in os_aio_linux_handler (request=0xab6fd188, m2=0xab6fd184, m1=0xab6fd180, global_segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
      #5  os_aio_handler (segment=1, m1=0xab6fd180, m2=0xab6fd184, request=0xab6fd188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
      #6  0x809ebdd0 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4350
      #7  0x808f6e41 in io_handler_thread (arg=0x813d98c4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
      #8  0xb7749295 in start_thread (arg=0xab6fdb40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 16 (Thread 0xac06ab40 (LWP 3679)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80349b74 in inline_mysql_cond_timedwait (src_file=0x80cad008 "/home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc", src_line=2691, abstime=0xac06a174, mutex=0x81382554 <LOCK_thread_cache>, that=0x81384500 <COND_thread_cache>) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
      #3  cache_thread (thd=0xae705198) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:2690
      #4  one_thread_per_connection_end (thd=0xae705198, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:2776
      #5  0x805033cf in do_handle_one_connection (connect=0x820aaff0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_connect.cc:1423
      #6  0x80503532 in handle_one_connection (arg=0x820aaff0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_connect.cc:1316
      #7  0x80b9548b in pfs_spawn_thread (arg=0x82160a80) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #8  0xb7749295 in start_thread (arg=0xac06ab40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 15 (Thread 0xa9efab40 (LWP 3657)):
      #0  0xb77ad570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
      #1  0xb77ad5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
      #2  0x8085a277 in LinuxAIOHandler::collect (this=0xa9efa114) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
      #3  0x8085a97b in LinuxAIOHandler::poll (this=0xa9efa114, m1=0xa9efa180, m2=0xa9efa184, request=0xa9efa188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
      #4  0x8085f337 in os_aio_linux_handler (request=0xa9efa188, m2=0xa9efa184, m1=0xa9efa180, global_segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
      #5  os_aio_handler (segment=4, m1=0xa9efa180, m2=0xa9efa184, request=0xa9efa188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
      #6  0x809ebdd0 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4350
      #7  0x808f6e41 in io_handler_thread (arg=0x813d98d0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
      #8  0xb7749295 in start_thread (arg=0xa9efab40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 14 (Thread 0xac020b40 (LWP 3891)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80349b74 in inline_mysql_cond_timedwait (src_file=0x80cad008 "/home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc", src_line=2691, abstime=0xac020174, mutex=0x81382554 <LOCK_thread_cache>, that=0x81384500 <COND_thread_cache>) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1211
      #3  cache_thread (thd=0xae6153e0) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:2690
      #4  one_thread_per_connection_end (thd=0xae6153e0, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:2776
      #5  0x805033cf in do_handle_one_connection (connect=0x81ecfd98) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_connect.cc:1423
      #6  0x80503532 in handle_one_connection (arg=0x81ecfd98) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/sql_connect.cc:1316
      #7  0x80b9548b in pfs_spawn_thread (arg=0x821608f0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #8  0xb7749295 in start_thread (arg=0xac020b40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 13 (Thread 0xabefeb40 (LWP 3653)):
      #0  0xb77ad570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
      #1  0xb77ad5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
      #2  0x8085a277 in LinuxAIOHandler::collect (this=0xabefe114) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
      #3  0x8085a97b in LinuxAIOHandler::poll (this=0xabefe114, m1=0xabefe180, m2=0xabefe184, request=0xabefe188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
      #4  0x8085f337 in os_aio_linux_handler (request=0xabefe188, m2=0xabefe184, m1=0xabefe180, global_segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
      #5  os_aio_handler (segment=0, m1=0xabefe180, m2=0xabefe184, request=0xabefe188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
      #6  0x809ebdd0 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4350
      #7  0x808f6e41 in io_handler_thread (arg=0x813d98c0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
      #8  0xb7749295 in start_thread (arg=0xabefeb40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 12 (Thread 0xa84f2b40 (LWP 3665)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80864bf5 in os_event::timed_wait (abstime=0xa84f1f7c, this=0x8203a418) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:275
      #3  os_event::wait_time_low (this=0x8203a418, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:385
      #4  0x808651d9 in os_event_wait_time_low (event=0x8203a418, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:485
      #5  0x80941eb8 in ib_wqueue_timedwait (wq=0x82087a68, wait_in_usecs=5000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/ut/ut0wqueue.cc:156
      #6  0x80a29233 in fts_optimize_thread (arg=0x82087a68) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fts/fts0opt.cc:2834
      #7  0xb7749295 in start_thread (arg=0xa84f2b40) at pthread_create.c:333
      #8  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 11 (Thread 0xa6cefb40 (LWP 3663)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80864bf5 in os_event::timed_wait (abstime=0xa6cef10c, this=0x81f490f8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:275
      #3  os_event::wait_time_low (this=0x81f490f8, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:385
      #4  0x808651d9 in os_event_wait_time_low (event=0x81f490f8, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:485
      #5  0x808f18df in srv_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:1689
      #6  0xb7749295 in start_thread (arg=0xa6cefb40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 10 (Thread 0xa96f9b40 (LWP 3658)):
      #0  0xb77ad570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
      #1  0xb77ad5e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
      #2  0x8085a277 in LinuxAIOHandler::collect (this=0xa96f9114) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1819
      #3  0x8085a97b in LinuxAIOHandler::poll (this=0xa96f9114, m1=0xa96f9180, m2=0xa96f9184, request=0xa96f9188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:1980
      #4  0x8085f337 in os_aio_linux_handler (request=0xa96f9188, m2=0xa96f9184, m1=0xa96f9180, global_segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:2034
      #5  os_aio_handler (segment=5, m1=0xa96f9180, m2=0xa96f9184, request=0xa96f9188) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0file.cc:5604
      #6  0x809ebdd0 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/fil/fil0fil.cc:4350
      #7  0x808f6e41 in io_handler_thread (arg=0x813d98d4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0start.cc:324
      #8  0xb7749295 in start_thread (arg=0xa96f9b40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 9 (Thread 0xa54ecb40 (LWP 3667)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x80865249 in os_event::wait (this=0x81f48f40) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:158
      #3  os_event::wait_low (reset_sig_count=131, this=0x81f48f40) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:325
      #4  os_event_wait_low (event=0x81f48f40, reset_sig_count=131) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:502
      #5  0x808f51d0 in srv_resume_thread (timeout_usec=0, wait=true, sig_count=131, slot=0x8131c8f0 <srv_sys+112>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:911
      #6  srv_purge_coordinator_suspend (rseg_history_len=<optimized out>, slot=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2659
      #7  srv_purge_coordinator_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2740
      #8  0xb7749295 in start_thread (arg=0xa54ecb40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 8 (Thread 0xa74f0b40 (LWP 3662)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80864bf5 in os_event::timed_wait (abstime=0xa74f008c, this=0x81f490a0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:275
      #3  os_event::wait_time_low (this=0x81f490a0, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:385
      #4  0x808651d9 in os_event_wait_time_low (event=0x81f490a0, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:485
      #5  0x808f2049 in srv_error_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:1853
      #6  0xb7749295 in start_thread (arg=0xa74f0b40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 7 (Thread 0xa1ce5b40 (LWP 3674)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0x807dc147 in inline_mysql_cond_wait (src_file=0x80df65bc "/home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/handler/ha_innodb.cc", src_line=288, mutex=0xa1ce51fc, that=0xa1ce5218) at /home/buildbot/buildbot/build/mariadb-10.4.13/include/mysql/psi/mysql_thread.h:1185
      #3  thd_destructor_proxy () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/handler/ha_innodb.cc:288
      #4  0x80b9548b in pfs_spawn_thread (arg=0x820b1460) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/perfschema/pfs.cc:1869
      #5  0xb7749295 in start_thread (arg=0xa1ce5b40) at pthread_create.c:333
      #6  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 6 (Thread 0xa64eeb40 (LWP 3664)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80864bf5 in os_event::timed_wait (abstime=0xa64ee0ac, this=0x8207d5a8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:275
      #3  os_event::wait_time_low (this=0x8207d5a8, time_in_usec=10000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:385
      #4  0x808651d9 in os_event_wait_time_low (event=0x8207d5a8, time_in_usec=10000000, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:485
      #5  0x809dfce4 in dict_stats_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/dict/dict0stats_bg.cc:476
      #6  0xb7749295 in start_thread (arg=0xa64eeb40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 5 (Thread 0xa8ef8b40 (LWP 3659)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80864bf5 in os_event::timed_wait (abstime=0xa8ef7f4c, this=0x81f491a8) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:275
      #3  os_event::wait_time_low (this=0x81f491a8, time_in_usec=30000, reset_sig_count=90) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:385
      #4  0x808651d9 in os_event_wait_time_low (event=0x81f491a8, time_in_usec=30000, reset_sig_count=90) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:485
      #5  0x80998476 in pc_sleep_if_needed (cur_time=1033883, sig_count=90, next_loop_time=1033913) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0flu.cc:2640
      #6  buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/buf/buf0flu.cc:3099
      #7  0xb7749295 in start_thread (arg=0xa8ef8b40) at pthread_create.c:333
      #8  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 4 (Thread 0xaf097b40 (LWP 3649)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ea6c in pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/pthread_cond_wait.S:187
      #2  0xb21dcbee in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #3  RecvBuf::front (timeout=..., this=0x81e9a84c) at gcs/src/gcs_gcomm.cpp:114
      #4  gcomm_recv (backend=0x81e51950, msg=0x81e51834, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:608
      #5  0xb21c95e1 in core_msg_recv (timeout=<optimized out>, recv_msg=<optimized out>, backend=0x81e51950) at gcs/src/gcs_core.cpp:480
      #6  gcs_core_recv (conn=0x81e517f8, recv_act=recv_act@entry=0xaf0971c4, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1184
      #7  0xb21d8259 in gcs_recv_thread (arg=0x81e515f8) at gcs/src/gcs.cpp:1398
      #8  0xb7749295 in start_thread (arg=0xaf097b40) at pthread_create.c:333
      #9  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 3 (Thread 0xa7cf1b40 (LWP 3661)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774ee36 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/i386/i686/../pthread_cond_timedwait.S:245
      #2  0x80864bf5 in os_event::timed_wait (abstime=0xa7cf10ac, this=0x820548d0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:275
      #3  os_event::wait_time_low (this=0x820548d0, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:385
      #4  0x808651d9 in os_event_wait_time_low (event=0x820548d0, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0event.cc:485
      #5  0x80835ef9 in lock_wait_timeout_thread () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/lock/lock0wait.cc:522
      #6  0xb7749295 in start_thread (arg=0xa7cf1b40) at pthread_create.c:333
      #7  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 2 (Thread 0xa5cedb40 (LWP 3666)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb775288a in nanosleep () at ../sysdeps/unix/syscall-template.S:84
      #2  0x808655f0 in os_thread_sleep (tm=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/os/os0thread.cc:231
      #3  0x808f3157 in srv_master_sleep () at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2336
      #4  srv_master_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.13/storage/innobase/srv/srv0srv.cc:2377
      #5  0xb7749295 in start_thread (arg=0xa5cedb40) at pthread_create.c:333
      #6  0xb71a50ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
       
      Thread 1 (Thread 0xb6f0e7c0 (LWP 3645)):
      #0  0xb77cfc31 in __kernel_vsyscall ()
      #1  0xb774fcc1 in __pthread_kill (threadid=3069241280, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:58
      #2  0x80be87cc in my_write_core (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/mysys/stacktrace.c:481
      #3  0x8064c846 in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/signal_handler.cc:343
      #4  <signal handler called>
      #5  0xb77cfc31 in __kernel_vsyscall ()
      #6  0xb719aecf in poll () at ../sysdeps/unix/syscall-template.S:84
      #7  0xb71b5630 in __poll_chk (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, fdslen=24) at poll_chk.c:27
      #8  0x803517c7 in poll (__timeout=-1, __nfds=2, __fds=0xbf97f370) at /usr/include/i386-linux-gnu/bits/poll2.h:41
      #9  handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:6473
      #10 0x80352d74 in mysqld_main (argc=162, argv=0xbf97f684) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/mysqld.cc:5909
      #11 0x8032cc77 in main (argc=23, argv=0xbf97f684) at /home/buildbot/buildbot/build/mariadb-10.4.13/sql/main.cc:25
       
      Compressed file /dev/shm/var/1/log/galera.galera_var_auto_inc_control_off-innodb/mysqld.1/data/core
       - saving '/dev/shm/var/1/log/galera.galera_var_auto_inc_control_off-innodb/' to '/dev/shm/var/log/galera.galera_var_auto_inc_control_off-innodb/'
       
      Retrying test galera.galera_var_auto_inc_control_off, attempt(2/3)...
       
      worker[1] > Restart  - not started
      worker[1] > Restart  - not started
      galera.galera_sst_rsync2 'innodb,release' w2 [ pass ]  31020
      worker[2] > Restart [mysqld.1 - pid: 16426, winpid: 16426] - using different config file
      worker[2] > Restart [mysqld.2 - pid: 17165, winpid: 17165] - using different config file
      ***Warnings generated in error logs during shutdown after running tests: galera.galera_var_auto_inc_control_off galera.galera_truncate galera.galera_var_OSU_method galera.galera_var_OSU_method2 galera.galera_toi_lock_exclusive galera.galera_update_limit galera.galera_truncate_temporary galera.galera_toi_truncate galera.galera_unicode_identifiers galera.galera_toi_ftwrl galera.galera_toi_drop_database galera.galera_transaction_replay galera.galera_transaction_read_only galera.galera_toi_lock_shared galera.galera_unicode_pk
       
      Attempting backtrace. You can use the following information to find out
      2020-04-24  8:26:01 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:26:32 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:27:03 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:27:34 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:28:05 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:28:36 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:29:07 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:29:38 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:30:09 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:30:40 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:31:11 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:31:42 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:32:13 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:32:44 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:33:15 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:33:46 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:34:17 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:34:48 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:35:19 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:35:50 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:36:21 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:36:52 0 [Warning] InnoDB: A long semaphore wait:
      2020-04-24  8:36:52 0 [Warning] InnoDB: A long semaphore wait:
      Attempting backtrace. You can use the following information to find out
       
      galera.galera_var_auto_inc_control_off 'innodb' w1 [ retry-pass ]   4026
       
      Retrying test galera.galera_var_auto_inc_control_off, attempt(3/3)...
       
      worker[1] > Restart [mysqld.1 - pid: 17355, winpid: 17355] - using different config file
      worker[1] > Restart [mysqld.2 - pid: 17393, winpid: 17393] - using different config file
      connection node_2;
      connection node_1;
      CREATE TABLE t1 (f1 INT PRIMARY KEY) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (1);
      connection node_2;
      SELECT COUNT(*) = 1 FROM t1;
      COUNT(*) = 1
      1
      connection node_1;
      UPDATE t1 SET f1 = 2 WHERE f1 = 1;
      connection node_2;
      SELECT COUNT(*) = 1 FROM t1 WHERE f1 = 2;
      COUNT(*) = 1
      1
      DROP TABLE t1;
      galera.galera_v1_row_events 'innodb'     w2 [ pass ]   2018
      worker[2] > Restart [mysqld.1 - pid: 17620, winpid: 17620] - using different config file
      worker[2] > Restart [mysqld.2 - pid: 17656, winpid: 17656] - using different config file
      galera.galera_var_auto_inc_control_on 'innodb' w1 [ pass ]   2019
      worker[1] > Restart [mysqld.1 - pid: 17880, winpid: 17880] - using different config file
      worker[1] > Restart [mysqld.2 - pid: 17916, winpid: 17916] - using different config file
      galera.galera_var_gtid_domain_id 'innodb' w2 [ pass ]   2020
      worker[2] > Restart [mysqld.1 - pid: 18140, winpid: 18140] - using different config file
      worker[2] > Restart [mysqld.2 - pid: 18177, winpid: 18177] - using different config file
      galera.galera_var_auto_inc_control_off 'innodb' w1 [ retry-pass ]   4023
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jplindst Jan Lindström
              Reporter:
              stepan.patryshev Stepan Patryshev
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated: