Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20848

Galera test failure on galera_sr.GCF-1060

Details

    Description

      galera_sr.GCF-1060 'innodb'              w1 [ fail ]
              Test ended at 2019-10-16 09:50:00
       
      CURRENT_TEST: galera_sr.GCF-1060
      mysqltest: In included file "./suite/galera/include/galera_concurrent_test.inc": 
      included from /usr/share/mysql/mysql-test/suite/galera_sr/t/GCF-1060.test at line 10:
      At line 64: query 'reap' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
       
      The result from queries just before the failure was:
      < snip >
      connect node_1a, 127.0.0.1, root, , test, $NODE_MYPORT_1;
      Running a concurrent test with the following queries:
      TRUNCATE TABLE t1
      INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')
      INSERT INTO t1 VALUE (4, 'z');
      connection node_1;
      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (f1 INTEGER PRIMARY KEY AUTO_INCREMENT, f2 CHAR(255)) ENGINE=InnoDB;
      SET SESSION wsrep_sync_wait = 0;
      SET SESSION wsrep_trx_fragment_size = 1;;;
      connection node_1a;
      SET SESSION wsrep_sync_wait = 0;
      SET SESSION wsrep_trx_fragment_size = 1;;;
      connection node_2;
      SET SESSION wsrep_sync_wait = 0;
      SET SESSION wsrep_trx_fragment_size = 1;;;
       
      More results from queries before failure can be found in /dev/shm/var/1/log/GCF-1060.log
       
       
      Server [mysqld.1 - pid: 19011, winpid: 19011, exit: 256] failed during test run
      Server log from this test:
      ----------SERVER LOG START-----------
      mysqld: /home/buildbot/buildbot/build/mariadb-10.4.9/wsrep-lib/src/transaction.cpp:1034: bool wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&): Assertion `0' failed.
      191016  9:49:59 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.4.9-MariaDB-1:10.4.9+maria~stretch-log
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=4
      max_threads=153
      thread_count=12
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63513 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x7fd3040009a8
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7fd35a5becc8 thread_stack 0x49000
      /usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5565b7fd9abe]
      /usr/sbin/mysqld(handle_fatal_signal+0x3af)[0x5565b7a66a3f]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0)[0x7fd37fe970e0]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcf)[0x7fd37e53cfff]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fd37e53e42a]
      /lib/x86_64-linux-gnu/libc.so.6(+0x2be67)[0x7fd37e535e67]
      /lib/x86_64-linux-gnu/libc.so.6(+0x2bf12)[0x7fd37e535f12]
      /usr/sbin/mysqld(_ZN5wsrep11transaction18abort_or_interruptERNS_11unique_lockINS_5mutexEEE+0xc9)[0x5565b807aba9]
      /usr/sbin/mysqld(_ZN5wsrep11transaction16certify_fragmentERNS_11unique_lockINS_5mutexEEE+0x73)[0x5565b807ac63]
      /usr/sbin/mysqld(_ZN5wsrep11transaction9after_rowEv+0x9d)[0x5565b807bd4d]
      /usr/sbin/mysqld(+0x86ac30)[0x5565b7a68c30]
      /usr/sbin/mysqld(_ZN7handler12ha_write_rowEPKh+0x24f)[0x5565b7a721cf]
      /usr/sbin/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_info+0x8d)[0x5565b781bdcd]
      /usr/sbin/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0x14fe)[0x5565b782384e]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x42ac)[0x5565b78526fc]
      /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_statebb+0x23a)[0x5565b785719a]
      /usr/sbin/mysqld(+0x6599f9)[0x5565b78579f9]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjbb+0x2294)[0x5565b785a664]
      /usr/sbin/mysqld(_Z10do_commandP3THD+0x11c)[0x5565b785affc]
      /usr/sbin/mysqld(_Z24do_handle_one_connectionP7CONNECT+0x242)[0x5565b7938352]
      /usr/sbin/mysqld(handle_one_connection+0x3d)[0x5565b793840d]
      /usr/sbin/mysqld(+0xd84fd1)[0x5565b7f82fd1]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4)[0x7fd37fe8d4a4]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fd37e5f2d0f]
       
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7fd30400fcd0): INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')
      Connection ID (thread ID): 8226
      Status: KILL_QUERY
       
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on
       
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      Writing a core file...
      Working directory at /dev/shm/var/1/mysqld.1/data
      Resource Limits:
      Limit                     Soft Limit           Hard Limit           Units     
      Max cpu time              unlimited            unlimited            seconds   
      Max file size             unlimited            unlimited            bytes     
      Max data size             unlimited            unlimited            bytes     
      Max stack size            8388608              unlimited            bytes     
      Max core file size        unlimited            unlimited            bytes     
      Max resident set          unlimited            unlimited            bytes     
      Max processes             23799                23799                processes 
      Max open files            1024                 1024                 files     
      Max locked memory         65536                65536                bytes     
      Max address space         unlimited            unlimited            bytes     
      Max file locks            unlimited            unlimited            locks     
      Max pending signals       23799                23799                signals   
      Max msgqueue size         819200               819200               bytes     
      Max nice priority         0                    0                    
      Max realtime priority     0                    0                    
      Max realtime timeout      unlimited            unlimited            us        
      Core pattern: core
       
      ----------SERVER LOG END-------------
       
       
       - found 'core' (0/0)
       
      Trying 'dbx' to get a backtrace
       
      Trying 'gdb' to get a backtrace from coredump /dev/shm/var/1/log/galera_sr.GCF-1060-innodb/mysqld.1/data/core
      Core generated by '/usr/sbin/mysqld'
      Output from gdb follows. The first stack trace is from the failing thread.
      The following stack traces are from all threads (so the failing one is
      duplicated).
      --------------------------
      [New LWP 19947]
      [New LWP 19034]
      [New LWP 19031]
      [New LWP 19012]
      [New LWP 19030]
      [New LWP 19017]
      [New LWP 19029]
      [New LWP 19018]
      [New LWP 20027]
      [New LWP 19028]
      [New LWP 19022]
      [New LWP 19026]
      [New LWP 19023]
      [New LWP 19046]
      [New LWP 19025]
      [New LWP 19035]
      [New LWP 19042]
      [New LWP 19016]
      [New LWP 19036]
      [New LWP 19039]
      [New LWP 19037]
      [New LWP 19015]
      [New LWP 19038]
      [New LWP 19041]
      [New LWP 19033]
      [New LWP 19040]
      [New LWP 19032]
      [New LWP 19043]
      [New LWP 19024]
      [New LWP 19948]
      [New LWP 19021]
      [New LWP 19020]
      [New LWP 19019]
      [New LWP 19014]
      [New LWP 19013]
      [Thread debugging using libthread_db enabled]
      Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
      Core was generated by `/usr/sbin/mysqld --defaults-group-suffix=.1 --defaults-file=/dev/shm/var/1/my.c'.
      Program terminated with signal SIGABRT, Aborted.
      #0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
      57	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
      [Current thread is 1 (Thread 0x7fd35a5bf700 (LWP 19947))]
      #0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
      #1  0x00005565b7fd9677 in my_write_core (sig=sig@entry=6) at ./mysys/stacktrace.c:481
      #2  0x00005565b7a669db in handle_fatal_signal (sig=6) at ./sql/signal_handler.cc:343
      #3  <signal handler called>
      #4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
      #5  0x00007fd37e53e42a in __GI_abort () at abort.c:89
      #6  0x00007fd37e535e67 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x5565b80c2962 "0", file=file@entry=0x5565b84ba4d0 "/home/buildbot/buildbot/build/mariadb-10.4.9/wsrep-lib/src/transaction.cpp", line=line@entry=1034, function=function@entry=0x5565b84b9d00 <wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)::__PRETTY_FUNCTION__> "bool wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)") at assert.c:92
      #7  0x00007fd37e535f12 in __GI___assert_fail (assertion=assertion@entry=0x5565b80c2962 "0", file=file@entry=0x5565b84ba4d0 "/home/buildbot/buildbot/build/mariadb-10.4.9/wsrep-lib/src/transaction.cpp", line=line@entry=1034, function=function@entry=0x5565b84b9d00 <wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)::__PRETTY_FUNCTION__> "bool wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)") at assert.c:101
      #8  0x00005565b807aba9 in wsrep::transaction::abort_or_interrupt (this=this@entry=0x7fd304006950, lock=...) at ./wsrep-lib/src/transaction.cpp:1034
      #9  0x00005565b807ac63 in wsrep::transaction::certify_fragment (this=this@entry=0x7fd304006950, lock=...) at ./wsrep-lib/src/transaction.cpp:1099
      #10 0x00005565b807bc20 in wsrep::transaction::streaming_step (this=this@entry=0x7fd304006950, lock=...) at ./wsrep-lib/src/transaction.cpp:1083
      #11 0x00005565b807bd4d in wsrep::transaction::after_row (this=this@entry=0x7fd304006950) at ./wsrep-lib/src/transaction.cpp:254
      #12 0x00005565b7a68c30 in wsrep::client_state::after_row (this=0x7fd3040068e8) at ./wsrep-lib/include/wsrep/client_state.hpp:339
      #13 wsrep_after_row (thd=0x7fd3040009a8) at ./sql/wsrep_trans_observer.h:170
      #14 wsrep_after_row (thd=0x7fd3040009a8) at ./sql/handler.cc:6485
      #15 0x00005565b7a721cf in handler::ha_write_row (this=0x7fd30401f6c0, buf=0x7fd30403e1c8 "\375\001") at ./sql/handler.cc:6684
      #16 0x00005565b781bdcd in write_record (thd=thd@entry=0x7fd3040009a8, table=table@entry=0x7fd3040198a8, info=info@entry=0x7fd35a5bb250) at ./sql/sql_insert.cc:2036
      #17 0x00005565b782384e in mysql_insert (thd=thd@entry=0x7fd3040009a8, table_list=<optimized out>, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at ./sql/sql_insert.cc:1076
      #18 0x00005565b78526fc in mysql_execute_command (thd=thd@entry=0x7fd3040009a8) at ./sql/sql_parse.cc:4524
      #19 0x00005565b785719a in mysql_parse (thd=thd@entry=0x7fd3040009a8, rawbuf=rawbuf@entry=0x7fd30400fcd0 "INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')", length=length@entry=44, parser_state=parser_state@entry=0x7fd35a5be000, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:7912
      #20 0x00005565b78579f9 in wsrep_mysql_parse (thd=thd@entry=0x7fd3040009a8, rawbuf=0x7fd30400fcd0 "INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')", length=44, parser_state=parser_state@entry=0x7fd35a5be000, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:7716
      #21 0x00005565b785a664 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7fd3040009a8, packet=packet@entry=0x7fd304007999 "INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')", packet_length=packet_length@entry=44, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:1826
      #22 0x00005565b785affc in do_command (thd=0x7fd3040009a8) at ./sql/sql_parse.cc:1359
      #23 0x00005565b7938352 in do_handle_one_connection (connect=connect@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1412
      #24 0x00005565b793840d in handle_one_connection (arg=arg@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1316
      #25 0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba9e32f8) at ./storage/perfschema/pfs.cc:1862
      #26 0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a5bf700) at pthread_create.c:456
      #27 0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 35 (Thread 0x7fd379080700 (LWP 19013)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7fde33d in inline_mysql_cond_timedwait (that=0x5565b9278a40 <COND_timer>, mutex=0x5565b9278a80 <LOCK_timer>, src_file=0x5565b82225c8 "/home/buildbot/buildbot/build/mariadb-10.4.9/mysys/thr_timer.c", src_line=292, abstime=0x7fd37907fce0) at ./include/mysql/psi/mysql_thread.h:1204
      #2  timer_handler (arg=arg@entry=0x0) at ./mysys/thr_timer.c:292
      #3  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba5d0908) at ./storage/perfschema/pfs.cc:1862
      #4  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd379080700) at pthread_create.c:456
      #5  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 34 (Thread 0x7fd3735fe700 (LWP 19014)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00007fd3785b9cc0 in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #2  galera::ServiceThd::thd_func (arg=0x5565ba6394d8) at galera/src/galera_service_thd.cpp:30
      #3  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3735fe700) at pthread_create.c:456
      #4  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 33 (Thread 0x7fd35b856700 (LWP 19019)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7eac98c in inline_mysql_cond_timedwait (src_file=0x5565b820c710 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/maria/ma_servicethread.c", src_line=116, abstime=0x7fd35b855c50, mutex=0x5565b9268a80 <LOCK_checkpoint>, that=0x5565b9268a40 <COND_checkpoint>) at ./include/mysql/psi/mysql_thread.h:1204
      #2  my_service_thread_sleep (control=control@entry=0x5565b89647a0 <checkpoint_control>, sleep_time=sleep_time@entry=29000000000) at ./storage/maria/ma_servicethread.c:115
      #3  0x00005565b7ea3f8e in ma_checkpoint_background (arg=arg@entry=0x1e) at ./storage/maria/ma_checkpoint.c:707
      #4  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba7377e8) at ./storage/perfschema/pfs.cc:1862
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35b856700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 32 (Thread 0x7fd35a455700 (LWP 19020)):
      #0  0x00007fd37fc846ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005565b7c7aff1 in LinuxAIOHandler::collect (this=this@entry=0x7fd35a454b00) at ./storage/innobase/os/os0file.cc:1809
      #2  0x00005565b7c7b632 in LinuxAIOHandler::poll (this=this@entry=0x7fd35a454b00, m1=m1@entry=0x7fd35a454b90, m2=m2@entry=0x7fd35a454b98, request=request@entry=0x7fd35a454ba0) at ./storage/innobase/os/os0file.cc:1968
      #3  0x00005565b7c7f728 in os_aio_linux_handler (request=0x7fd35a454ba0, m2=0x7fd35a454b98, m1=0x7fd35a454b90, global_segment=<optimized out>) at ./storage/innobase/os/os0file.cc:2022
      #4  os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7fd35a454b90, m2=m2@entry=0x7fd35a454b98, request=request@entry=0x7fd35a454ba0) at ./storage/innobase/os/os0file.cc:5561
      #5  0x00005565b7e0052a in fil_aio_wait (segment=segment@entry=0) at ./storage/innobase/fil/fil0fil.cc:4357
      #6  0x00005565b7d12998 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a455700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 31 (Thread 0x7fd359c54700 (LWP 19021)):
      #0  0x00007fd37fc846ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005565b7c7aff1 in LinuxAIOHandler::collect (this=this@entry=0x7fd359c53b00) at ./storage/innobase/os/os0file.cc:1809
      #2  0x00005565b7c7b632 in LinuxAIOHandler::poll (this=this@entry=0x7fd359c53b00, m1=m1@entry=0x7fd359c53b90, m2=m2@entry=0x7fd359c53b98, request=request@entry=0x7fd359c53ba0) at ./storage/innobase/os/os0file.cc:1968
      #3  0x00005565b7c7f728 in os_aio_linux_handler (request=0x7fd359c53ba0, m2=0x7fd359c53b98, m1=0x7fd359c53b90, global_segment=<optimized out>) at ./storage/innobase/os/os0file.cc:2022
      #4  os_aio_handler (segment=segment@entry=1, m1=m1@entry=0x7fd359c53b90, m2=m2@entry=0x7fd359c53b98, request=request@entry=0x7fd359c53ba0) at ./storage/innobase/os/os0file.cc:5561
      #5  0x00005565b7e0052a in fil_aio_wait (segment=segment@entry=1) at ./storage/innobase/fil/fil0fil.cc:4357
      #6  0x00005565b7d12998 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd359c54700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 30 (Thread 0x7fd35a574700 (LWP 19948)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b79402b5 in inline_mysql_cond_wait (src_file=0x5565b80b20a0 "/home/buildbot/buildbot/build/mariadb-10.4.9/sql/mdl.cc", src_line=1143, mutex=0x7fd3080009e8, that=0x7fd308000a18) at ./include/mysql/psi/mysql_thread.h:1167
      #2  MDL_wait::timed_wait (this=this@entry=0x7fd3080009e8, owner=0x7fd308000998, abs_timeout=abs_timeout@entry=0x7fd35a5700a0, set_status_on_timeout=set_status_on_timeout@entry=false, wait_state_name=<optimized out>) at ./sql/mdl.cc:1143
      #3  0x00005565b7942136 in MDL_context::acquire_lock (this=this@entry=0x7fd3080009e8, mdl_request=0x7fd308011f18, lock_wait_timeout=lock_wait_timeout@entry=86400) at ./sql/mdl.cc:2326
      #4  0x00005565b79427c1 in MDL_context::acquire_locks (this=this@entry=0x7fd3080009e8, mdl_requests=mdl_requests@entry=0x7fd35a5701b0, lock_wait_timeout=lock_wait_timeout@entry=86400) at ./sql/mdl.cc:2447
      #5  0x00005565b77eeff3 in lock_table_names (thd=thd@entry=0x7fd3080008c8, options=..., tables_start=tables_start@entry=0x7fd308011ad0, tables_end=tables_end@entry=0x0, lock_wait_timeout=<optimized out>, flags=flags@entry=0) at ./sql/sql_base.cc:4085
      #6  0x00005565b79530af in lock_table_names (flags=0, lock_wait_timeout=<optimized out>, table_list_end=0x0, table_list=0x7fd308011ad0, thd=0x7fd3080008c8) at ./sql/sql_base.h:247
      #7  Sql_cmd_truncate_table::lock_table (this=this@entry=0x7fd308012188, thd=thd@entry=0x7fd3080008c8, table_ref=table_ref@entry=0x7fd308011ad0, hton_can_recreate=hton_can_recreate@entry=0x7fd35a57043f) at ./sql/sql_truncate.cc:311
      #8  0x00005565b795323a in Sql_cmd_truncate_table::truncate_table (this=this@entry=0x7fd308012188, thd=thd@entry=0x7fd3080008c8, table_ref=table_ref@entry=0x7fd308011ad0) at ./sql/sql_truncate.cc:423
      #9  0x00005565b795346e in Sql_cmd_truncate_table::execute (this=0x7fd308012188, thd=0x7fd3080008c8) at ./sql/sql_truncate.cc:505
      #10 0x00005565b78503f5 in mysql_execute_command (thd=thd@entry=0x7fd3080008c8) at ./sql/sql_parse.cc:6094
      #11 0x00005565b785719a in mysql_parse (thd=thd@entry=0x7fd3080008c8, rawbuf=rawbuf@entry=0x7fd308011a10 "TRUNCATE TABLE t1", length=length@entry=17, parser_state=parser_state@entry=0x7fd35a573000, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:7912
      #12 0x00005565b78579f9 in wsrep_mysql_parse (thd=thd@entry=0x7fd3080008c8, rawbuf=0x7fd308011a10 "TRUNCATE TABLE t1", length=17, parser_state=parser_state@entry=0x7fd35a573000, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:7716
      #13 0x00005565b785a664 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7fd3080008c8, packet=packet@entry=0x7fd3080099b9 "TRUNCATE TABLE t1", packet_length=packet_length@entry=17, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:1826
      #14 0x00005565b785affc in do_command (thd=0x7fd3080008c8) at ./sql/sql_parse.cc:1359
      #15 0x00005565b7938352 in do_handle_one_connection (connect=connect@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1412
      #16 0x00005565b793840d in handle_one_connection (arg=arg@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1316
      #17 0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba9d5ae8) at ./storage/perfschema/pfs.cc:1862
      #18 0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a574700) at pthread_create.c:456
      #19 0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 29 (Thread 0x7fd353fff700 (LWP 19024)):
      #0  0x00007fd37fc846ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005565b7c7aff1 in LinuxAIOHandler::collect (this=this@entry=0x7fd353ffeb00) at ./storage/innobase/os/os0file.cc:1809
      #2  0x00005565b7c7b632 in LinuxAIOHandler::poll (this=this@entry=0x7fd353ffeb00, m1=m1@entry=0x7fd353ffeb90, m2=m2@entry=0x7fd353ffeb98, request=request@entry=0x7fd353ffeba0) at ./storage/innobase/os/os0file.cc:1968
      #3  0x00005565b7c7f728 in os_aio_linux_handler (request=0x7fd353ffeba0, m2=0x7fd353ffeb98, m1=0x7fd353ffeb90, global_segment=<optimized out>) at ./storage/innobase/os/os0file.cc:2022
      #4  os_aio_handler (segment=segment@entry=4, m1=m1@entry=0x7fd353ffeb90, m2=m2@entry=0x7fd353ffeb98, request=request@entry=0x7fd353ffeba0) at ./storage/innobase/os/os0file.cc:5561
      #5  0x00005565b7e0052a in fil_aio_wait (segment=segment@entry=4) at ./storage/innobase/fil/fil0fil.cc:4357
      #6  0x00005565b7d12998 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd353fff700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 28 (Thread 0x7fd35a655700 (LWP 19043)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b77a6a73 in inline_mysql_cond_wait (that=<optimized out>, mutex=<optimized out>, src_file=0x5565b8096e30 "/home/buildbot/buildbot/build/mariadb-10.4.9/sql/slave.cc", src_line=519) at ./include/mysql/psi/mysql_thread.h:1167
      #2  handle_slave_background (arg=arg@entry=0x0) at ./sql/slave.cc:519
      #3  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565baa4ac58) at ./storage/perfschema/pfs.cc:1862
      #4  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a655700) at pthread_create.c:456
      #5  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 27 (Thread 0x7fd34bfff700 (LWP 19032)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7c843f2 in os_event::timed_wait (this=this@entry=0x5565ba8dd990, abstime=abstime@entry=0x7fd34bffeac0) at ./storage/innobase/os/os0event.cc:275
      #2  0x00005565b7c84631 in os_event::wait_time_low (this=0x5565ba8dd990, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=reset_sig_count@entry=1) at ./storage/innobase/os/os0event.cc:385
      #3  0x00005565b7c849da in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=reset_sig_count@entry=1) at ./storage/innobase/os/os0event.cc:485
      #4  0x00005565b7d5a27a in ib_wqueue_timedwait (wq=wq@entry=0x5565ba95eaf0, wait_in_usecs=wait_in_usecs@entry=5000000) at ./storage/innobase/ut/ut0wqueue.cc:163
      #5  0x00005565b7e39f86 in fts_optimize_thread (arg=0x5565ba95eaf0) at ./storage/innobase/fts/fts0opt.cc:2865
      #6  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd34bfff700) at pthread_create.c:456
      #7  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 26 (Thread 0x7fd3437fe700 (LWP 19040)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c84a26 in os_event::wait (this=0x5565ba77bbb0) at ./storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=1, this=0x5565ba77bbb0) at ./storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x5565ba77bbb0, reset_sig_count=reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:502
      #4  0x00005565b7d9ca0c in buf_resize_thread () at ./storage/innobase/buf/buf0buf.cc:3199
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3437fe700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 25 (Thread 0x7fd3527fc700 (LWP 19033)):
      #0  0x00007fd37fe967fd in nanosleep () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00005565b7c84c70 in os_thread_sleep (tm=tm@entry=1000000) at ./storage/innobase/os/os0thread.cc:225
      #2  0x00005565b7d0f1b1 in srv_master_sleep () at ./storage/innobase/srv/srv0srv.cc:2334
      #3  srv_master_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2375
      #4  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3527fc700) at pthread_create.c:456
      #5  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 24 (Thread 0x7fd342ffd700 (LWP 19041)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c04053 in inline_mysql_cond_wait (src_file=0x5565b81bdd08 "/home/buildbot/buildbot/build/mariadb-10.4.9/storage/innobase/handler/ha_innodb.cc", src_line=288, mutex=0x7fd342ffccc0, that=0x7fd342ffccf0) at ./include/mysql/psi/mysql_thread.h:1178
      #2  thd_destructor_proxy () at ./storage/innobase/handler/ha_innodb.cc:288
      #3  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba8f35a8) at ./storage/perfschema/pfs.cc:1862
      #4  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd342ffd700) at pthread_create.c:456
      #5  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 23 (Thread 0x7fd3497fa700 (LWP 19038)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c84a26 in os_event::wait (this=0x5565ba77bad0) at ./storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=1, this=0x5565ba77bad0) at ./storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x5565ba77bad0, reset_sig_count=reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:502
      #4  0x00005565b7da4157 in buf_dump_thread () at ./storage/innobase/buf/buf0dump.cc:833
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3497fa700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 22 (Thread 0x7fd3715fa700 (LWP 19015)):
      #0  0x00007fd37e5f3303 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00007fd37853b57f in asio::detail::epoll_reactor::run (this=0x5565ba6823e0, block=block@entry=true, ops=...) at /usr/include/asio/detail/impl/epoll_reactor.ipp:391
      #2  0x00007fd3785560ec in asio::detail::task_io_service::do_run_one (ec=..., this_thread=..., lock=..., this=0x5565ba682280) at /usr/include/asio/detail/impl/task_io_service.ipp:355
      #3  asio::detail::task_io_service::run (ec=..., this=0x5565ba682280) at /usr/include/asio/detail/impl/task_io_service.ipp:148
      #4  asio::io_service::run (this=<optimized out>) at /usr/include/asio/impl/io_service.ipp:58
      #5  gcomm::AsioProtonet::event_loop (this=<optimized out>, period=...) at gcomm/src/asio_protonet.cpp:116
      #6  0x00007fd378584868 in GCommConn::run (this=0x5565ba680f50) at gcs/src/gcs_gcomm.cpp:498
      #7  0x00007fd37858cc19 in GCommConn::run_fn (arg=<optimized out>) at gcs/src/gcs_gcomm.cpp:209
      #8  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3715fa700) at pthread_create.c:456
      #9  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 21 (Thread 0x7fd349ffb700 (LWP 19037)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c84a26 in os_event::wait (this=0x5565ba77b980) at ./storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=44146, this=0x5565ba77b980) at ./storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x5565ba77b980, reset_sig_count=reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:502
      #4  0x00005565b7d0fc5f in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x5565b8961b28 <srv_sys+360>) at ./storage/innobase/srv/srv0srv.cc:909
      #5  srv_worker_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2517
      #6  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd349ffb700) at pthread_create.c:456
      #7  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 20 (Thread 0x7fd348ff9700 (LWP 19039)):
      #0  0x00007fd37fe967fd in nanosleep () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00005565b7c84c70 in os_thread_sleep (tm=tm@entry=1000000) at ./storage/innobase/os/os0thread.cc:225
      #2  0x00005565b7d8e8e0 in btr_defragment_thread () at ./storage/innobase/btr/btr0defragment.cc:717
      #3  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd348ff9700) at pthread_create.c:456
      #4  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 19 (Thread 0x7fd34a7fc700 (LWP 19036)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c84a26 in os_event::wait (this=0x5565ba77b910) at ./storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=44094, this=0x5565ba77b910) at ./storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x5565ba77b910, reset_sig_count=reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:502
      #4  0x00005565b7d0fc5f in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x5565b8961af0 <srv_sys+304>) at ./storage/innobase/srv/srv0srv.cc:909
      #5  srv_worker_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2517
      #6  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd34a7fc700) at pthread_create.c:456
      #7  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 18 (Thread 0x7fd370df9700 (LWP 19016)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00007fd37858207b in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #2  RecvBuf::front (timeout=..., this=0x5565ba681188) at gcs/src/gcs_gcomm.cpp:124
      #3  gcomm_recv (backend=<optimized out>, msg=0x5565ba6424c0, timeout=9223372035999999999) at gcs/src/gcs_gcomm.cpp:673
      #4  0x00007fd3785712ff in core_msg_recv (timeout=9223372035999999999, recv_msg=0x5565ba6424c0, backend=0x5565ba642620) at gcs/src/gcs_core.cpp:480
      #5  gcs_core_recv (conn=0x5565ba642460, recv_act=recv_act@entry=0x7fd370df8c80, timeout=9223372035999999999) at gcs/src/gcs_core.cpp:1184
      #6  0x00007fd37857e47f in gcs_recv_thread (arg=0x5565ba6421e0) at gcs/src/gcs.cpp:1398
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd370df9700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 17 (Thread 0x7fd378053700 (LWP 19042)):
      #0  0x00007fd37e53df26 in __GI___sigwaitinfo (set=<optimized out>, set@entry=0x7fd378052c10, info=info@entry=0x7fd378052c90) at ../sysdeps/unix/sysv/linux/sigwaitinfo.c:56
      #1  0x00005565b7788923 in my_sigwait (code=<synthetic pointer>, sig=0x7fd378052c08, set=0x7fd378052c10) at ./include/my_pthread.h:198
      #2  signal_hand (arg=arg@entry=0x0) at ./sql/mysqld.cc:3243
      #3  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba977858) at ./storage/perfschema/pfs.cc:1862
      #4  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd378053700) at pthread_create.c:456
      #5  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 16 (Thread 0x7fd34affd700 (LWP 19035)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c84a26 in os_event::wait (this=0x5565ba77b8a0) at ./storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=52656, this=0x5565ba77b8a0) at ./storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x5565ba77b8a0, reset_sig_count=reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:502
      #4  0x00005565b7d0fc5f in srv_resume_thread (sig_count=0, timeout_usec=0, wait=true, slot=0x5565b8961ab8 <srv_sys+248>) at ./storage/innobase/srv/srv0srv.cc:909
      #5  srv_worker_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2517
      #6  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd34affd700) at pthread_create.c:456
      #7  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 15 (Thread 0x7fd3537fe700 (LWP 19025)):
      #0  0x00007fd37fc846ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005565b7c7aff1 in LinuxAIOHandler::collect (this=this@entry=0x7fd3537fdb00) at ./storage/innobase/os/os0file.cc:1809
      #2  0x00005565b7c7b632 in LinuxAIOHandler::poll (this=this@entry=0x7fd3537fdb00, m1=m1@entry=0x7fd3537fdb90, m2=m2@entry=0x7fd3537fdb98, request=request@entry=0x7fd3537fdba0) at ./storage/innobase/os/os0file.cc:1968
      #3  0x00005565b7c7f728 in os_aio_linux_handler (request=0x7fd3537fdba0, m2=0x7fd3537fdb98, m1=0x7fd3537fdb90, global_segment=<optimized out>) at ./storage/innobase/os/os0file.cc:2022
      #4  os_aio_handler (segment=segment@entry=5, m1=m1@entry=0x7fd3537fdb90, m2=m2@entry=0x7fd3537fdb98, request=request@entry=0x7fd3537fdba0) at ./storage/innobase/os/os0file.cc:5561
      #5  0x00005565b7e0052a in fil_aio_wait (segment=segment@entry=5) at ./storage/innobase/fil/fil0fil.cc:4357
      #6  0x00005565b7d12998 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3537fe700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 14 (Thread 0x7fd35a60a700 (LWP 19046)):
      #0  0x00007fd37e5e98bd in poll () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00005565b803fffa in poll (__timeout=28800000, __nfds=1, __fds=0x7fd35a609548) at /usr/include/x86_64-linux-gnu/bits/poll2.h:46
      #2  vio_io_wait (vio=vio@entry=0x5565ba9d5ae8, event=<optimized out>, timeout=28800000) at ./vio/viosocket.c:980
      #3  0x00005565b80400f0 in vio_socket_io_wait (vio=0x5565ba9d5ae8, event=<optimized out>) at ./vio/viosocket.c:118
      #4  0x00005565b80401dd in vio_read (vio=0x5565ba9d5ae8, buf=0x7fd30c007998 "\001", size=4) at ./vio/viosocket.c:199
      #5  0x00005565b7b71f42 in my_real_read (net=0x7fd30c000c18, complen=complen@entry=0x7fd35a6097f8, header=header@entry=1 '\001') at ./sql/net_serv.cc:962
      #6  0x00005565b7b73178 in my_net_read_packet_reallen (net=net@entry=0x7fd30c000c18, read_from_server=read_from_server@entry=1 '\001', reallen=reallen@entry=0x7fd35a609848) at ./sql/net_serv.cc:1243
      #7  0x00005565b7b732dd in my_net_read_packet (net=net@entry=0x7fd30c000c18, read_from_server=read_from_server@entry=1 '\001') at ./sql/net_serv.cc:1227
      #8  0x00005565b785af70 in do_command (thd=0x7fd30c0009a8) at ./sql/sql_parse.cc:1232
      #9  0x00005565b7938352 in do_handle_one_connection (connect=connect@entry=0x5565baa2b8e8) at ./sql/sql_connect.cc:1412
      #10 0x00005565b793840d in handle_one_connection (arg=arg@entry=0x5565baa2b8e8) at ./sql/sql_connect.cc:1316
      #11 0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565baa54c58) at ./storage/perfschema/pfs.cc:1862
      #12 0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a60a700) at pthread_create.c:456
      #13 0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 13 (Thread 0x7fd358c52700 (LWP 19023)):
      #0  0x00007fd37fc846ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005565b7c7aff1 in LinuxAIOHandler::collect (this=this@entry=0x7fd358c51b00) at ./storage/innobase/os/os0file.cc:1809
      #2  0x00005565b7c7b632 in LinuxAIOHandler::poll (this=this@entry=0x7fd358c51b00, m1=m1@entry=0x7fd358c51b90, m2=m2@entry=0x7fd358c51b98, request=request@entry=0x7fd358c51ba0) at ./storage/innobase/os/os0file.cc:1968
      #3  0x00005565b7c7f728 in os_aio_linux_handler (request=0x7fd358c51ba0, m2=0x7fd358c51b98, m1=0x7fd358c51b90, global_segment=<optimized out>) at ./storage/innobase/os/os0file.cc:2022
      #4  os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7fd358c51b90, m2=m2@entry=0x7fd358c51b98, request=request@entry=0x7fd358c51ba0) at ./storage/innobase/os/os0file.cc:5561
      #5  0x00005565b7e0052a in fil_aio_wait (segment=segment@entry=3) at ./storage/innobase/fil/fil0fil.cc:4357
      #6  0x00005565b7d12998 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd358c52700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 12 (Thread 0x7fd352ffd700 (LWP 19026)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7c843f2 in os_event::timed_wait (this=this@entry=0x5565ba77bb40, abstime=abstime@entry=0x7fd352ffc980) at ./storage/innobase/os/os0event.cc:275
      #2  0x00005565b7c84631 in os_event::wait_time_low (this=0x5565ba77bb40, time_in_usec=time_in_usec@entry=997000, reset_sig_count=reset_sig_count@entry=2) at ./storage/innobase/os/os0event.cc:385
      #3  0x00005565b7c849da in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=997000, reset_sig_count=reset_sig_count@entry=2) at ./storage/innobase/os/os0event.cc:485
      #4  0x00005565b7dae946 in pc_sleep_if_needed (cur_time=987607, sig_count=2, next_loop_time=988604) at ./storage/innobase/buf/buf0flu.cc:2643
      #5  buf_flush_page_cleaner_coordinator () at ./storage/innobase/buf/buf0flu.cc:3102
      #6  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd352ffd700) at pthread_create.c:456
      #7  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 11 (Thread 0x7fd359453700 (LWP 19022)):
      #0  0x00007fd37fc846ca in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005565b7c7aff1 in LinuxAIOHandler::collect (this=this@entry=0x7fd359452b00) at ./storage/innobase/os/os0file.cc:1809
      #2  0x00005565b7c7b632 in LinuxAIOHandler::poll (this=this@entry=0x7fd359452b00, m1=m1@entry=0x7fd359452b90, m2=m2@entry=0x7fd359452b98, request=request@entry=0x7fd359452ba0) at ./storage/innobase/os/os0file.cc:1968
      #3  0x00005565b7c7f728 in os_aio_linux_handler (request=0x7fd359452ba0, m2=0x7fd359452b98, m1=0x7fd359452b90, global_segment=<optimized out>) at ./storage/innobase/os/os0file.cc:2022
      #4  os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7fd359452b90, m2=m2@entry=0x7fd359452b98, request=request@entry=0x7fd359452ba0) at ./storage/innobase/os/os0file.cc:5561
      #5  0x00005565b7e0052a in fil_aio_wait (segment=segment@entry=2) at ./storage/innobase/fil/fil0fil.cc:4357
      #6  0x00005565b7d12998 in io_handler_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0start.cc:324
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd359453700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 10 (Thread 0x7fd351ffb700 (LWP 19028)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7c843f2 in os_event::timed_wait (this=this@entry=0x5565ba914c80, abstime=abstime@entry=0x7fd351ffac00) at ./storage/innobase/os/os0event.cc:275
      #2  0x00005565b7c84631 in os_event::wait_time_low (this=this@entry=0x5565ba914c80, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
      #3  0x00005565b7c849da in os_event_wait_time_low (event=event@entry=0x5565ba914c80, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=<optimized out>) at ./storage/innobase/os/os0event.cc:485
      #4  0x00005565b7c59ef0 in lock_wait_timeout_thread () at ./storage/innobase/lock/lock0wait.cc:522
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd351ffb700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 9 (Thread 0x7fd35a529700 (LWP 20027)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b778a3a4 in inline_mysql_cond_timedwait (src_file=0x5565b808cca8 "/home/buildbot/buildbot/build/mariadb-10.4.9/sql/mysqld.cc", src_line=2686, abstime=0x7fd35a528c30, mutex=0x5565b89e3980 <LOCK_thread_cache>, that=0x5565b89e6200 <COND_thread_cache>) at ./include/mysql/psi/mysql_thread.h:1204
      #2  cache_thread (thd=0x7fd3000009a8) at ./sql/mysqld.cc:2685
      #3  one_thread_per_connection_end (thd=0x7fd3000009a8, put_in_cache=<optimized out>) at ./sql/mysqld.cc:2771
      #4  0x00005565b79382e3 in do_handle_one_connection (connect=connect@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1423
      #5  0x00005565b793840d in handle_one_connection (arg=arg@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1316
      #6  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565baa1d158) at ./storage/perfschema/pfs.cc:1862
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a529700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 8 (Thread 0x7fd378289700 (LWP 19018)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00007fd3785fbcac in gu::Lock::wait (this=<synthetic pointer>, cond=...) at galerautils/src/gu_lock.hpp:45
      #2  galera::Monitor<galera::ReplicatorSMM::ApplyOrder>::enter (this=this@entry=0x5565ba639ce0, obj=...) at galera/src/monitor.hpp:156
      #3  0x00007fd3785ef085 in galera::ReplicatorSMM::apply_trx (this=this@entry=0x5565ba638d80, recv_ctx=recv_ctx@entry=0x7fd3782883f0, ts=...) at galera/src/replicator_smm.cpp:477
      #4  0x00007fd3785f6b2d in galera::ReplicatorSMM::process_trx (this=0x5565ba638d80, recv_ctx=0x7fd3782883f0, ts_ptr=...) at galera/src/replicator_smm.cpp:2136
      #5  0x00007fd3785c59a4 in galera::GcsActionSource::process_writeset (this=0x5565ba641d60, recv_ctx=<optimized out>, act=..., exit_loop=@0x7fd378287fff: false) at galera/src/gcs_action_source.cpp:62
      #6  0x00007fd3785c67ef in galera::GcsActionSource::dispatch (this=this@entry=0x5565ba641d60, recv_ctx=recv_ctx@entry=0x7fd3782883f0, act=..., exit_loop=@0x7fd378287fff: false) at galera/src/gcs_action_source.cpp:109
      #7  0x00007fd3785c6d45 in galera::GcsActionSource::process (this=0x5565ba641d60, recv_ctx=0x7fd3782883f0, exit_loop=@0x7fd378287fff: false) at galera/src/gcs_action_source.cpp:182
      #8  0x00007fd3785f18d3 in galera::ReplicatorSMM::async_recv (this=0x5565ba638d80, recv_ctx=0x7fd3782883f0) at galera/src/replicator_smm.cpp:391
      #9  0x00007fd378611e6b in galera_recv (gh=<optimized out>, recv_ctx=<optimized out>) at galera/src/wsrep_provider.cpp:236
      #10 0x00005565b80812ae in wsrep::wsrep_provider_v26::run_applier (this=<optimized out>, applier_ctx=<optimized out>) at ./wsrep-lib/src/wsrep_provider_v26.cpp:690
      #11 0x00005565b79da572 in wsrep_replication_process (thd=0x7fd35c0009a8, arg=<optimized out>) at ./sql/wsrep_thd.cc:62
      #12 0x00005565b79c85fe in start_wsrep_THD (arg=arg@entry=0x5565ba67ede0) at ./sql/wsrep_mysqld.cc:2708
      #13 0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba65a628) at ./storage/perfschema/pfs.cc:1862
      #14 0x00007fd37fe8d4a4 in start_thread (arg=0x7fd378289700) at pthread_create.c:456
      #15 0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 7 (Thread 0x7fd3517fa700 (LWP 19029)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7c843f2 in os_event::timed_wait (this=this@entry=0x5565ba77b9f0, abstime=abstime@entry=0x7fd3517f9b60) at ./storage/innobase/os/os0event.cc:275
      #2  0x00005565b7c84631 in os_event::wait_time_low (this=0x5565ba77b9f0, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
      #3  0x00005565b7c849da in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=1000000, reset_sig_count=<optimized out>) at ./storage/innobase/os/os0event.cc:485
      #4  0x00005565b7d0e10c in srv_error_monitor_thread () at ./storage/innobase/srv/srv0srv.cc:1851
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3517fa700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 6 (Thread 0x7fd3782d4700 (LWP 19017)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b79d8f73 in inline_mysql_cond_wait (src_line=69, src_file=0x5565b80caa58 "/home/buildbot/buildbot/build/mariadb-10.4.9/sql/wsrep_thd.h", mutex=0x7fd368012a68, that=0x7fd368012a98) at ./include/mysql/psi/mysql_thread.h:1178
      #2  Wsrep_thd_queue::pop_front (this=0x7fd368012a10) at ./sql/wsrep_thd.h:69
      #3  wsrep_rollback_process (rollbacker=0x7fd3680009a8, arg=<optimized out>) at ./sql/wsrep_thd.cc:266
      #4  0x00005565b79c85fe in start_wsrep_THD (arg=arg@entry=0x5565ba67ffe0) at ./sql/wsrep_mysqld.cc:2708
      #5  0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba699738) at ./storage/perfschema/pfs.cc:1862
      #6  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd3782d4700) at pthread_create.c:456
      #7  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 5 (Thread 0x7fd350ff9700 (LWP 19030)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7c843f2 in os_event::timed_wait (this=this@entry=0x5565ba77ba60, abstime=abstime@entry=0x7fd350ff8c90) at ./storage/innobase/os/os0event.cc:275
      #2  0x00005565b7c84631 in os_event::wait_time_low (this=0x5565ba77ba60, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=1) at ./storage/innobase/os/os0event.cc:385
      #3  0x00005565b7c849da in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=5000000, reset_sig_count=<optimized out>) at ./storage/innobase/os/os0event.cc:485
      #4  0x00005565b7d0da85 in srv_monitor_thread () at ./storage/innobase/srv/srv0srv.cc:1687
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd350ff9700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 4 (Thread 0x7fd38022d4c0 (LWP 19012)):
      #0  0x00007fd37e5e98bd in poll () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00007fd37e6033be in __poll_chk (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, fdslen=<optimized out>) at poll_chk.c:27
      #2  0x00005565b77921df in poll (__timeout=-1, __nfds=2, __fds=0x7ffc9773c2d0) at /usr/include/x86_64-linux-gnu/bits/poll2.h:41
      #3  handle_connections_sockets () at ./sql/mysqld.cc:6455
      #4  0x00005565b779373f in mysqld_main (argc=<optimized out>, argv=<optimized out>) at ./sql/mysqld.cc:5891
      #5  0x00007fd37e52a2e1 in __libc_start_main (main=0x5565b776cf50 <main(int, char**)>, argc=23, argv=0x7ffc9773ca08, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc9773c9f8) at ../csu/libc-start.c:291
      #6  0x00005565b778653a in _start ()
       
      Thread 3 (Thread 0x7fd343fff700 (LWP 19031)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005565b7c843f2 in os_event::timed_wait (this=this@entry=0x5565ba946110, abstime=abstime@entry=0x7fd343ffebe0) at ./storage/innobase/os/os0event.cc:275
      #2  0x00005565b7c84631 in os_event::wait_time_low (this=0x5565ba946110, time_in_usec=time_in_usec@entry=10000000, reset_sig_count=1, reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:385
      #3  0x00005565b7c849da in os_event_wait_time_low (event=<optimized out>, time_in_usec=time_in_usec@entry=10000000, reset_sig_count=reset_sig_count@entry=0) at ./storage/innobase/os/os0event.cc:485
      #4  0x00005565b7df3ad5 in dict_stats_thread () at ./storage/innobase/dict/dict0stats_bg.cc:476
      #5  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd343fff700) at pthread_create.c:456
      #6  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 2 (Thread 0x7fd34b7fe700 (LWP 19034)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005565b7c84a26 in os_event::wait (this=0x5565ba77b830) at ./storage/innobase/os/os0event.cc:158
      #2  os_event::wait_low (reset_sig_count=12078, this=0x5565ba77b830) at ./storage/innobase/os/os0event.cc:325
      #3  os_event_wait_low (event=0x5565ba77b830, reset_sig_count=reset_sig_count@entry=12078) at ./storage/innobase/os/os0event.cc:502
      #4  0x00005565b7d11e45 in srv_resume_thread (timeout_usec=0, wait=true, sig_count=12078, slot=0x5565b8961a80 <srv_sys+192>) at ./storage/innobase/srv/srv0srv.cc:909
      #5  srv_purge_coordinator_suspend (rseg_history_len=<optimized out>, slot=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2657
      #6  srv_purge_coordinator_thread (arg=<optimized out>) at ./storage/innobase/srv/srv0srv.cc:2738
      #7  0x00007fd37fe8d4a4 in start_thread (arg=0x7fd34b7fe700) at pthread_create.c:456
      #8  0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Thread 1 (Thread 0x7fd35a5bf700 (LWP 19947)):
      #0  __pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
      #1  0x00005565b7fd9677 in my_write_core (sig=sig@entry=6) at ./mysys/stacktrace.c:481
      #2  0x00005565b7a669db in handle_fatal_signal (sig=6) at ./sql/signal_handler.cc:343
      #3  <signal handler called>
      #4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
      #5  0x00007fd37e53e42a in __GI_abort () at abort.c:89
      #6  0x00007fd37e535e67 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x5565b80c2962 "0", file=file@entry=0x5565b84ba4d0 "/home/buildbot/buildbot/build/mariadb-10.4.9/wsrep-lib/src/transaction.cpp", line=line@entry=1034, function=function@entry=0x5565b84b9d00 <wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)::__PRETTY_FUNCTION__> "bool wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)") at assert.c:92
      #7  0x00007fd37e535f12 in __GI___assert_fail (assertion=assertion@entry=0x5565b80c2962 "0", file=file@entry=0x5565b84ba4d0 "/home/buildbot/buildbot/build/mariadb-10.4.9/wsrep-lib/src/transaction.cpp", line=line@entry=1034, function=function@entry=0x5565b84b9d00 <wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)::__PRETTY_FUNCTION__> "bool wsrep::transaction::abort_or_interrupt(wsrep::unique_lock<wsrep::mutex>&)") at assert.c:101
      #8  0x00005565b807aba9 in wsrep::transaction::abort_or_interrupt (this=this@entry=0x7fd304006950, lock=...) at ./wsrep-lib/src/transaction.cpp:1034
      #9  0x00005565b807ac63 in wsrep::transaction::certify_fragment (this=this@entry=0x7fd304006950, lock=...) at ./wsrep-lib/src/transaction.cpp:1099
      #10 0x00005565b807bc20 in wsrep::transaction::streaming_step (this=this@entry=0x7fd304006950, lock=...) at ./wsrep-lib/src/transaction.cpp:1083
      #11 0x00005565b807bd4d in wsrep::transaction::after_row (this=this@entry=0x7fd304006950) at ./wsrep-lib/src/transaction.cpp:254
      #12 0x00005565b7a68c30 in wsrep::client_state::after_row (this=0x7fd3040068e8) at ./wsrep-lib/include/wsrep/client_state.hpp:339
      #13 wsrep_after_row (thd=0x7fd3040009a8) at ./sql/wsrep_trans_observer.h:170
      #14 wsrep_after_row (thd=0x7fd3040009a8) at ./sql/handler.cc:6485
      #15 0x00005565b7a721cf in handler::ha_write_row (this=0x7fd30401f6c0, buf=0x7fd30403e1c8 "\375\001") at ./sql/handler.cc:6684
      #16 0x00005565b781bdcd in write_record (thd=thd@entry=0x7fd3040009a8, table=table@entry=0x7fd3040198a8, info=info@entry=0x7fd35a5bb250) at ./sql/sql_insert.cc:2036
      #17 0x00005565b782384e in mysql_insert (thd=thd@entry=0x7fd3040009a8, table_list=<optimized out>, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at ./sql/sql_insert.cc:1076
      #18 0x00005565b78526fc in mysql_execute_command (thd=thd@entry=0x7fd3040009a8) at ./sql/sql_parse.cc:4524
      #19 0x00005565b785719a in mysql_parse (thd=thd@entry=0x7fd3040009a8, rawbuf=rawbuf@entry=0x7fd30400fcd0 "INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')", length=length@entry=44, parser_state=parser_state@entry=0x7fd35a5be000, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:7912
      #20 0x00005565b78579f9 in wsrep_mysql_parse (thd=thd@entry=0x7fd3040009a8, rawbuf=0x7fd30400fcd0 "INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')", length=44, parser_state=parser_state@entry=0x7fd35a5be000, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:7716
      #21 0x00005565b785a664 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x7fd3040009a8, packet=packet@entry=0x7fd304007999 "INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')", packet_length=packet_length@entry=44, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at ./sql/sql_parse.cc:1826
      #22 0x00005565b785affc in do_command (thd=0x7fd3040009a8) at ./sql/sql_parse.cc:1359
      #23 0x00005565b7938352 in do_handle_one_connection (connect=connect@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1412
      #24 0x00005565b793840d in handle_one_connection (arg=arg@entry=0x5565ba99fff8) at ./sql/sql_connect.cc:1316
      #25 0x00005565b7f82fd1 in pfs_spawn_thread (arg=0x5565ba9e32f8) at ./storage/perfschema/pfs.cc:1862
      #26 0x00007fd37fe8d4a4 in start_thread (arg=0x7fd35a5bf700) at pthread_create.c:456
      #27 0x00007fd37e5f2d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
       
      Compressed file /dev/shm/var/1/log/galera_sr.GCF-1060-innodb/mysqld.1/data/core
       - skipping '/dev/shm/var/1/log/galera_sr.GCF-1060-innodb/'
      

      Attachments

        Issue Links

          Activity

            Reopening since it is failed again.

            stdout:

            ES 10.4.11-5, e844a75, ubuntu-1604

            2019-12-30T19:34:36.8851602Z galera_sr.GCF-1060 'innodb'              w1 [ fail ]
            2019-12-30T19:34:36.8853064Z         Test ended at 2019-12-30 19:34:36
            2019-12-30T19:34:36.8853560Z 
            2019-12-30T19:34:36.8854160Z CURRENT_TEST: galera_sr.GCF-1060
            2019-12-30T19:34:36.8854619Z mysqltest: In included file "./suite/galera/include/galera_concurrent_test.inc": 
            2019-12-30T19:34:36.8855313Z included from /__w/1/s/mysql-test/suite/galera_sr/t/GCF-1060.test at line 10:
            2019-12-30T19:34:36.8856100Z At line 76: query 'reap' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
            2019-12-30T19:34:36.8858257Z 
            2019-12-30T19:34:36.8858662Z The result from queries just before the failure was:
            2019-12-30T19:34:36.8858952Z < snip >
            2019-12-30T19:34:36.8859205Z connect node_1a, 127.0.0.1, root, , test, $NODE_MYPORT_1;
            2019-12-30T19:34:36.8859459Z Running a concurrent test with the following queries:
            2019-12-30T19:34:36.8859715Z TRUNCATE TABLE t1
            2019-12-30T19:34:36.8860279Z INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')
            2019-12-30T19:34:36.8861260Z INSERT INTO t1 VALUE (4, 'z');
            2019-12-30T19:34:36.8861717Z connection node_1;
            2019-12-30T19:34:36.8861976Z DROP TABLE IF EXISTS t1;
            2019-12-30T19:34:36.8862239Z CREATE TABLE t1 (f1 INTEGER PRIMARY KEY AUTO_INCREMENT, f2 CHAR(255)) ENGINE=InnoDB;
            2019-12-30T19:34:36.8862510Z SET SESSION wsrep_sync_wait = 0;
            2019-12-30T19:34:36.8862756Z SET SESSION wsrep_trx_fragment_size = 1;;;
            2019-12-30T19:34:36.8862990Z connection node_1a;
            2019-12-30T19:34:36.8863238Z SET SESSION wsrep_sync_wait = 0;
            2019-12-30T19:34:36.8863712Z SET SESSION wsrep_trx_fragment_size = 1;;;
            2019-12-30T19:34:36.8864047Z connection node_2;
            2019-12-30T19:34:36.8864244Z SET SESSION wsrep_sync_wait = 0;
            2019-12-30T19:34:36.8864457Z SET SESSION wsrep_trx_fragment_size = 1;;;
            2019-12-30T19:34:36.8864648Z 
            2019-12-30T19:34:36.8865131Z More results from queries before failure can be found in /var/tmp/mtr/1/log/GCF-1060.log
            2019-12-30T19:34:36.8865415Z 
            2019-12-30T19:34:36.8865619Z 
            2019-12-30T19:34:36.8866327Z Server [mysqld.1 - pid: 46958, winpid: 46958, exit: 256] failed during test run
            2019-12-30T19:34:36.8866695Z Server log from this test:
            2019-12-30T19:34:36.8867601Z ----------SERVER LOG START-----------
            2019-12-30T19:34:36.8868310Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8868659Z schema:  test
            2019-12-30T19:34:36.8868923Z request: (24906 	seqno 27114 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8869485Z granted: (24925 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8870110Z 2019-12-30 19:34:35 24925 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi
            2019-12-30T19:34:36.8870789Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8871208Z schema:  test
            2019-12-30T19:34:36.8871443Z request: (24906 	seqno 27179 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8871940Z granted: (24965 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8872502Z 2019-12-30 19:34:35 24965 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi
            2019-12-30T19:34:36.8873014Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8873307Z schema:  test
            2019-12-30T19:34:36.8873539Z request: (24906 	seqno 27239 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8874026Z granted: (25001 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8874579Z 2019-12-30 19:34:35 25001 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi
            2019-12-30T19:34:36.8875086Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8875380Z schema:  test
            2019-12-30T19:34:36.8875613Z request: (24906 	seqno 27293 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8876100Z granted: (25032 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8876885Z 2019-12-30 19:34:35 25032 [Note] WSREP: rollbacker aborting SR thd: (25032 139902393857792)
            2019-12-30T19:34:36.8877862Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8878287Z schema:  test
            2019-12-30T19:34:36.8878561Z request: (24906 	seqno 27319 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8879153Z granted: (25049 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8879791Z 2019-12-30 19:34:35 25049 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi
            2019-12-30T19:34:36.8880575Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8895481Z schema:  test
            2019-12-30T19:34:36.8895943Z request: (24906 	seqno 27358 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8896606Z granted: (25071 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8897301Z 2019-12-30 19:34:35 25071 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi
            2019-12-30T19:34:36.8898548Z 2019-12-30 19:34:36 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8898904Z schema:  test
            2019-12-30T19:34:36.8899163Z request: (24906 	seqno 27561 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8899767Z granted: (25183 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8900397Z 2019-12-30 19:34:36 25183 [Note] WSREP: rollbacker aborting SR thd: (25183 139902393857792)
            2019-12-30T19:34:36.8902006Z 2019-12-30 19:34:36 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8902621Z schema:  test
            2019-12-30T19:34:36.8902942Z request: (24906 	seqno 27614 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8903499Z granted: (25214 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8904076Z 2019-12-30 19:34:36 25214 [Note] WSREP: rollbacker aborting SR thd: (25214 139902393857792)
            2019-12-30T19:34:36.8904635Z 2019-12-30 19:34:36 24906 [Note] WSREP: MDL conflict, DDL vs SR
            2019-12-30T19:34:36.8904922Z schema:  test
            2019-12-30T19:34:36.8905141Z request: (24906 	seqno 27654 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            2019-12-30T19:34:36.8905630Z granted: (25238 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2019-12-30T19:34:36.8912288Z 2019-12-30 19:34:36 25238 [Note] WSREP: rollbacker aborting SR thd: (25238 139902393857792)
            2019-12-30T19:34:36.8913101Z 2019-12-30 19:34:36 0x7f3d907d3700  InnoDB: Assertion failure in file /__w/1/s/BUILD/storage/innobase/trx/trx0roll.cc line 294
            2019-12-30T19:34:36.8913451Z InnoDB: We intentionally generate a memory trap.
            2019-12-30T19:34:36.8913708Z InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
            2019-12-30T19:34:36.8913940Z InnoDB: If you get repeated assertion failures or crashes, even
            2019-12-30T19:34:36.8914182Z InnoDB: immediately after the mysqld startup, there may be
            2019-12-30T19:34:36.8914415Z InnoDB: corruption in the InnoDB tablespace. Please refer to
            2019-12-30T19:34:36.8914888Z InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
            2019-12-30T19:34:36.8915169Z InnoDB: about forcing recovery.
            2019-12-30T19:34:36.8915384Z 191230 19:34:36 [ERROR] mysqld got signal 6 ;
            2019-12-30T19:34:36.8915615Z This could be because you hit a bug. It is also possible that this binary
            2019-12-30T19:34:36.8915859Z or one of the libraries it was linked against is corrupt, improperly built,
            2019-12-30T19:34:36.8916101Z or misconfigured. This error can also be caused by malfunctioning hardware.
            2019-12-30T19:34:36.8916302Z 
            2019-12-30T19:34:36.8916733Z To report this bug, see https://mariadb.com/kb/en/reporting-bugs
            2019-12-30T19:34:36.8916981Z 
            2019-12-30T19:34:36.8917190Z We will try our best to scrape up some info that will hopefully help
            2019-12-30T19:34:36.8917874Z diagnose the problem, but since we have already crashed, 
            2019-12-30T19:34:36.8918506Z something is definitely wrong and this may fail.
            2019-12-30T19:34:36.8918955Z 
            2019-12-30T19:34:36.8919507Z Server version: 10.4.11-5-MariaDB-enterprise-log
            2019-12-30T19:34:36.8919844Z key_buffer_size=1048576
            2019-12-30T19:34:36.8920084Z read_buffer_size=131072
            2019-12-30T19:34:36.8920309Z max_used_connections=4
            2019-12-30T19:34:36.8920552Z max_threads=153
            2019-12-30T19:34:36.8920903Z thread_count=13
            2019-12-30T19:34:36.8921225Z It is possible that mysqld could use up to 
            2019-12-30T19:34:36.8921461Z key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63529 K  bytes of memory
            2019-12-30T19:34:36.8922081Z Hope that's ok; if not, decrease some variables in the equation.
            2019-12-30T19:34:36.8922338Z 
            2019-12-30T19:34:36.8922539Z Thread pointer: 0x7f3d80048398
            2019-12-30T19:34:36.8922759Z Attempting backtrace. You can use the following information to find out
            2019-12-30T19:34:36.8922997Z where mysqld died. If you see no messages after this, something went
            2019-12-30T19:34:36.8923222Z terribly wrong...
            2019-12-30T19:34:36.8923431Z stack_bottom = 0x7f3d907888e8 thread_stack 0x49000
            2019-12-30T19:34:36.8924032Z mysqld: /__w/1/s/BUILD/wsrep-lib/include/wsrep/client_state.hpp:498: int wsrep::client_state::after_rollback(): Assertion `owning_thread_id_ == wsrep::this_thread::get_id()' failed.
            2019-12-30T19:34:36.8924597Z ----------SERVER LOG END-------------
            2019-12-30T19:34:36.8924834Z 
            2019-12-30T19:34:36.8925006Z 
            2019-12-30T19:34:36.8926150Z  - saving '/var/tmp/mtr/1/log/galera_sr.GCF-1060-innodb/' to '/var/tmp/mtr/log/galera_sr.GCF-1060-innodb/'
            2019-12-30T19:34:36.8937124Z ***Warnings generated in error logs during shutdown after running tests: galera_sr.mysql-wsrep-features#8 galera_sr.GCF-1060 galera_sr.GCF-1051 galera_sr.GCF-1008 galera_sr.GCF-1018 galera_sr.galera_sr_load_data_splitting galera_sr.GCF-1043B galera_sr.galera_sr_many_fragments galera_sr.galera_sr_kill_all_pcrecovery galera_sr.mysql-wsrep-features#9
            2019-12-30T19:34:36.8938511Z 
            2019-12-30T19:34:36.8938884Z Attempting backtrace. You can use the following information to find out
            2019-12-30T19:34:36.8939713Z mysqld: /__w/1/s/BUILD/wsrep-lib/include/wsrep/client_state.hpp:498: int wsrep::client_state::after_rollback(): Assertion `owning_thread_id_ == wsrep::this_thread::get_id()' failed.
            2019-12-30T19:34:36.8940107Z 
            

            stepan.patryshev Stepan Patryshev (Inactive) added a comment - - edited Reopening since it is failed again. stdout : ES 10.4.11-5, e844a75, ubuntu-1604 2019-12-30T19:34:36.8851602Z galera_sr.GCF-1060 'innodb' w1 [ fail ] 2019-12-30T19:34:36.8853064Z Test ended at 2019-12-30 19:34:36 2019-12-30T19:34:36.8853560Z 2019-12-30T19:34:36.8854160Z CURRENT_TEST: galera_sr.GCF-1060 2019-12-30T19:34:36.8854619Z mysqltest: In included file "./suite/galera/include/galera_concurrent_test.inc": 2019-12-30T19:34:36.8855313Z included from /__w/1/s/mysql-test/suite/galera_sr/t/GCF-1060.test at line 10: 2019-12-30T19:34:36.8856100Z At line 76: query 'reap' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0... 2019-12-30T19:34:36.8858257Z 2019-12-30T19:34:36.8858662Z The result from queries just before the failure was: 2019-12-30T19:34:36.8858952Z < snip > 2019-12-30T19:34:36.8859205Z connect node_1a, 127.0.0.1, root, , test, $NODE_MYPORT_1; 2019-12-30T19:34:36.8859459Z Running a concurrent test with the following queries: 2019-12-30T19:34:36.8859715Z TRUNCATE TABLE t1 2019-12-30T19:34:36.8860279Z INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x') 2019-12-30T19:34:36.8861260Z INSERT INTO t1 VALUE (4, 'z'); 2019-12-30T19:34:36.8861717Z connection node_1; 2019-12-30T19:34:36.8861976Z DROP TABLE IF EXISTS t1; 2019-12-30T19:34:36.8862239Z CREATE TABLE t1 (f1 INTEGER PRIMARY KEY AUTO_INCREMENT, f2 CHAR(255)) ENGINE=InnoDB; 2019-12-30T19:34:36.8862510Z SET SESSION wsrep_sync_wait = 0; 2019-12-30T19:34:36.8862756Z SET SESSION wsrep_trx_fragment_size = 1;;; 2019-12-30T19:34:36.8862990Z connection node_1a; 2019-12-30T19:34:36.8863238Z SET SESSION wsrep_sync_wait = 0; 2019-12-30T19:34:36.8863712Z SET SESSION wsrep_trx_fragment_size = 1;;; 2019-12-30T19:34:36.8864047Z connection node_2; 2019-12-30T19:34:36.8864244Z SET SESSION wsrep_sync_wait = 0; 2019-12-30T19:34:36.8864457Z SET SESSION wsrep_trx_fragment_size = 1;;; 2019-12-30T19:34:36.8864648Z 2019-12-30T19:34:36.8865131Z More results from queries before failure can be found in /var/tmp/mtr/1/log/GCF-1060.log 2019-12-30T19:34:36.8865415Z 2019-12-30T19:34:36.8865619Z 2019-12-30T19:34:36.8866327Z Server [mysqld.1 - pid: 46958, winpid: 46958, exit: 256] failed during test run 2019-12-30T19:34:36.8866695Z Server log from this test: 2019-12-30T19:34:36.8867601Z ----------SERVER LOG START----------- 2019-12-30T19:34:36.8868310Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8868659Z schema: test 2019-12-30T19:34:36.8868923Z request: (24906 seqno 27114 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8869485Z granted: (24925 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8870110Z 2019-12-30 19:34:35 24925 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi 2019-12-30T19:34:36.8870789Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8871208Z schema: test 2019-12-30T19:34:36.8871443Z request: (24906 seqno 27179 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8871940Z granted: (24965 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8872502Z 2019-12-30 19:34:35 24965 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi 2019-12-30T19:34:36.8873014Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8873307Z schema: test 2019-12-30T19:34:36.8873539Z request: (24906 seqno 27239 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8874026Z granted: (25001 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8874579Z 2019-12-30 19:34:35 25001 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi 2019-12-30T19:34:36.8875086Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8875380Z schema: test 2019-12-30T19:34:36.8875613Z request: (24906 seqno 27293 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8876100Z granted: (25032 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8876885Z 2019-12-30 19:34:35 25032 [Note] WSREP: rollbacker aborting SR thd: (25032 139902393857792) 2019-12-30T19:34:36.8877862Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8878287Z schema: test 2019-12-30T19:34:36.8878561Z request: (24906 seqno 27319 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8879153Z granted: (25049 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8879791Z 2019-12-30 19:34:35 25049 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi 2019-12-30T19:34:36.8880575Z 2019-12-30 19:34:35 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8895481Z schema: test 2019-12-30T19:34:36.8895943Z request: (24906 seqno 27358 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8896606Z granted: (25071 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8897301Z 2019-12-30 19:34:35 25071 [Note] WSREP: wsrep_rollback_streaming_aborted_by_toi 2019-12-30T19:34:36.8898548Z 2019-12-30 19:34:36 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8898904Z schema: test 2019-12-30T19:34:36.8899163Z request: (24906 seqno 27561 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8899767Z granted: (25183 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8900397Z 2019-12-30 19:34:36 25183 [Note] WSREP: rollbacker aborting SR thd: (25183 139902393857792) 2019-12-30T19:34:36.8902006Z 2019-12-30 19:34:36 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8902621Z schema: test 2019-12-30T19:34:36.8902942Z request: (24906 seqno 27614 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8903499Z granted: (25214 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8904076Z 2019-12-30 19:34:36 25214 [Note] WSREP: rollbacker aborting SR thd: (25214 139902393857792) 2019-12-30T19:34:36.8904635Z 2019-12-30 19:34:36 24906 [Note] WSREP: MDL conflict, DDL vs SR 2019-12-30T19:34:36.8904922Z schema: test 2019-12-30T19:34:36.8905141Z request: (24906 seqno 27654 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) 2019-12-30T19:34:36.8905630Z granted: (25238 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2019-12-30T19:34:36.8912288Z 2019-12-30 19:34:36 25238 [Note] WSREP: rollbacker aborting SR thd: (25238 139902393857792) 2019-12-30T19:34:36.8913101Z 2019-12-30 19:34:36 0x7f3d907d3700 InnoDB: Assertion failure in file /__w/1/s/BUILD/storage/innobase/trx/trx0roll.cc line 294 2019-12-30T19:34:36.8913451Z InnoDB: We intentionally generate a memory trap. 2019-12-30T19:34:36.8913708Z InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ 2019-12-30T19:34:36.8913940Z InnoDB: If you get repeated assertion failures or crashes, even 2019-12-30T19:34:36.8914182Z InnoDB: immediately after the mysqld startup, there may be 2019-12-30T19:34:36.8914415Z InnoDB: corruption in the InnoDB tablespace. Please refer to 2019-12-30T19:34:36.8914888Z InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/ 2019-12-30T19:34:36.8915169Z InnoDB: about forcing recovery. 2019-12-30T19:34:36.8915384Z 191230 19:34:36 [ERROR] mysqld got signal 6 ; 2019-12-30T19:34:36.8915615Z This could be because you hit a bug. It is also possible that this binary 2019-12-30T19:34:36.8915859Z or one of the libraries it was linked against is corrupt, improperly built, 2019-12-30T19:34:36.8916101Z or misconfigured. This error can also be caused by malfunctioning hardware. 2019-12-30T19:34:36.8916302Z 2019-12-30T19:34:36.8916733Z To report this bug, see https://mariadb.com/kb/en/reporting-bugs 2019-12-30T19:34:36.8916981Z 2019-12-30T19:34:36.8917190Z We will try our best to scrape up some info that will hopefully help 2019-12-30T19:34:36.8917874Z diagnose the problem, but since we have already crashed, 2019-12-30T19:34:36.8918506Z something is definitely wrong and this may fail. 2019-12-30T19:34:36.8918955Z 2019-12-30T19:34:36.8919507Z Server version: 10.4.11-5-MariaDB-enterprise-log 2019-12-30T19:34:36.8919844Z key_buffer_size=1048576 2019-12-30T19:34:36.8920084Z read_buffer_size=131072 2019-12-30T19:34:36.8920309Z max_used_connections=4 2019-12-30T19:34:36.8920552Z max_threads=153 2019-12-30T19:34:36.8920903Z thread_count=13 2019-12-30T19:34:36.8921225Z It is possible that mysqld could use up to 2019-12-30T19:34:36.8921461Z key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63529 K bytes of memory 2019-12-30T19:34:36.8922081Z Hope that's ok; if not, decrease some variables in the equation. 2019-12-30T19:34:36.8922338Z 2019-12-30T19:34:36.8922539Z Thread pointer: 0x7f3d80048398 2019-12-30T19:34:36.8922759Z Attempting backtrace. You can use the following information to find out 2019-12-30T19:34:36.8922997Z where mysqld died. If you see no messages after this, something went 2019-12-30T19:34:36.8923222Z terribly wrong... 2019-12-30T19:34:36.8923431Z stack_bottom = 0x7f3d907888e8 thread_stack 0x49000 2019-12-30T19:34:36.8924032Z mysqld: /__w/1/s/BUILD/wsrep-lib/include/wsrep/client_state.hpp:498: int wsrep::client_state::after_rollback(): Assertion `owning_thread_id_ == wsrep::this_thread::get_id()' failed. 2019-12-30T19:34:36.8924597Z ----------SERVER LOG END------------- 2019-12-30T19:34:36.8924834Z 2019-12-30T19:34:36.8925006Z 2019-12-30T19:34:36.8926150Z - saving '/var/tmp/mtr/1/log/galera_sr.GCF-1060-innodb/' to '/var/tmp/mtr/log/galera_sr.GCF-1060-innodb/' 2019-12-30T19:34:36.8937124Z ***Warnings generated in error logs during shutdown after running tests: galera_sr.mysql-wsrep-features#8 galera_sr.GCF-1060 galera_sr.GCF-1051 galera_sr.GCF-1008 galera_sr.GCF-1018 galera_sr.galera_sr_load_data_splitting galera_sr.GCF-1043B galera_sr.galera_sr_many_fragments galera_sr.galera_sr_kill_all_pcrecovery galera_sr.mysql-wsrep-features#9 2019-12-30T19:34:36.8938511Z 2019-12-30T19:34:36.8938884Z Attempting backtrace. You can use the following information to find out 2019-12-30T19:34:36.8939713Z mysqld: /__w/1/s/BUILD/wsrep-lib/include/wsrep/client_state.hpp:498: int wsrep::client_state::after_rollback(): Assertion `owning_thread_id_ == wsrep::this_thread::get_id()' failed. 2019-12-30T19:34:36.8940107Z

            I'm closing this ticket, as there was a more recent ticket about this test: MDEV-21677.
            Moreover, MDEV-21677 has been closed too:
            https://jira.mariadb.org/browse/MDEV-21677?focusedCommentId=146335&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-146335
            Should test galera_sr.GCF-1060 fail again. A new ticket should be opened.

            sciascid Daniele Sciascia added a comment - I'm closing this ticket, as there was a more recent ticket about this test: MDEV-21677 . Moreover, MDEV-21677 has been closed too: https://jira.mariadb.org/browse/MDEV-21677?focusedCommentId=146335&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-146335 Should test galera_sr.GCF-1060 fail again. A new ticket should be opened.

            I am reopening it since it is still disabled in 10.4/10.5 CS/ES.
            We should close it only after enabling it and verifying that it does not fail on CI.

            stepan.patryshev Stepan Patryshev (Inactive) added a comment - I am reopening it since it is still disabled in 10.4/10.5 CS/ES. We should close it only after enabling it and verifying that it does not fail on CI.

            sciascid This could be a new bug but test still fails:

            • How to repeat: ./mtr galera_sr.GCF-1060 --enable-disabled --repeat=20

            galera_sr.GCF-1060 'innodb'              [ 3 fail ]
                    Test ended at 2020-09-14 12:11:29
             
            CURRENT_TEST: galera_sr.GCF-1060
            mysqltest: In included file "./suite/galera/include/galera_concurrent_test.inc": 
            included from /home/jan/mysql/10.4-bugs/mysql-test/suite/galera_sr/t/GCF-1060.test at line 10:
            At line 76: query 'reap' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
             
            The result from queries just before the failure was:
            < snip >
            connect node_1a, 127.0.0.1, root, , test, $NODE_MYPORT_1;
            Running a concurrent test with the following queries:
            TRUNCATE TABLE t1
            INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x')
            INSERT INTO t1 VALUE (4, 'z');
            connection node_1;
            DROP TABLE IF EXISTS t1;
            CREATE TABLE t1 (f1 INTEGER PRIMARY KEY AUTO_INCREMENT, f2 CHAR(255)) ENGINE=InnoDB;
            SET SESSION wsrep_sync_wait = 0;
            SET SESSION wsrep_trx_fragment_size = 1;;;
            connection node_1a;
            SET SESSION wsrep_sync_wait = 0;
            SET SESSION wsrep_trx_fragment_size = 1;;;
            connection node_2;
            SET SESSION wsrep_sync_wait = 0;
            SET SESSION wsrep_trx_fragment_size = 1;;;
             
            More results from queries before failure can be found in /home/jan/mysql/10.4-bugs/mysql-test/var/log/GCF-1060.log
             
             
            Server [mysqld.1 - pid: 612043, winpid: 612043, exit: 256] failed during test run
            Server log from this test:
            ----------SERVER LOG START-----------
            2020-09-14 12:11:20 230 [Note] WSREP: MDL conflict, DDL vs SR
            schema:  test
            request: (230 	seqno 1341 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            granted: (258 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            2020-09-14 12:11:28 230 [Note] WSREP: MDL conflict, DDL vs SR
            schema:  test
            request: (230 	seqno 1628 	wsrep (toi, exec, aborted) cmd 3 8 	TRUNCATE TABLE t1)
            granted: (304 	seqno -1 	wsrep (high priority, exec, executing) cmd 0 161 	(null))
            mysqld: /home/jan/mysql/10.4-bugs/storage/innobase/lock/lock0lock.cc:6219: void lock_unlock_table_autoinc(trx_t*): Assertion `trx_state_eq(trx, TRX_STATE_NOT_STARTED) || !trx_state_eq(trx, TRX_STATE_COMMITTED_IN_MEMORY)' failed.
            200914 12:11:28 [ERROR] mysqld got signal 6 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.
             
            To report this bug, see https://mariadb.com/kb/en/reporting-bugs
             
            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed, 
            something is definitely wrong and this may fail.
             
            Server version: 10.4.15-MariaDB-debug-log
            key_buffer_size=1048576
            read_buffer_size=131072
            max_used_connections=4
            max_threads=153
            thread_count=13
            It is possible that mysqld could use up to 
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63637 K  bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.
             
            Thread pointer: 0x62b000317270
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            stack_bottom = 0x7f9e7af3a930 thread_stack 0x5fc00
            mysqld: /home/jan/mysql/10.4-bugs/wsrep-lib/include/wsrep/client_state.hpp:501: int wsrep::client_state::after_rollback(): Assertion `owning_thread_id_ == wsrep::this_thread::get_id()' failed.
            ----------SERVER LOG END-------------
             
             
             - saving '/home/jan/mysql/10.4-bugs/mysql-test/var/log/galera_sr.GCF-1060-innodb/' to '/home/jan/mysql/10.4-bugs/mysql-test/var/log/galera_sr.GCF-1060-innodb/'
            

            jplindst Jan Lindström (Inactive) added a comment - sciascid This could be a new bug but test still fails: How to repeat: ./mtr galera_sr.GCF-1060 --enable-disabled --repeat=20 galera_sr.GCF-1060 'innodb' [ 3 fail ] Test ended at 2020-09-14 12:11:29   CURRENT_TEST: galera_sr.GCF-1060 mysqltest: In included file "./suite/galera/include/galera_concurrent_test.inc": included from /home/jan/mysql/10.4-bugs/mysql-test/suite/galera_sr/t/GCF-1060.test at line 10: At line 76: query 'reap' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...   The result from queries just before the failure was: < snip > connect node_1a, 127.0.0.1, root, , test, $NODE_MYPORT_1; Running a concurrent test with the following queries: TRUNCATE TABLE t1 INSERT INTO t1 VALUE (1,'x'),(2,'x'),(3,'x') INSERT INTO t1 VALUE (4, 'z'); connection node_1; DROP TABLE IF EXISTS t1; CREATE TABLE t1 (f1 INTEGER PRIMARY KEY AUTO_INCREMENT, f2 CHAR(255)) ENGINE=InnoDB; SET SESSION wsrep_sync_wait = 0; SET SESSION wsrep_trx_fragment_size = 1;;; connection node_1a; SET SESSION wsrep_sync_wait = 0; SET SESSION wsrep_trx_fragment_size = 1;;; connection node_2; SET SESSION wsrep_sync_wait = 0; SET SESSION wsrep_trx_fragment_size = 1;;;   More results from queries before failure can be found in /home/jan/mysql/10.4-bugs/mysql-test/var/log/GCF-1060.log     Server [mysqld.1 - pid: 612043, winpid: 612043, exit: 256] failed during test run Server log from this test: ----------SERVER LOG START----------- 2020-09-14 12:11:20 230 [Note] WSREP: MDL conflict, DDL vs SR schema: test request: (230 seqno 1341 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) granted: (258 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) 2020-09-14 12:11:28 230 [Note] WSREP: MDL conflict, DDL vs SR schema: test request: (230 seqno 1628 wsrep (toi, exec, aborted) cmd 3 8 TRUNCATE TABLE t1) granted: (304 seqno -1 wsrep (high priority, exec, executing) cmd 0 161 (null)) mysqld: /home/jan/mysql/10.4-bugs/storage/innobase/lock/lock0lock.cc:6219: void lock_unlock_table_autoinc(trx_t*): Assertion `trx_state_eq(trx, TRX_STATE_NOT_STARTED) || !trx_state_eq(trx, TRX_STATE_COMMITTED_IN_MEMORY)' failed. 200914 12:11:28 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware.   To report this bug, see https://mariadb.com/kb/en/reporting-bugs   We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail.   Server version: 10.4.15-MariaDB-debug-log key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=4 max_threads=153 thread_count=13 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63637 K bytes of memory Hope that's ok; if not, decrease some variables in the equation.   Thread pointer: 0x62b000317270 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x7f9e7af3a930 thread_stack 0x5fc00 mysqld: /home/jan/mysql/10.4-bugs/wsrep-lib/include/wsrep/client_state.hpp:501: int wsrep::client_state::after_rollback(): Assertion `owning_thread_id_ == wsrep::this_thread::get_id()' failed. ----------SERVER LOG END-------------     - saving '/home/jan/mysql/10.4-bugs/mysql-test/var/log/galera_sr.GCF-1060-innodb/' to '/home/jan/mysql/10.4-bugs/mysql-test/var/log/galera_sr.GCF-1060-innodb/'

            People

              jplindst Jan Lindström (Inactive)
              jplindst Jan Lindström (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.