Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10331

innodb.innodb_sys_semaphore_waits failed in buildbot: InnoDB: Error: semaphore wait has lasted > 1 seconds

    Details

      Description

      http://buildbot.askmonty.org/buildbot/builders/p8-xenial-bintar-debug/builds/39/steps/test/logs/mysqld.1.err.4

      innodb.innodb_sys_semaphore_waits 'innodb_plugin' w4 [ fail ]
              Test ended at 2016-07-02 13:49:45
       
      CURRENT_TEST: innodb.innodb_sys_semaphore_waits
      mysqltest: At line 111: query 'drop table if exists t1' failed: 2013: Lost connection to MySQL server during query
       
      The result from queries just before the failure was:
      < snip >
      insert into t1 values(0, 0);
      set DEBUG_DBUG='+d,fatal-semaphore-timeout';
      set autocommit=0;
      # Sending query on con1,
      # the session will hold lock table mutex and sleep
      SELECT * from t1 where id = 0 FOR UPDATE;
      connection con2;
      set autocommit=0;
      # Sending query on con2,
      # the session will be blocked on the lock table mutex and
      # thus be put into sync arry
      SELECT * from t1 where id = 0 FOR UPDATE;
      connection default;
      # Waitting for mysqld to crash
      # Mysqld crash was detected
      # Waitting for reconnect after mysqld restarts
      # Reconnected after mysqld was successfully restarted
      # Cleaning up before exit
      set DEBUG_DBUG=NULL;
      drop table if exists t1;
       
      More results from queries before failure can be found in /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/4/log/innodb_sys_semaphore_waits.log
       
       - saving '/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/4/log/innodb.innodb_sys_semaphore_waits-innodb_plugin/' to '/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/log/innodb.innodb_sys_semaphore_waits-innodb_plugin/'
       - found 'core' (0/5)
       
      Trying 'dbx' to get a backtrace
       
      Trying 'gdb' to get a backtrace
      Core generated by '/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld'
      Output from gdb follows. The first stack trace is from the failing thread.
      The following stack traces are from all threads (so the failing one is
      duplicated).
      --------------------------
      [New LWP 16344]
      [New LWP 16327]
      [New LWP 16324]
      [New LWP 16328]
      [New LWP 16329]
      [New LWP 16326]
      [New LWP 16330]
      [New LWP 16333]
      [New LWP 16331]
      [New LWP 16345]
      [New LWP 16332]
      [New LWP 16346]
      [New LWP 16343]
      [New LWP 16348]
      [New LWP 16347]
      [New LWP 16353]
      [New LWP 16351]
      [New LWP 16352]
      [New LWP 16354]
      [New LWP 16355]
      [New LWP 16361]
      [New LWP 16370]
      [New LWP 16371]
      [Thread debugging using libthread_db enabled]
      Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
      Core was generated by `/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld --default'.
      Program terminated with signal SIGABRT, Aborted.
      #0  0x00003fffa4272e4c in pthread_kill () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      [Current thread is 1 (Thread 0x3fff94ffefb0 (LWP 16344))]
      #0  0x00003fffa4272e4c in pthread_kill () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00000000253946c0 in my_write_core (sig=6) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysys/stacktrace.c:456
      #2  0x0000000024b1b894 in handle_fatal_signal (sig=6) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/signal_handler.cc:272
      #3  <signal handler called>
      #4  0x00003fffa3ccf27c in raise () from /lib/powerpc64le-linux-gnu/libc.so.6
      #5  0x00003fffa3cd18f4 in abort () from /lib/powerpc64le-linux-gnu/libc.so.6
      #6  0x00003fff9ca5c350 in srv_error_monitor_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1958
      #7  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #8  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 23 (Thread 0x3fffa3887bb0 (LWP 16371)):
      #0  0x00003fffa427127c in pthread_cond_wait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949764 in os_cond_wait (cond=0x3fff9a5c4640, fast_mutex=0x3fff9a5c4600) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:214
      #2  0x00003fff9c94a03c in os_event_wait_low (event=0x3fff9a5c4600, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:592
      #3  0x00003fff9ca69b38 in sync_array_wait_event (arr=0x3fffa3330640, index=0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/sync/sync0arr.cc:465
      #4  0x00003fff9ca717f8 in mutex_spin_wait (mutex=0x3fff9a760078, file_name=0x3fff9cb17c80 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/lock/lock0lock.cc", line=5025) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/sync/sync0sync.cc:604
      #5  0x00003fff9c8f3b94 in mutex_enter_func (mutex=0x3fff9a760078, file_name=0x3fff9cb17c80 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/lock/lock0lock.cc", line=5025) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/include/sync0sync.ic:225
      #6  0x00003fff9c8f3c54 in pfs_mutex_enter_func (mutex=0x3fff9a760078, file_name=0x3fff9cb17c80 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/lock/lock0lock.cc", line=5025) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/include/sync0sync.ic:251
      #7  0x00003fff9c907894 in lock_table (flags=0, table=0x3fff8de703f8, mode=LOCK_IX, thr=0x3fff8d461708) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/lock/lock0lock.cc:5025
      #8  0x00003fff9ca282d4 in row_search_for_mysql (buf=0x3fff91b82488 "\377", mode=1, prebuilt=0x3fff8d461078, match_mode=0, direction=0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/row/row0sel.cc:4108
      #9  0x00003fff9c88e848 in ha_innodb::index_read (this=0x3fff8daa2088, buf=0x3fff91b82488 "\377", key_ptr=0x0, key_len=0, find_flag=HA_READ_AFTER_KEY) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:9197
      #10 0x00003fff9c88fa10 in ha_innodb::index_first (this=0x3fff8daa2088, buf=0x3fff91b82488 "\377") at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:9605
      #11 0x00003fff9c88fd64 in ha_innodb::rnd_next (this=0x3fff8daa2088, buf=0x3fff91b82488 "\377") at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:9702
      #12 0x0000000024b23b4c in handler::ha_rnd_next (this=0x3fff8daa2088, buf=0x3fff91b82488 "\377") at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/handler.cc:2574
      #13 0x0000000024ceaa70 in rr_sequential (info=0x3fff8d9188b8) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/records.cc:474
      #14 0x00000000248b4870 in join_init_read_record (tab=0x3fff8d9187f8) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:19226
      #15 0x00000000248b1cf4 in sub_select (join=0x3fff8d916c90, join_tab=0x3fff8d9187f8, end_of_records=false) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:18297
      #16 0x00000000248b1070 in do_select (join=0x3fff8d916c90, procedure=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:17843
      #17 0x0000000024888aac in JOIN::exec_inner (this=0x3fff8d916c90) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:3341
      #18 0x0000000024887e98 in JOIN::exec (this=0x3fff8d916c90) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:3153
      #19 0x0000000024889298 in mysql_select (thd=0x3fff91a84070, tables=0x3fff8d916290, wild_num=1, fields=..., conds=0x3fff8d916a38, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2148272640, result=0x3fff8d916c70, unit=0x3fff91a87ac8, select_lex=0x3fff91a881f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:3537
      #20 0x000000002487ec8c in handle_select (thd=0x3fff91a84070, lex=0x3fff91a87a00, result=0x3fff8d916c70, setup_tables_done_option=0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:377
      #21 0x0000000024840f00 in execute_sqlcom_select (thd=0x3fff91a84070, all_tables=0x3fff8d916290) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:6309
      #22 0x0000000024836b90 in mysql_execute_command (thd=0x3fff91a84070) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:3352
      #23 0x0000000024844d78 in mysql_parse (thd=0x3fff91a84070, rawbuf=0x3fff8d916088 "SELECT * from t1 where id = 0 FOR UPDATE", length=40, parser_state=0x3fffa38863b8, is_com_multi=false, is_next_command=false) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:7741
      #24 0x0000000024832140 in dispatch_command (command=COM_QUERY, thd=0x3fff91a84070, packet=0x3fff8d811071 "", packet_length=40, is_com_multi=false, is_next_command=false) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:1790
      #25 0x0000000024830b14 in do_command (thd=0x3fff91a84070) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:1353
      #26 0x00000000249ac270 in do_handle_one_connection (connect=0x3fffa32e25b0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_connect.cc:1354
      #27 0x00000000249abec8 in handle_one_connection (arg=0x3fffa32e25b0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_connect.cc:1260
      #28 0x00000000251a0704 in pfs_spawn_thread (arg=0x3fffa325dff0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/perfschema/pfs.cc:1862
      #29 0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #30 0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 22 (Thread 0x3fffa38e7bb0 (LWP 16370)):
      #0  0x00003fffa3da9638 in select () from /lib/powerpc64le-linux-gnu/libc.so.6
      #1  0x00003fff9c94b45c in os_thread_sleep (tm=3600000000) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0thread.cc:261
      #2  0x00003fff9c9078d8 in lock_table (flags=0, table=0x3fff8de703f8, mode=LOCK_IX, thr=0x3fff8cf41a08) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/lock/lock0lock.cc:5027
      #3  0x00003fff9ca282d4 in row_search_for_mysql (buf=0x3fff91fa0b88 "\377", mode=1, prebuilt=0x3fff8cf41078, match_mode=0, direction=0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/row/row0sel.cc:4108
      #4  0x00003fff9c88e848 in ha_innodb::index_read (this=0x3fff8de81088, buf=0x3fff91fa0b88 "\377", key_ptr=0x0, key_len=0, find_flag=HA_READ_AFTER_KEY) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:9197
      #5  0x00003fff9c88fa10 in ha_innodb::index_first (this=0x3fff8de81088, buf=0x3fff91fa0b88 "\377") at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:9605
      #6  0x00003fff9c88fd64 in ha_innodb::rnd_next (this=0x3fff8de81088, buf=0x3fff91fa0b88 "\377") at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:9702
      #7  0x0000000024b23b4c in handler::ha_rnd_next (this=0x3fff8de81088, buf=0x3fff91fa0b88 "\377") at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/handler.cc:2574
      #8  0x0000000024ceaa70 in rr_sequential (info=0x3fff8dd188b8) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/records.cc:474
      #9  0x00000000248b4870 in join_init_read_record (tab=0x3fff8dd187f8) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:19226
      #10 0x00000000248b1cf4 in sub_select (join=0x3fff8dd16c90, join_tab=0x3fff8dd187f8, end_of_records=false) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:18297
      #11 0x00000000248b1070 in do_select (join=0x3fff8dd16c90, procedure=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:17843
      #12 0x0000000024888aac in JOIN::exec_inner (this=0x3fff8dd16c90) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:3341
      #13 0x0000000024887e98 in JOIN::exec (this=0x3fff8dd16c90) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:3153
      #14 0x0000000024889298 in mysql_select (thd=0x3fff91ea4070, tables=0x3fff8dd16290, wild_num=1, fields=..., conds=0x3fff8dd16a38, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2148272640, result=0x3fff8dd16c70, unit=0x3fff91ea7ac8, select_lex=0x3fff91ea81f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:3537
      #15 0x000000002487ec8c in handle_select (thd=0x3fff91ea4070, lex=0x3fff91ea7a00, result=0x3fff8dd16c70, setup_tables_done_option=0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_select.cc:377
      #16 0x0000000024840f00 in execute_sqlcom_select (thd=0x3fff91ea4070, all_tables=0x3fff8dd16290) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:6309
      #17 0x0000000024836b90 in mysql_execute_command (thd=0x3fff91ea4070) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:3352
      #18 0x0000000024844d78 in mysql_parse (thd=0x3fff91ea4070, rawbuf=0x3fff8dd16088 "SELECT * from t1 where id = 0 FOR UPDATE", length=40, parser_state=0x3fffa38e63b8, is_com_multi=false, is_next_command=false) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:7741
      #19 0x0000000024832140 in dispatch_command (command=COM_QUERY, thd=0x3fff91ea4070, packet=0x3fff8dc11071 "", packet_length=40, is_com_multi=false, is_next_command=false) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:1790
      #20 0x0000000024830b14 in do_command (thd=0x3fff91ea4070) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:1353
      #21 0x00000000249ac270 in do_handle_one_connection (connect=0x3fffa32e24d0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_connect.cc:1354
      #22 0x00000000249abec8 in handle_one_connection (arg=0x3fffa32e24d0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_connect.cc:1260
      #23 0x00000000251a0704 in pfs_spawn_thread (arg=0x3fffa325daf0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/perfschema/pfs.cc:1862
      #24 0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #25 0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 21 (Thread 0x3fffa3947bb0 (LWP 16361)):
      #0  0x00003fffa3da6818 in poll () from /lib/powerpc64le-linux-gnu/libc.so.6
      #1  0x000000002540ddcc in vio_io_wait (vio=0x3fffa325d5f0, event=VIO_IO_EVENT_READ, timeout=28800000) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/vio/viosocket.c:925
      #2  0x000000002540c01c in vio_socket_io_wait (vio=0x3fffa325d5f0, event=VIO_IO_EVENT_READ) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/vio/viosocket.c:108
      #3  0x000000002540c258 in vio_read (vio=0x3fffa325d5f0, buf=0x3fff8e4e1070 "\001", size=4) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/vio/viosocket.c:184
      #4  0x0000000024742914 in my_real_read (net=0x3fff92674330, complen=0x3fffa3946b10, header=1 '\001') at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/net_serv.cc:885
      #5  0x0000000024743358 in my_net_read_packet (net=0x3fff92674330, read_from_server=1 '\001') at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/net_serv.cc:1111
      #6  0x0000000024830610 in do_command (thd=0x3fff92674070) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_parse.cc:1235
      #7  0x00000000249ac270 in do_handle_one_connection (connect=0x3fffa32e2310) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_connect.cc:1354
      #8  0x00000000249abec8 in handle_one_connection (arg=0x3fffa32e2310) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/sql_connect.cc:1260
      #9  0x00000000251a0704 in pfs_spawn_thread (arg=0x3fffa325d370) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/perfschema/pfs.cc:1862
      #10 0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #11 0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 20 (Thread 0x3fffa39a7bb0 (LWP 16355)):
      #0  0x00003fffa4278514 in sigwait () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x000000002472ea10 in signal_hand (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld.cc:3558
      #2  0x00000000251a0704 in pfs_spawn_thread (arg=0x3fffa325b2f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/perfschema/pfs.cc:1862
      #3  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #4  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 19 (Thread 0x3fff8fffefb0 (LWP 16354)):
      #0  0x00003fffa3da9638 in select () from /lib/powerpc64le-linux-gnu/libc.so.6
      #1  0x00003fff9c94b45c in os_thread_sleep (tm=1000000) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0thread.cc:261
      #2  0x00003fff9c76292c in btr_defragment_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/btr/btr0defragment.cc:758
      #3  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #4  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 18 (Thread 0x3fff90ffefb0 (LWP 16352)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949604 in os_cond_wait_timed (cond=0x3fff9a5c4e40, fast_mutex=0x3fff9a5c4e00, abstime=0x3fff90ffe4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:177
      #2  0x00003fff9c94a25c in os_event_wait_time_low (event=0x3fff9a5c4e00, time_in_usec=10000000, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:705
      #3  0x00003fff9c7fd544 in dict_stats_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/dict/dict0stats_bg.cc:545
      #4  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #5  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 17 (Thread 0x3fff917fefb0 (LWP 16351)):
      #0  0x00003fffa427127c in pthread_cond_wait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949764 in os_cond_wait (cond=0x3fffa3260e40, fast_mutex=0x3fffa3260e00) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:214
      #2  0x00003fff9c94a03c in os_event_wait_low (event=0x3fffa3260e00, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:592
      #3  0x00003fff9c78848c in buf_dump_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/buf/buf0dump.cc:703
      #4  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #5  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 16 (Thread 0x3fff907fefb0 (LWP 16353)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949604 in os_cond_wait_timed (cond=0x3fff9a5ca9c0, fast_mutex=0x3fff9a5ca980, abstime=0x3fff907fe3e0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:177
      #2  0x00003fff9c94a25c in os_event_wait_time_low (event=0x3fff9a5ca980, time_in_usec=5000000, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:705
      #3  0x00003fff9caba090 in ib_wqueue_timedwait (wq=0x3fffa32f0df8, wait_in_usecs=5000000) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/ut/ut0wqueue.cc:154
      #4  0x00003fff9c8652cc in fts_optimize_thread (arg=0x3fffa32f0df8) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fts/fts0opt.cc:3064
      #5  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #6  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 15 (Thread 0x3fff937fefb0 (LWP 16347)):
      #0  0x00003fffa427127c in pthread_cond_wait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949764 in os_cond_wait (cond=0x3fffa3260cc0, fast_mutex=0x3fffa3260c80) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:214
      #2  0x00003fff9c94a03c in os_event_wait_low (event=0x3fffa3260c80, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:592
      #3  0x00003fff9ca5ea6c in srv_purge_coordinator_suspend (slot=0x3fffa3300700, rseg_history_len=6) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2848
      #4  0x00003fff9ca5f0d8 in srv_purge_coordinator_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2967
      #5  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #6  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 14 (Thread 0x3fff95ffefb0 (LWP 16348)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949604 in os_cond_wait_timed (cond=0x3fff9a5c42c0, fast_mutex=0x3fff9a5c4280, abstime=0x3fff95ffe450) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:177
      #2  0x00003fff9c94a25c in os_event_wait_time_low (event=0x3fff9a5c4280, time_in_usec=1000000, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:705
      #3  0x00003fff9c79204c in page_cleaner_sleep_if_needed (next_loop_time=1467481752231) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/buf/buf0flu.cc:2274
      #4  0x00003fff9c792188 in buf_flush_page_cleaner_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/buf/buf0flu.cc:2310
      #5  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #6  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 13 (Thread 0x3fff957fefb0 (LWP 16343)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949604 in os_cond_wait_timed (cond=0x3fff9a5c4740, fast_mutex=0x3fff9a5c4700, abstime=0x3fff957fe4d0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:177
      #2  0x00003fff9c94a25c in os_event_wait_time_low (event=0x3fff9a5c4700, time_in_usec=1000000, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:705
      #3  0x00003fff9c913f40 in lock_wait_timeout_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/lock/lock0wait.cc:536
      #4  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #5  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 12 (Thread 0x3fff93ffefb0 (LWP 16346)):
      #0  0x00003fffa3da9638 in select () from /lib/powerpc64le-linux-gnu/libc.so.6
      #1  0x00003fff9c94b45c in os_thread_sleep (tm=1000000) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0thread.cc:261
      #2  0x00003fff9ca5dcb8 in srv_master_sleep () at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2514
      #3  0x00003fff9ca5de84 in srv_master_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2558
      #4  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #5  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 11 (Thread 0x3fff97b8efb0 (LWP 16332)):
      #0  0x00003fffa4770738 in ?? () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #1  0x00003fffa47707a0 in io_getevents () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #2  0x00003fff9c9463fc in os_aio_linux_collect (array=0x3fffa33701f0, segment=0, seg_size=256) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5326
      #3  0x00003fff9c94689c in os_aio_linux_handle (global_seg=4, message1=0x3fff97b8e4e0, message2=0x3fff97b8e4e8, type=0x3fff97b8e4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5478
      #4  0x00003fff9c8187fc in fil_aio_wait (segment=4) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fil/fil0fil.cc:6023
      #5  0x00003fff9ca630e0 in io_handler_thread (arg=0x3fff9cbde520 <n+32>) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0start.cc:515
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 10 (Thread 0x3fff947fefb0 (LWP 16345)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00003fff9c949604 in os_cond_wait_timed (cond=0x3fffa3260dc0, fast_mutex=0x3fffa3260d80, abstime=0x3fff947fe460) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:177
      #2  0x00003fff9c94a25c in os_event_wait_time_low (event=0x3fffa3260d80, time_in_usec=5000000, reset_sig_count=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0sync.cc:705
      #3  0x00003fff9ca5bb70 in srv_monitor_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1727
      #4  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #5  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 9 (Thread 0x3fff9838efb0 (LWP 16331)):
      #0  0x00003fffa4770738 in ?? () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #1  0x00003fffa47707a0 in io_getevents () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #2  0x00003fff9c9463fc in os_aio_linux_collect (array=0x3fffa3370150, segment=1, seg_size=256) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5326
      #3  0x00003fff9c94689c in os_aio_linux_handle (global_seg=3, message1=0x3fff9838e4e0, message2=0x3fff9838e4e8, type=0x3fff9838e4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5478
      #4  0x00003fff9c8187fc in fil_aio_wait (segment=3) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fil/fil0fil.cc:6023
      #5  0x00003fff9ca630e0 in io_handler_thread (arg=0x3fff9cbde518 <n+24>) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0start.cc:515
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 8 (Thread 0x3fff9738efb0 (LWP 16333)):
      #0  0x00003fffa4770738 in ?? () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #1  0x00003fffa47707a0 in io_getevents () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #2  0x00003fff9c9463fc in os_aio_linux_collect (array=0x3fffa33701f0, segment=1, seg_size=256) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5326
      #3  0x00003fff9c94689c in os_aio_linux_handle (global_seg=5, message1=0x3fff9738e4e0, message2=0x3fff9738e4e8, type=0x3fff9738e4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5478
      #4  0x00003fff9c8187fc in fil_aio_wait (segment=5) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fil/fil0fil.cc:6023
      #5  0x00003fff9ca630e0 in io_handler_thread (arg=0x3fff9cbde528 <n+40>) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0start.cc:515
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 7 (Thread 0x3fff98b8efb0 (LWP 16330)):
      #0  0x00003fffa4770738 in ?? () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #1  0x00003fffa47707a0 in io_getevents () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #2  0x00003fff9c9463fc in os_aio_linux_collect (array=0x3fffa3370150, segment=0, seg_size=256) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5326
      #3  0x00003fff9c94689c in os_aio_linux_handle (global_seg=2, message1=0x3fff98b8e4e0, message2=0x3fff98b8e4e8, type=0x3fff98b8e4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5478
      #4  0x00003fff9c8187fc in fil_aio_wait (segment=2) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fil/fil0fil.cc:6023
      #5  0x00003fff9ca630e0 in io_handler_thread (arg=0x3fff9cbde510 <n+16>) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0start.cc:515
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 6 (Thread 0x3fff9d3fefb0 (LWP 16326)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x000000002539b9e8 in safe_cond_timedwait (cond=0x262f1e70 <COND_timer>, mp=0x262f1dc0 <LOCK_timer>, abstime=0x3fff9d3fe4d0, file=0x25641428 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysys/thr_mutex.c:545
      #2  0x000000002539d040 in inline_mysql_cond_timedwait (that=0x262f1e70 <COND_timer>, mutex=0x262f1dc0 <LOCK_timer>, abstime=0x3fff9d3fe4d0, src_file=0x25641488 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysys/thr_timer.c", src_line=292) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175
      #3  0x000000002539df64 in timer_handler (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysys/thr_timer.c:292
      #4  0x00000000251a0704 in pfs_spawn_thread (arg=0x3fffa3251cf0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/perfschema/pfs.cc:1862
      #5  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #6  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 5 (Thread 0x3fff9938efb0 (LWP 16329)):
      #0  0x00003fffa4770738 in ?? () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #1  0x00003fffa47707a0 in io_getevents () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #2  0x00003fff9c9463fc in os_aio_linux_collect (array=0x3fffa33700b0, segment=0, seg_size=256) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5326
      #3  0x00003fff9c94689c in os_aio_linux_handle (global_seg=1, message1=0x3fff9938e4e0, message2=0x3fff9938e4e8, type=0x3fff9938e4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5478
      #4  0x00003fff9c8187fc in fil_aio_wait (segment=1) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fil/fil0fil.cc:6023
      #5  0x00003fff9ca630e0 in io_handler_thread (arg=0x3fff9cbde508 <n+8>) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0start.cc:515
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 4 (Thread 0x3fff99b8efb0 (LWP 16328)):
      #0  0x00003fffa4770738 in ?? () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #1  0x00003fffa47707a0 in io_getevents () from /lib/powerpc64le-linux-gnu/libaio.so.1
      #2  0x00003fff9c9463fc in os_aio_linux_collect (array=0x3fffa3370100, segment=0, seg_size=256) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5326
      #3  0x00003fff9c94689c in os_aio_linux_handle (global_seg=0, message1=0x3fff99b8e4e0, message2=0x3fff99b8e4e8, type=0x3fff99b8e4f0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/os/os0file.cc:5478
      #4  0x00003fff9c8187fc in fil_aio_wait (segment=0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/fil/fil0fil.cc:6023
      #5  0x00003fff9ca630e0 in io_handler_thread (arg=0x3fff9cbde500 <n>) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0start.cc:515
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 3 (Thread 0x3fffa486c810 (LWP 16324)):
      #0  0x00003fffa3da6818 in poll () from /lib/powerpc64le-linux-gnu/libc.so.6
      #1  0x0000000024735b84 in handle_connections_sockets () at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld.cc:6560
      #2  0x0000000024734c64 in mysqld_main (argc=146, argv=0x3fffa33e0ac0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld.cc:6027
      #3  0x0000000024725d98 in main (argc=23, argv=0x3fffce2ad698) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/main.cc:25
       
      Thread 2 (Thread 0x3fff9b3fefb0 (LWP 16327)):
      #0  0x00003fffa42717e0 in pthread_cond_timedwait@@GLIBC_2.17 () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x000000002539b9e8 in safe_cond_timedwait (cond=0x262e8a90 <COND_checkpoint>, mp=0x262e89e0 <LOCK_checkpoint>, abstime=0x3fff9b3fe3d0, file=0x2561e0d8 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysys/thr_mutex.c:545
      #2  0x000000002522b398 in inline_mysql_cond_timedwait (that=0x262e8a90 <COND_checkpoint>, mutex=0x262e89e0 <LOCK_checkpoint>, abstime=0x3fff9b3fe3d0, src_file=0x2561e138 "/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/maria/ma_servicethread.c", src_line=116) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175
      #3  0x000000002522ba28 in my_service_thread_sleep (control=0x2589d2a8 <checkpoint_control>, sleep_time=29000000000) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/maria/ma_servicethread.c:115
      #4  0x000000002521c688 in ma_checkpoint_background (arg=0x1e) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/maria/ma_checkpoint.c:708
      #5  0x00000000251a0704 in pfs_spawn_thread (arg=0x3fffa3255b70) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/perfschema/pfs.cc:1862
      #6  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #7  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
       
      Thread 1 (Thread 0x3fff94ffefb0 (LWP 16344)):
      #0  0x00003fffa4272e4c in pthread_kill () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #1  0x00000000253946c0 in my_write_core (sig=6) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysys/stacktrace.c:456
      #2  0x0000000024b1b894 in handle_fatal_signal (sig=6) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/signal_handler.cc:272
      #3  <signal handler called>
      #4  0x00003fffa3ccf27c in raise () from /lib/powerpc64le-linux-gnu/libc.so.6
      #5  0x00003fffa3cd18f4 in abort () from /lib/powerpc64le-linux-gnu/libc.so.6
      #6  0x00003fff9ca5c350 in srv_error_monitor_thread (arg=0x0) at /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1958
      #7  0x00003fffa42684a0 in start_thread () from /lib/powerpc64le-linux-gnu/libpthread.so.0
      #8  0x00003fffa3db7e74 in clone () from /lib/powerpc64le-linux-gnu/libc.so.6
      

      Version: '10.2.1-MariaDB-debug'  socket: '/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/tmp/4/mysqld.1.sock'  port: 16060  Source distribution
      InnoDB: Error: semaphore wait has lasted > 1 seconds
      InnoDB: We intentionally crash the server, because it appears to be hung.
      2016-07-02 13:49:11 3fff94ffefb0  InnoDB: Assertion failure in thread 70366949011376 in file srv0srv.cc line 1958
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      160702 13:49:11 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.2.1-MariaDB-debug
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=3
      max_threads=153
      thread_count=3
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63022 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48c00
      /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld(my_print_stacktrace+0x3c)[0x2539457c]
      /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld(handle_fatal_signal+0x41c)[0x24b1b468]
      [0x3fffa47f04d8]
      /lib/powerpc64le-linux-gnu/libc.so.6(gsignal+0xac)[0x3fffa3ccf27c]
      /lib/powerpc64le-linux-gnu/libc.so.6(abort+0x2ec)[0x3fffa3cd19ec]
      /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/4/plugins/ha_innodb.so(+0x3bc350)[0x3fff9ca5c350]
      /lib/powerpc64le-linux-gnu/libpthread.so.0(+0x84a0)[0x3fffa42684a0]
      /lib/powerpc64le-linux-gnu/libc.so.6(clone+0xe4)[0x3fffa3db7e74]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      Writing a core file
      2016-07-02 13:49:12 70367320524816 [Note] /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld (mysqld 10.2.1-MariaDB-debug) starting as process 16405 ...
      2016-07-02 13:49:12 70367320524816 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4162)
      2016-07-02 13:49:12 70367320524816 [Note] Plugin 'partition' is disabled.
      2016-07-02 13:49:12 70367320524816 [Note] Plugin 'SEQUENCE' is disabled.
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Using mutexes to ref count buffer pool pages
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB:  InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB:  InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!!
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: The InnoDB memory heap is disabled
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Compressed tables use zlib 1.2.8
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Using Linux native AIO
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Using POWER8 crc32 instructions
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Initializing buffer pool, size = 8.0M
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Completed initialization of buffer pool
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Highest supported file format is Barracuda.
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: The log sequence numbers 1629546 and 1629546 in ibdata files do not match the log sequence number 1632829 in the ib_logfiles!
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Database was not shutdown normally!
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Starting crash recovery.
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Reading tablespace information from the .ibd files...
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: Restoring possible half-written data pages 
      2016-07-02 13:49:12 70367320524816 [Note] InnoDB: from the doublewrite buffer...
      2016-07-02 13:49:14 70367320524816 [Note] InnoDB: 128 rollback segment(s) are active.
      2016-07-02 13:49:15 70367320524816 [Note] InnoDB: Waiting for purge to start
      2016-07-02 13:49:15 70367320524816 [Note] InnoDB: 5.6.31 started; log sequence number 1632829
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_CMP' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'XTRADB_READ_VIEW' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'XTRADB_RSEG' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'CHANGED_PAGE_BITMAPS' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'XTRADB_INTERNAL_HASH_TABLES' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_MUTEXES' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'FEEDBACK' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
      2016-07-02 13:49:16 70367320524816 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
      2016-07-02 13:49:16 70367320524816 [Warning] /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld: unknown option '--loose-skip-plugin-innodb-changed-pages'
      2016-07-02 13:49:16 70367320524816 [Note] Recovering after a crash using tc.log
      2016-07-02 13:49:16 70367003537328 [Note] InnoDB: Dumping buffer pool(s) not yet started
      2016-07-02 13:49:17 70367320524816 [Note] Starting crash recovery...
      2016-07-02 13:49:17 70367320524816 [Note] Crash recovery finished.
      2016-07-02 13:49:17 70367320524816 [Note] Server socket created on IP: '127.0.0.1'.
      2016-07-02 13:49:17 70367320524816 [Note] /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld: ready for connections.
      Version: '10.2.1-MariaDB-debug'  socket: '/var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/tmp/4/mysqld.1.sock'  port: 16060  Source distribution
      InnoDB: Error: semaphore wait has lasted > 1 seconds
      InnoDB: We intentionally crash the server, because it appears to be hung.
      2016-07-02 13:49:29 3fff9bffefb0  InnoDB: Assertion failure in thread 70367066451888 in file srv0srv.cc line 1958
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      160702 13:49:29 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.2.1-MariaDB-debug
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=1
      max_threads=153
      thread_count=1
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63022 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48c00
      /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld(my_print_stacktrace+0x3c)[0x4717457c]
      /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/sql/mysqld(handle_fatal_signal+0x41c)[0x468fb468]
      [0x3fffab1d04d8]
      /lib/powerpc64le-linux-gnu/libc.so.6(gsignal+0xac)[0x3fffaa6af27c]
      /lib/powerpc64le-linux-gnu/libc.so.6(abort+0x2ec)[0x3fffaa6b19ec]
      /var/lib/buildbot/maria-slave/p8-xenial-bintar-debug/build/mysql-test/var/4/plugins/ha_innodb.so(+0x3bc350)[0x3fffa365c350]
      mysys/stacktrace.c:246(my_print_stacktrace)[0x3fffaac484a0]
      /lib/powerpc64le-linux-gnu/libc.so.6(clone+0xe4)[0x3fffaa797e74]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      Writing a core file
      CURRENT_TEST: innodb.innodb_simulate_comp_failures_small
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jplindst Jan Lindström
                Reporter:
                elenst Elena Stepanova
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: