Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7878

innodb_scrub_background fails sporadically in buildbot (Failing assertion: offset > 0 && offset < UNIV_PAGE_SIZE)

    XMLWordPrintable

Details

    Description

      http://buildbot.askmonty.org/buildbot/builders/p8-rhel71-bintar/builds/219/steps/test/logs/stdio

      innodb.innodb_scrub_background 'ecb,xtradb' w2 [ fail ]
              Test ended at 2015-03-29 12:16:42
       
      CURRENT_TEST: innodb.innodb_scrub_background
      mysqltest: At line 90: query 'create table t3 (
      a int auto_increment primary key,
      b varchar(256),
      c text, index(b)) engine = innodb row_format=$format' failed: 2013: Lost connection to MySQL server during query
       
      The result from queries just before the failure was:
      < snip >
      delete from t1;
      #
      # Test delete+rollback+delete
      #
      create table t2 (
      a int auto_increment primary key,
      b varchar(256),
      c text, index(b)) engine = innodb row_format=dynamic;
      # Populate table with rows
      begin;
      delete from t2;
      rollback;
      delete from t2;
      #
      # Test insert+rollback
      #
      create table t3 (
      a int auto_increment primary key,
      b varchar(256),
      c text, index(b)) engine = innodb row_format=dynamic;
       
      More results from queries before failure can be found in /home/buildbot/maria-slave/power8-vlp06-bintar/build/mysql-test/var/2/log/innodb_scrub_background.log
       
       
      Server [mysqld.1 - pid: 12910, winpid: 12910, exit: 256] failed during test run
      Server log from this test:
      ----------SERVER LOG START-----------
      150329 12:13:48 [Note] Plugin 'partition' is disabled.
      150329 12:13:48 [Note] Plugin 'SEQUENCE' is disabled.
      150329 12:13:48 [Note] InnoDB: Using mutexes to ref count buffer pool pages
      150329 12:13:48 [Note] InnoDB: The InnoDB memory heap is disabled
      150329 12:13:48 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
      150329 12:13:48 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
      150329 12:13:48 [Note] InnoDB: Compressed tables use zlib 1.2.3
      150329 12:13:48 [Note] InnoDB: Using Linux native AIO
      150329 12:13:48 [Note] InnoDB: Not using CPU crc32 instructions
      150329 12:13:50 [Note] InnoDB: Initializing buffer pool, size = 8.0M
      150329 12:13:50 [Note] InnoDB: Completed initialization of buffer pool
      150329 12:13:50 [Note] InnoDB: Highest supported file format is Barracuda.
      150329 12:13:50 [Note] InnoDB: Redo log crypto: unencrypted key ver.
      150329 12:13:51 [Note] InnoDB: 128 rollback segment(s) are active.
      150329 12:13:51 [Note] InnoDB: Waiting for purge to start
      150329 12:13:51 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.22-72.0 started; log sequence number 1629612
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
      150329 12:13:51 [Note] Plugin 'XTRADB_RSEG' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
      150329 12:13:51 [Note] Plugin 'XTRADB_INTERNAL_HASH_TABLES' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
      150329 12:13:51 [Note] Plugin 'FEEDBACK' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_CMP' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_MUTEXES' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
      150329 12:13:51 [Note] Plugin 'XTRADB_READ_VIEW' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
      150329 12:13:51 [Note] Plugin 'INNODB_CHANGED_PAGES' is disabled.
      150329 12:13:51 [Warning] /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld: unknown variable 'loose-innodb-scrub-force-testing=ON'
      150329 12:13:51 [Warning] /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld: unknown variable 'loose-debug-sync-timeout=300'
      150329 12:13:52 [Note] Server socket created on IP: '::'.
      150329 12:13:52 [Note] Event Scheduler: Loaded 0 events
      150329 12:13:52 [Note] WSREP: Read nil XID from storage engines, skipping position init
      150329 12:13:52 [Note] WSREP: wsrep_load(): loading provider library 'none'
      150329 12:13:52 [Note] /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld: ready for connections.
      Version: '10.1.3-MariaDB-wsrep-log'  socket: '/home/buildbot/maria-slave/power8-vlp06-bintar/build/mysql-test/var/tmp/2/mysqld.1.sock'  port: 16040  MariaDB Server, wsrep_25.10.r4144
      2015-03-29 12:16:35 3fff77fff170  InnoDB: Assertion failure in thread 70366462472560 in file fil0crypt.cc line 474
      InnoDB: Failing assertion: offset > 0 && offset < UNIV_PAGE_SIZE
      InnoDB: We intentionally generate a memory trap.
      InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
      InnoDB: If you get repeated assertion failures or crashes, even
      InnoDB: immediately after the mysqld startup, there may be
      InnoDB: corruption in the InnoDB tablespace. Please refer to
      InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
      InnoDB: about forcing recovery.
      150329 12:16:35 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see http://kb.askmonty.org/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.1.3-MariaDB-wsrep-log
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=1
      max_threads=153
      thread_count=1
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63014 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x0x0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x0 thread_stack 0x48000
      mysys/stacktrace.c:246(my_print_stacktrace)[0x36650bf4]
      sql/signal_handler.cc:153(handle_fatal_signal)[0x3605a080]
      linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fff98470478]
      /opt/at8.0-3-rc1/lib64/power8/libc.so.6(gsignal+0x48)[0x3fff97c4ee38]
      /opt/at8.0-3-rc1/lib64/power8/libc.so.6(abort+0x2ac)[0x3fff97c519dc]
      fil/fil0crypt.cc:474(fil_space_write_crypt_data_low)[0x365a69cc]
      fil/fil0crypt.cc:2039(fil_crypt_flush_space)[0x365acaa8]
      /opt/at8.0-3-rc1/lib64/power8/libpthread.so.0(+0x7cac)[0x3fff98437cac]
      /opt/at8.0-3-rc1/lib64/power8/libc.so.6(clone+0x98)[0x3fff97d3a7c0]
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      Writing a core file
      ----------SERVER LOG END-------------
       
       
       - saving '/home/buildbot/maria-slave/power8-vlp06-bintar/build/mysql-test/var/2/log/innodb.innodb_scrub_background-ecb,xtradb/' to '/home/buildbot/maria-slave/power8-vlp06-bintar/build/mysql-test/var/log/innodb.innodb_scrub_background-ecb,xtradb/'
       - found 'core.12911' (0/5)
       
      Trying 'dbx' to get a backtrace
      worker[2] > Restart  - not started
       
      Trying 'gdb' to get a backtrace
      worker[2] > Restart [mysqld.1 - pid: 14289, winpid: 14289] - running with different options '--binlog-format=statement' != '--auto-increment-increment=1 --loose-thread-pool-size=4'
      Core generated by '/home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld'
      worker[1] > Restart [mysqld.1 - pid: 14249, winpid: 14249] - running with different options '--innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-tables --default-storage-engine=MyISAM' != '--innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-tables --loose-enable-partition --loose-innodb-lock-wait-timeout=2 --default-storage-engine=MyISAM'
      Output from gdb follows. The first stack trace is from the failing thread.
      The following stack traces are from all threads (so the failing one is
      duplicated).
      --------------------------
      [New LWP 12953]
      [New LWP 12976]
      [New LWP 12955]
      [New LWP 12954]
      [New LWP 12956]
      [New LWP 12914]
      [New LWP 12958]
      [New LWP 12911]
      [New LWP 12957]
      [New LWP 12923]
      [New LWP 12945]
      [New LWP 12915]
      [New LWP 12946]
      [New LWP 12943]
      [New LWP 12922]
      [New LWP 12944]
      [New LWP 12947]
      [New LWP 12952]
      [New LWP 12920]
      [New LWP 12950]
      [New LWP 12924]
      [New LWP 12951]
      [New LWP 12949]
      [New LWP 12925]
      [New LWP 12921]
      [New LWP 12948]
       
      warning: Could not load shared library symbols for linux-vdso64.so.1.
      Do you need "set solib-search-path" or "set sysroot"?
       
      warning: File "/opt/at8.0-3-rc1/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".
      To enable execution of this file add
      	add-auto-load-safe-path /opt/at8.0-3-rc1/lib64/power8/libthread_db-1.0.so
      line to your configuration file "/home/buildbot/.gdbinit".
      To completely disable this security protection add
      	set auto-load safe-path /
      line to your configuration file "/home/buildbot/.gdbinit".
      For more information about this security protection see the
      "Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
      	info "(gdb)Auto-loading safe path"
       
      warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
       
      warning: File "/opt/at8.0-3-rc1/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".
       
      warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
      Core was generated by `/home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld --defaults-grou'.
      Program terminated with signal 6, Aborted.
      #0  0x00003fff98440ed0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
      60	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
      #0  0x00003fff98440ed0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
      #1  0x0000000036650630 in my_write_core (sig=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/mysys/stacktrace.c:456
      #2  0x000000003605a120 in handle_fatal_signal (sig=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/signal_handler.cc:266
      #3  <signal handler called>
      #4  0x00003fff97c4ee38 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:55
      #5  0x00003fff97c5193c in __GI_abort () at abort.c:89
      #6  0x00000000365a69cc in fil_space_write_crypt_data_low (crypt_data=0x3fff64098b20, type=<optimized out>, page=<optimized out>, offset=0, maxsize=18446744073709551614, mtr=0x3fff77ffe250) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:474
      #7  0x00000000365acaa8 in fil_space_write_crypt_data (mtr=0x3fff77ffe250, maxsize=18446744073709551614, offset=0, page=<optimized out>, space=6) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:541
      #8  fil_crypt_flush_space (space=6, state=0x3fff77ffe190) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2038
      #9  fil_crypt_complete_rotate_space (key_state=0x3fff77ffe158, state=0x3fff77ffe190) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2109
      #10 fil_crypt_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2210
      #11 0x0000000000000000 in ?? ()
       
      Thread 26 (LWP 12948):
      #0  0x00003fff984448e8 in fsync () at ../sysdeps/unix/syscall-template.S:81
      #1  0x000000003641e784 in os_file_fsync (file=6) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:2686
      #2  os_file_flush_func (file=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:2786
      #3  0x000000003659b6c8 in pfs_os_file_flush_func (src_file=0x367bb288 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc", src_line=6106, file=6) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/include/os0file.ic:386
      #4  fil_flush (space_id=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:6106
      #5  0x0000000036536418 in buf_dblwr_flush_buffered_writes () at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0dblwr.cc:1022
      #6  0x0000000036542de4 in buf_flush_common (flush_type=BUF_FLUSH_LIST, page_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0flu.cc:1851
      #7  buf_flush_list (min_n=<optimized out>, n_processed=0x3fff89ffe6c0, lsn_limit=18446744073709551615) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0flu.cc:2134
      #8  0x0000000036546bc8 in page_cleaner_do_flush_batch (lsn_limit=18446744073709551615, n_to_flush=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0flu.cc:2418
      #9  buf_flush_page_cleaner_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0flu.cc:2790
      #10 0x00003fff98437cac in start_thread (arg=0x3fff89fff170) at pthread_create.c:312
      #11 0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 25 (LWP 12921):
      #0  0x00003fff98390738 in __io_getevents_0_4 () from /lib64/libaio.so.1
      #1  0x00003fff983907a0 in io_getevents () from /lib64/libaio.so.1
      #2  0x0000000036423c58 in os_aio_linux_collect (seg_size=<optimized out>, segment=<optimized out>, array=0x1003aece440) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5589
      #3  os_aio_linux_handle (global_seg=1, message1=0x3fff90dce720, message2=0x3fff90dce728, type=0x3fff90dce730, space_id=0x3fff90dce738) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5780
      #4  0x0000000036597e3c in fil_aio_wait (segment=1) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:5959
      #5  0x00000000364aa080 in io_handler_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0start.cc:534
      #6  0x00003fff98437cac in start_thread (arg=0x3fff90dcf170) at pthread_create.c:312
      #7  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 24 (LWP 12925):
      #0  0x00003fff98390738 in __io_getevents_0_4 () from /lib64/libaio.so.1
      #1  0x00003fff983907a0 in io_getevents () from /lib64/libaio.so.1
      #2  0x0000000036423c58 in os_aio_linux_collect (seg_size=<optimized out>, segment=<optimized out>, array=0x1003aefa9a0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5589
      #3  os_aio_linux_handle (global_seg=5, message1=0x3fff8a7fe720, message2=0x3fff8a7fe728, type=0x3fff8a7fe730, space_id=0x3fff8a7fe738) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5780
      #4  0x0000000036597e3c in fil_aio_wait (segment=5) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:5959
      #5  0x00000000364aa080 in io_handler_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0start.cc:534
      #6  0x00003fff98437cac in start_thread (arg=0x3fff8a7ff170) at pthread_create.c:312
      #7  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 23 (LWP 12949):
      #0  0x00003fff97d2d314 in select () at ../sysdeps/unix/syscall-template.S:81
      #1  0x000000003642778c in os_thread_sleep (tm=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0thread.cc:285
      #2  0x0000000036547878 in page_cleaner_sleep_if_needed (next_loop_time=1427645801653) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0flu.cc:2648
      #3  buf_flush_lru_manager_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0flu.cc:2918
      #4  0x00003fff98437cac in start_thread (arg=0x3fff7efff170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 22 (LWP 12951):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003b4003f0, mutex=0x1003b4003b0, abstime=0x3fff7dffe620) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff7dffe620, fast_mutex=0x1003b4003b0, cond=0x1003b4003f0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003b4003b0, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x000000003658db44 in dict_stats_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/dict/dict0stats_bg.cc:546
      #4  0x00003fff98437cac in start_thread (arg=0x3fff7dfff170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 21 (LWP 12924):
      #0  0x00003fff98390738 in __io_getevents_0_4 () from /lib64/libaio.so.1
      #1  0x00003fff983907a0 in io_getevents () from /lib64/libaio.so.1
      #2  0x0000000036423c58 in os_aio_linux_collect (seg_size=<optimized out>, segment=<optimized out>, array=0x1003aefa9a0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5589
      #3  os_aio_linux_handle (global_seg=4, message1=0x3fff8affe720, message2=0x3fff8affe728, type=0x3fff8affe730, space_id=0x3fff8affe738) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5780
      #4  0x0000000036597e3c in fil_aio_wait (segment=4) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:5959
      #5  0x00000000364aa080 in io_handler_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0start.cc:534
      #6  0x00003fff98437cac in start_thread (arg=0x3fff8afff170) at pthread_create.c:312
      #7  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 20 (LWP 12950):
      #0  0x00003fff9843f7b0 in __pthread_cond_wait (cond=0x1003ac7a860, mutex=0x1003ac7a820) at pthread_cond_wait.c:188
      #1  0x0000000036425b38 in os_cond_wait (fast_mutex=0x1003ac7a820, cond=0x1003ac7a860) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:214
      #2  os_event_wait_low (event=0x1003ac7a820, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:610
      #3  0x000000003653c030 in buf_dump_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0dump.cc:594
      #4  0x00003fff98437cac in start_thread (arg=0x3fff7e7ff170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 19 (LWP 12920):
      #0  0x00003fff98390738 in __io_getevents_0_4 () from /lib64/libaio.so.1
      #1  0x00003fff983907a0 in io_getevents () from /lib64/libaio.so.1
      #2  0x0000000036423c58 in os_aio_linux_collect (seg_size=<optimized out>, segment=<optimized out>, array=0x1003aee46e0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5589
      #3  os_aio_linux_handle (global_seg=0, message1=0x3fff915ce720, message2=0x3fff915ce728, type=0x3fff915ce730, space_id=0x3fff915ce738) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5780
      #4  0x0000000036597e3c in fil_aio_wait (segment=0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:5959
      #5  0x00000000364aa080 in io_handler_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0start.cc:534
      #6  0x00003fff98437cac in start_thread (arg=0x3fff915cf170) at pthread_create.c:312
      #7  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 18 (LWP 12952):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003b1eaf60, mutex=0x1003b1eaf20, abstime=0x3fff7d7fe570) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff7d7fe570, fast_mutex=0x1003b1eaf20, cond=0x1003b1eaf60) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003b1eaf20, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000364e4550 in ib_wqueue_timedwait (wq=0x1003b1ead78, wait_in_usecs=5000000) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/ut/ut0wqueue.cc:154
      #4  0x00000000365e0b78 in fts_optimize_thread (arg=0x1003b1ead78) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fts/fts0opt.cc:3003
      #5  0x00003fff98437cac in start_thread (arg=0x3fff7d7ff170) at pthread_create.c:312
      #6  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 17 (LWP 12947):
      #0  0x00003fff9843f7b0 in __pthread_cond_wait (cond=0x1003b406530, mutex=0x1003b4064f0) at pthread_cond_wait.c:188
      #1  0x0000000036425b38 in os_cond_wait (fast_mutex=0x1003b4064f0, cond=0x1003b406530) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:214
      #2  os_event_wait_low (event=0x1003b4064f0, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:610
      #3  0x00000000364af920 in sync_array_wait_event (arr=0x1003ad8dc70, index=0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/sync/sync0arr.cc:474
      #4  0x00000000364b2348 in rw_lock_s_lock_spin (_lock=0x37430b18 <dict_operation_lock>, pass=<optimized out>, priority_lock=<optimized out>, high_priority=<optimized out>, file_name=0x367a3848 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0purge.cc", line=754) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/sync/sync0rw.cc:551
      #5  0x0000000036489030 in rw_lock_s_lock_func (lock=<optimized out>, pass=0, file_name=0x367a3848 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0purge.cc", line=754) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/include/sync0rw.ic:432
      #6  pfs_rw_lock_s_lock_func (lock=<optimized out>, pass=0, file_name=0x367a3848 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0purge.cc", line=754) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/include/sync0rw.ic:1052
      #7  row_purge_parse_undo_rec (thr=<optimized out>, updated_extern=0x3fff7f7fe36f, undo_rec=0x3fff800086e8 "\002\022\016\005\023", node=0x1003b634438) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0purge.cc:754
      #8  row_purge (thr=0x1003b631e80, undo_rec=0x3fff800086e8 "\002\022\016\005\023", node=0x1003b634438) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0purge.cc:893
      #9  row_purge_step (thr=0x1003b631e80) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0purge.cc:976
      #10 0x0000000036449274 in que_thr_step (thr=0x1003b631e80) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/que/que0que.cc:1115
      #11 que_run_threads_low (thr=0x1003b631e80) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/que/que0que.cc:1177
      #12 que_run_threads (thr=0x1003b631e80) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/que/que0que.cc:1218
      #13 0x00000000364bd048 in trx_purge (n_purge_threads=1, batch_size=300, truncate=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/trx/trx0purge.cc:1251
      #14 0x00000000364a9394 in srv_do_purge (n_total_purged=<synthetic pointer>, n_threads=1) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:3428
      #15 srv_purge_coordinator_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:3612
      #16 0x00003fff98437cac in start_thread (arg=0x3fff7f7ff170) at pthread_create.c:312
      #17 0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 16 (LWP 12944):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003ac7a740, mutex=0x1003ac7a700, abstime=0x3fff88ffe590) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff88ffe590, fast_mutex=0x1003ac7a700, cond=0x1003ac7a740) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003ac7a700, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000364a5644 in srv_error_monitor_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:2388
      #4  0x00003fff98437cac in start_thread (arg=0x3fff88fff170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 15 (LWP 12922):
      #0  0x00003fff98390738 in __io_getevents_0_4 () from /lib64/libaio.so.1
      #1  0x00003fff983907a0 in io_getevents () from /lib64/libaio.so.1
      #2  0x0000000036423c58 in os_aio_linux_collect (seg_size=<optimized out>, segment=<optimized out>, array=0x1003ad8db70) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5589
      #3  os_aio_linux_handle (global_seg=2, message1=0x3fff8bffe720, message2=0x3fff8bffe728, type=0x3fff8bffe730, space_id=0x3fff8bffe738) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5780
      #4  0x0000000036597e3c in fil_aio_wait (segment=2) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:5959
      #5  0x00000000364aa080 in io_handler_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0start.cc:534
      #6  0x00003fff98437cac in start_thread (arg=0x3fff8bfff170) at pthread_create.c:312
      #7  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 14 (LWP 12943):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003b1d3440, mutex=0x1003b1d3400, abstime=0x3fff897fe630) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff897fe630, fast_mutex=0x1003b1d3400, cond=0x1003b1d3440) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003b1d3400, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000363ef4e4 in lock_wait_timeout_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/lock/lock0wait.cc:534
      #4  0x00003fff98437cac in start_thread (arg=0x3fff897ff170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 13 (LWP 12946):
      #0  0x00003fff9843f7b0 in __pthread_cond_wait (cond=0x1003b406530, mutex=0x1003b4064f0) at pthread_cond_wait.c:188
      #1  0x0000000036425b38 in os_cond_wait (fast_mutex=0x1003b4064f0, cond=0x1003b406530) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:214
      #2  os_event_wait_low (event=0x1003b4064f0, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:610
      #3  0x00000000364af920 in sync_array_wait_event (arr=0x1003ad8dc70, index=2) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/sync/sync0arr.cc:474
      #4  0x00000000364b2a84 in rw_lock_x_lock_func (lock=0x37430b18 <dict_operation_lock>, pass=0, file_name=0x367a9a60 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc", line=2814, priority_lock=<optimized out>, high_priority=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/sync/sync0rw.cc:908
      #5  0x00000000364a08e4 in pfs_rw_lock_x_lock_func (lock=<optimized out>, pass=0, file_name=0x367a9a60 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc", line=2814) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/include/sync0rw.ic:899
      #6  srv_master_evict_from_table_cache (pct_check=100) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:2814
      #7  0x00000000364a7b44 in srv_master_do_idle_tasks () at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:3014
      #8  srv_master_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:3178
      #9  0x00003fff98437cac in start_thread (arg=0x3fff7ffff170) at pthread_create.c:312
      #10 0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 12 (LWP 12915):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x373cba08 <COND_checkpoint>, mutex=0x373cb9d0 <LOCK_checkpoint>, abstime=0x3fff9267e598) at pthread_cond_timedwait.c:199
      #1  0x000000003626caec in inline_mysql_cond_timedwait (src_file=0x36773a00 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/maria/ma_servicethread.c", src_line=119, abstime=<optimized out>, mutex=0x373cb9d0 <LOCK_checkpoint>, that=0x373cba08 <COND_checkpoint>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/include/mysql/psi/mysql_thread.h:1191
      #2  my_service_thread_sleep (control=0x36b26dc8 <checkpoint_control>, sleep_time=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/maria/ma_servicethread.c:118
      #3  0x0000000036260e34 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/maria/ma_checkpoint.c:709
      #4  0x0000000036379240 in pfs_spawn_thread (arg=0x1003adab478) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/perfschema/pfs.cc:1860
      #5  0x00003fff98437cac in start_thread (arg=0x3fff9267f170) at pthread_create.c:312
      #6  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 11 (LWP 12945):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003ac7a7d0, mutex=0x1003ac7a790, abstime=0x3fff887fe600) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff887fe600, fast_mutex=0x1003ac7a790, cond=0x1003ac7a7d0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003ac7a790, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000364a4e8c in srv_monitor_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0srv.cc:2111
      #4  0x00003fff98437cac in start_thread (arg=0x3fff887ff170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 10 (LWP 12923):
      #0  0x00003fff98390738 in __io_getevents_0_4 () from /lib64/libaio.so.1
      #1  0x00003fff983907a0 in io_getevents () from /lib64/libaio.so.1
      #2  0x0000000036423c58 in os_aio_linux_collect (seg_size=<optimized out>, segment=<optimized out>, array=0x1003ad8db70) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5589
      #3  os_aio_linux_handle (global_seg=3, message1=0x3fff8b7fe720, message2=0x3fff8b7fe728, type=0x3fff8b7fe730, space_id=0x3fff8b7fe738) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0file.cc:5780
      #4  0x0000000036597e3c in fil_aio_wait (segment=3) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0fil.cc:5959
      #5  0x00000000364aa080 in io_handler_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/srv/srv0start.cc:534
      #6  0x00003fff98437cac in start_thread (arg=0x3fff8b7ff170) at pthread_create.c:312
      #7  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 9 (LWP 12957):
      #0  0x00003fff97d2d314 in select () at ../sysdeps/unix/syscall-template.S:81
      #1  0x000000003642778c in os_thread_sleep (tm=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0thread.cc:285
      #2  0x0000000036521b4c in btr_defragment_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/btr/btr0defragment.cc:743
      #3  0x00003fff98437cac in start_thread (arg=0x3fff76fff170) at pthread_create.c:312
      #4  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 8 (LWP 12911):
      #0  0x00003fff97d2ab38 in poll () at ../sysdeps/unix/syscall-template.S:81
      #1  0x00003fff97d53764 in __poll_chk (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>, fdslen=<optimized out>) at poll_chk.c:27
      #2  0x0000000035dc0e58 in poll (__timeout=-1, __nfds=2, __fds=<optimized out>) at /opt/at8.0-3-rc1/include/bits/poll2.h:41
      #3  handle_connections_sockets () at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld.cc:6374
      #4  0x0000000035dcb19c in mysqld_main (argc=153, argv=0x1003ac795e8) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld.cc:5802
      #5  0x0000000035da68b0 in main (argc=<optimized out>, argv=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/main.cc:25
       
      Thread 7 (LWP 12958):
      #0  0x00003fff984456f4 in do_sigwait (sig=0x3fff90536620, set=<optimized out>) at ../sysdeps/unix/sysv/linux/sigwait.c:61
      #1  __sigwait (set=<optimized out>, sig=0x3fff90536620) at ../sysdeps/unix/sysv/linux/sigwait.c:98
      #2  0x0000000035dbe28c in signal_hand (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/mysqld.cc:3443
      #3  0x0000000036379240 in pfs_spawn_thread (arg=0x1003b20dc58) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/perfschema/pfs.cc:1860
      #4  0x00003fff98437cac in start_thread (arg=0x3fff90537170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 6 (LWP 12914):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x3743a058 <COND_timer>, mutex=0x37439ff0 <LOCK_timer>, abstime=0x3fff93a4e698) at pthread_cond_timedwait.c:199
      #1  0x0000000036657088 in inline_mysql_cond_timedwait (that=0x3743a058 <COND_timer>, mutex=0x37439ff0 <LOCK_timer>, src_file=0x367cbfc0 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/mysys/thr_timer.c", src_line=292, abstime=0x3fff93a4e698) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/include/mysql/psi/mysql_thread.h:1191
      #2  timer_handler (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/mysys/thr_timer.c:292
      #3  0x0000000036379240 in pfs_spawn_thread (arg=0x1003ad7f918) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/perfschema/pfs.cc:1860
      #4  0x00003fff98437cac in start_thread (arg=0x3fff93a4f170) at pthread_create.c:312
      #5  0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 5 (LWP 12956):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003b1e80a0, mutex=0x1003b1e8060, abstime=0x3fff777fdff0) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff777fdff0, fast_mutex=0x1003b1e8060, cond=0x1003b1e80a0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003b1e8060, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000365ace7c in fil_crypt_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2163
      #4  0x0000000000000000 in ?? ()
       
      Thread 4 (LWP 12954):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003b1e80a0, mutex=0x1003b1e8060, abstime=0x3fff7cffdff0) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff7cffdff0, fast_mutex=0x1003b1e8060, cond=0x1003b1e80a0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003b1e8060, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000365ace7c in fil_crypt_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2163
      #4  0x0000000000000000 in ?? ()
       
      Thread 3 (LWP 12955):
      #0  0x00003fff9843fca0 in __pthread_cond_timedwait (cond=0x1003b1e80a0, mutex=0x1003b1e8060, abstime=0x3fff7c7fdff0) at pthread_cond_timedwait.c:199
      #1  0x0000000036425d38 in os_cond_wait_timed (abstime=0x3fff7c7fdff0, fast_mutex=0x1003b1e8060, cond=0x1003b1e80a0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:177
      #2  os_event_wait_time_low (event=0x1003b1e8060, time_in_usec=<optimized out>, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:723
      #3  0x00000000365ace7c in fil_crypt_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2163
      #4  0x0000000000000000 in ?? ()
       
      Thread 2 (LWP 12976):
      #0  0x00003fff9843f7b0 in __pthread_cond_wait (cond=0x1003b01ef20, mutex=0x1003b01eee0) at pthread_cond_wait.c:188
      #1  0x0000000036425b38 in os_cond_wait (fast_mutex=0x1003b01eee0, cond=0x1003b01ef20) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:214
      #2  os_event_wait_low (event=0x1003b01eee0, reset_sig_count=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/os/os0sync.cc:610
      #3  0x00000000364af920 in sync_array_wait_event (arr=0x1003ad8dc70, index=1) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/sync/sync0arr.cc:474
      #4  0x00000000364b2a84 in rw_lock_x_lock_func (lock=0x3fff915ff320, pass=0, file_name=0x367bea48 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fsp/fsp0fsp.cc", line=735, priority_lock=<optimized out>, high_priority=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/sync/sync0rw.cc:908
      #5  0x000000003652bc60 in pfs_rw_lock_x_lock_func (line=735, file_name=0x367bea48 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fsp/fsp0fsp.cc", pass=0, lock=0x3fff915ff320) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/include/sync0rw.ic:907
      #6  buf_page_get_gen (space=6, zip_size=0, offset=0, rw_latch=2, guess=<optimized out>, mode=10, file=0x367bea48 "/home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fsp/fsp0fsp.cc", line=735, mtr=0x3fff904d0660) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/buf/buf0buf.cc:3199
      #7  0x00000000365bff24 in fsp_header_init (space=6, size=4, mtr=0x3fff904d0660) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fsp/fsp0fsp.cc:735
      #8  0x000000003655c134 in dict_build_table_def_step (thr=0x3fff640336e0, node=0x3fff64033058) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/dict/dict0crea.cc:323
      #9  0x000000003655d4e4 in dict_create_table_step (thr=0x3fff640336e0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/dict/dict0crea.cc:1044
      #10 0x0000000036449488 in que_thr_step (thr=0x3fff640336e0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/que/que0que.cc:1123
      #11 que_run_threads_low (thr=0x3fff640336e0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/que/que0que.cc:1177
      #12 que_run_threads (thr=0x3fff640336e0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/que/que0que.cc:1218
      #13 0x0000000036479388 in row_create_table_for_mysql (table=0x3fff640323b8, trx=0x3fff64031948, commit=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/row/row0mysql.cc:2352
      #14 0x00000000363ac31c in create_table_def (flags2=<optimized out>, flags=33, remote_path=0x3fff904d12c8 "", temp_path=0x3fff904d10c8 "", table_name=0x3fff904d0ec8 "test/t3", form=0x3fff904d18b8, trx=0x3fff64031948) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/handler/ha_innodb.cc:10976
      #15 ha_innobase::create (this=<optimized out>, name=<optimized out>, form=0x3fff904d18b8, create_info=0x3fff904d44d8) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/handler/ha_innodb.cc:12081
      #16 0x00000000360655dc in handler::ha_create (this=0x3fff640309a0, name=<optimized out>, form=0x3fff904d18b8, info=0x3fff904d44d8) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/handler.cc:4320
      #17 0x00000000360662a8 in ha_create_table (thd=0x1003b27fef8, path=<optimized out>, db=0x3fff6400b9e0 "test", table_name=0x3fff6400b3c8 "t3", create_info=0x3fff904d44d8, frm=0x3fff904d4060) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/handler.cc:4689
      #18 0x0000000035f5098c in rea_create_table (thd=0x1003b27fef8, frm=0x3fff904d4060, path=0x3fff904d4070 "./test/t3", db=0x3fff6400b9e0 "test", table_name=0x3fff6400b3c8 "t3", create_info=0x3fff904d44d8, file=0x3fff6400c408, no_ha_create_table=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/unireg.cc:394
      #19 0x0000000035f1852c in create_table_impl (thd=0x1003b27fef8, orig_db=0x3fff6400b9e0 "test", orig_table_name=0x3fff6400b3c8 "t3", db=0x3fff6400b9e0 "test", table_name=0x3fff6400b3c8 "t3", path=0x3fff904d4070 "./test/t3", options=..., create_info=0x3fff904d44d8, alter_info=0x3fff904d4448, create_table_mode=0, is_trans=0x3fff904d42f7, key_info=0x3fff904d4058, key_count=0x3fff904d4054, frm=0x3fff904d4060) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_table.cc:4804
      #20 0x0000000035f18bf8 in mysql_create_table_no_lock (thd=0x1003b27fef8, db=0x3fff6400b9e0 "test", table_name=0x3fff6400b3c8 "t3", create_info=0x3fff904d44d8, alter_info=0x3fff904d4448, is_trans=0x3fff904d42f7, create_table_mode=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_table.cc:4919
      #21 0x0000000035f18de4 in mysql_create_table (thd=0x1003b27fef8, create_table=<optimized out>, create_info=0x3fff904d44d8, alter_info=0x3fff904d4448) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_table.cc:4980
      #22 0x0000000035e806b8 in mysql_execute_command (thd=0x1003b27fef8) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_parse.cc:3409
      #23 0x0000000035e81548 in mysql_parse (thd=0x1003b27fef8, rawbuf=0x3fff6400b260 "create table t3 (\na int auto_increment primary key,\nb varchar(256),\nc text, index(b)) engine = innodb row_format=dynamic", length=<optimized out>, parser_state=0x3fff904d59e0) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_parse.cc:7165
      #24 0x0000000035e84438 in dispatch_command (command=<optimized out>, thd=<optimized out>, packet=0x1003b285449 "create table t3 (\na int auto_increment primary key,\nb varchar(256),\nc text, index(b)) engine = innodb row_format=dynamic", packet_length=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_parse.cc:1462
      #25 0x0000000035e84b88 in do_command (thd=0x1003b27fef8) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_parse.cc:1090
      #26 0x0000000035f71664 in do_handle_one_connection (thd_arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_connect.cc:1347
      #27 0x0000000035f71924 in handle_one_connection (arg=0x1003b27fef8) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/sql_connect.cc:1258
      #28 0x0000000036379240 in pfs_spawn_thread (arg=0x1003b212798) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/perfschema/pfs.cc:1860
      #29 0x00003fff98437cac in start_thread (arg=0x3fff904d7170) at pthread_create.c:312
      #30 0x00003fff97d3a7c0 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
       
      Thread 1 (LWP 12953):
      #0  0x00003fff98440ed0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
      #1  0x0000000036650630 in my_write_core (sig=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/mysys/stacktrace.c:456
      #2  0x000000003605a120 in handle_fatal_signal (sig=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/sql/signal_handler.cc:266
      #3  <signal handler called>
      #4  0x00003fff97c4ee38 in __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:55
      #5  0x00003fff97c5193c in __GI_abort () at abort.c:89
      #6  0x00000000365a69cc in fil_space_write_crypt_data_low (crypt_data=0x3fff64098b20, type=<optimized out>, page=<optimized out>, offset=0, maxsize=18446744073709551614, mtr=0x3fff77ffe250) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:474
      #7  0x00000000365acaa8 in fil_space_write_crypt_data (mtr=0x3fff77ffe250, maxsize=18446744073709551614, offset=0, page=<optimized out>, space=6) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:541
      #8  fil_crypt_flush_space (space=6, state=0x3fff77ffe190) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2038
      #9  fil_crypt_complete_rotate_space (key_state=0x3fff77ffe158, state=0x3fff77ffe190) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2109
      #10 fil_crypt_thread (arg=<optimized out>) at /home/buildbot/maria-slave/power8-vlp06-bintar/build/storage/xtradb/fil/fil0crypt.cc:2210
      #11 0x0000000000000000 in ?? ()
       
      Compressed file /home/buildbot/maria-slave/power8-vlp06-bintar/build/mysql-test/var/log/innodb.innodb_scrub_background-ecb,xtradb/mysqld.1/data/core.12911
      main.partition_innodb_semi_consistent 'xtradb' w1 [ pass ]  30794
      ***Warnings generated in error logs during shutdown after running tests: innodb.innodb_scrub_background
       
      Attempting backtrace. You can use the following information to find out

      Attachments

        Issue Links

          Activity

            People

              jplindst Jan Lindström (Inactive)
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.