[MDEV-12742] rpl.rpl_binlog_errors, binlog_encryption.rpl_binlog_errors crash sporadically in buildbot Created: 2017-05-09  Updated: 2023-11-29  Resolved: 2023-11-29

Status: Closed
Project: MariaDB Server
Component/s: Replication, Tests
Affects Version/s: 10.2
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Andrei Elkin
Resolution: Won't Fix Votes: 0
Labels: None


 Description   

Most recent occurrence:
http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/7944/steps/test_2/logs/mysqld.1.err.1

Earlier occurrence, but with better stack trace:
http://buildbot.askmonty.org/buildbot/builders/p8-rhel71-bintar-debug/builds/1834/steps/test/logs/stdio

rpl.rpl_binlog_errors 'innodb,stmt'      w1 [ fail ]
        Test ended at 2017-04-22 09:36:55
 
CURRENT_TEST: rpl.rpl_binlog_errors
mysqltest: In included file "./extra/rpl_tests/rpl_binlog_errors.inc": 
included from /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/suite/rpl/t/rpl_binlog_errors.test at line 2:
At line 291: query 'RESET MASTER' failed: 2013: Lost connection to MySQL server during query
 
The result from queries just before the failure was:
< snip >
count(*)
4
SELECT count(*) FROM t4;
count(*)
4
DELETE FROM t2;
DELETE FROM t4;
# assert: must show zero entries
SELECT count(*) FROM t2;
count(*)
0
SELECT count(*) FROM t4;
count(*)
0
SET SQL_LOG_BIN=1;
SET GLOBAL debug_dbug=@old_debug;
###################### TEST #10
call mtr.add_suppression("MYSQL_BIN_LOG::open failed to sync the index file.");
call mtr.add_suppression("Could not open .*");
RESET MASTER;
 
More results from queries before failure can be found in /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/1/log/rpl_binlog_errors.log
 
 
Server [mysqld.1 - pid: 27816, winpid: 27816, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
2017-04-22  9:36:50 70366577066192 [Note] /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld (mysqld 10.2.6-MariaDB-debug-log) starting as process 27817 ...
2017-04-22  9:36:50 70366577066192 [Warning] Could not increase number of max_open_files to more than 1024 (request: 4162)
2017-04-22  9:36:50 70366577066192 [Note] Plugin 'SEQUENCE' is disabled.
2017-04-22  9:36:50 70366577066192 [Note] Plugin 'partition' is disabled.
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Uses event mutexes
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Compressed tables use zlib 1.2.8
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Using Linux native AIO
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Number of pools: 1
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Using POWER8 crc32 instructions
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Initializing buffer pool, total size = 8M, instances = 1, chunk size = 8M
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Completed initialization of buffer pool
2017-04-22  9:36:50 70366345032064 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2017-04-22  9:36:50 70366577066192 [Note] InnoDB: Highest supported file format is Barracuda.
2017-04-22  9:36:51 70366577066192 [Note] InnoDB: Creating shared tablespace for temporary tables
2017-04-22  9:36:51 70366577066192 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2017-04-22  9:36:51 70366577066192 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2017-04-22  9:36:52 70366577066192 [Note] InnoDB: 128 out of 128 rollback segments are active.
2017-04-22  9:36:52 70366577066192 [Note] InnoDB: Waiting for purge to start
2017-04-22  9:36:52 70366577066192 [Note] InnoDB: 5.7.14 started; log sequence number 1634127
2017-04-22  9:36:52 70366202425728 [Note] InnoDB: Loading buffer pool(s) from /dev/shm/var_auto_z6P_/1/mysqld.1/data/ib_buffer_pool
2017-04-22  9:36:52 70366202425728 [Note] InnoDB: Buffer pool(s) load completed at 170422  9:36:52
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_SYS_SEMAPHORE_WAITS' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_CMP' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_TABLESPACES_ENCRYPTION' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'FEEDBACK' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_MUTEXES' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
2017-04-22  9:36:52 70366577066192 [Note] Plugin 'INNODB_TABLESPACES_SCRUBBING' is disabled.
2017-04-22  9:36:52 70366577066192 [Warning] /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld: unknown option '--loose-pam-debug'
2017-04-22  9:36:52 70366577066192 [Note] Server socket created on IP: '127.0.0.1'.
2017-04-22  9:36:52 70366577066192 [Note] Reading of all Master_info entries succeded
2017-04-22  9:36:52 70366577066192 [Note] Added new Master_info '' to hash table
2017-04-22  9:36:52 70366577066192 [Note] /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld: ready for connections.
Version: '10.2.6-MariaDB-debug-log'  socket: '/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/tmp/1/mysqld.1.sock'  port: 16020  Source distribution
2017-04-22  9:36:52 70366467359104 [Note] Deleted Master_info file '/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/1/mysqld.1/data/master.info'.
2017-04-22  9:36:52 70366467359104 [Note] Deleted Master_info file '/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/1/mysqld.1/data/relay-log.info'.
2017-04-22  9:36:52 70366464016768 [Note] Start binlog_dump to slave_server(2), pos(master-bin.000001, 4)
2017-04-22  9:36:52 70366464016768 [Warning] Aborted connection 13 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error writing communication packets)
2017-04-22  9:36:52 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:53 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:53 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 0 "Internal error/check (Not system error)")
2017-04-22  9:36:53 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:53 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 11 "Resource temporarily unavailable")
2017-04-22  9:36:54 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:54 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 0 "Internal error/check (Not system error)")
2017-04-22  9:36:54 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:54 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 0 "Internal error/check (Not system error)")
2017-04-22  9:36:54 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:54 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 0 "Internal error/check (Not system error)")
2017-04-22  9:36:54 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:54 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 11 "Resource temporarily unavailable")
2017-04-22  9:36:54 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:54 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 11 "Resource temporarily unavailable")
2017-04-22  9:36:54 70366463623552 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-04-22  9:36:54 70366463623552 [ERROR] mysqld: Error writing file 'master-bin' (errno: 11 "Resource temporarily unavailable")
170422  9:36:54 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.2.6-MariaDB-debug-log
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=6
max_threads=153
thread_count=13
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63061 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0x3fff50000b00
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x3fff68ffe6e8 thread_stack 0x49000
/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld(my_print_stacktrace+0x3c)[0x5be78ea4]
/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld(handle_fatal_signal+0x504)[0x5b3c98d0]
linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fff7ecd0478]
/opt/at8.0/lib64/power8/libc.so.6(+0xf0508)[0x3fff7e4c0508]
/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld(+0xd70410)[0x5b7e0410]
/opt/at8.0/lib64/power8/libc.so.6(fnmatch+0xa0)[0x3fff7e4c1cf0]
/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld(+0x1441e20)[0x5beb1e20]
mysys/stacktrace.c:267(my_print_stacktrace)[0x5beb25bc]
mtr/mtr0mtr.cc:575(mtr_t::commit())[0x5beb12c4]
que/que0que.cc:992(que_thr_step)[0x5b83b674]
que/que0que.cc:1114(que_run_threads_low)[0x5b83bc90]
que/que0que.cc:1156(que_run_threads(que_thr_t*))[0x5b83bf00]
srv/srv0srv.cc:2656(srv_task_execute)[0x5b944f4c]
srv/srv0srv.cc:2705(srv_worker_thread)[0x5b9451b0]
/opt/at8.0/lib64/power8/libpthread.so.0(+0x7cec)[0x3fff7ec97cec]
/opt/at8.0/lib64/power8/libc.so.6(clone+0x98)[0x3fff7e501140]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): 
Connection ID (thread ID): 4
Status: NOT_KILLED
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
----------SERVER LOG END-------------
 
 
 - saving '/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/1/log/rpl.rpl_binlog_errors-innodb,stmt/' to '/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/log/rpl.rpl_binlog_errors-innodb,stmt/'
 - found 'core.27817' (0/5)
 
Trying 'dbx' to get a backtrace
 
Trying 'gdb' to get a backtrace from coredump /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysql-test/var/log/rpl.rpl_binlog_errors-innodb,stmt/mysqld.1/data/core.27817
Core generated by '/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld'
Output from gdb follows. The first stack trace is from the failing thread.
The following stack traces are from all threads (so the failing one is
duplicated).
--------------------------
[New LWP 27845]
[New LWP 27854]
[New LWP 27856]
[New LWP 27861]
[New LWP 27857]
[New LWP 27870]
[New LWP 27868]
[New LWP 27859]
[New LWP 27872]
[New LWP 27878]
[New LWP 27817]
[New LWP 27860]
[New LWP 27887]
[New LWP 27838]
[New LWP 27820]
[New LWP 27825]
[New LWP 27883]
[New LWP 27822]
[New LWP 27840]
[New LWP 27827]
[New LWP 27884]
[New LWP 27824]
[New LWP 27839]
[New LWP 27841]
[New LWP 27889]
[New LWP 27828]
[New LWP 27843]
[New LWP 27858]
[New LWP 27826]
[New LWP 27844]
[New LWP 27888]
[New LWP 27823]
[New LWP 27842]
 
warning: Could not load shared library symbols for linux-vdso64.so.1.
Do you need "set solib-search-path" or "set sysroot"?
 
warning: File "/opt/at8.0/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".
To enable execution of this file add
	add-auto-load-safe-path /opt/at8.0/lib64/power8/libthread_db-1.0.so
line to your configuration file "/home/buildbot/.gdbinit".
To completely disable this security protection add
	set auto-load safe-path /
line to your configuration file "/home/buildbot/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
	info "(gdb)Auto-loading safe path"
 
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
 
warning: File "/opt/at8.0/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld --default'.
Program terminated with signal 11, Segmentation fault.
#0  0x00003fff7eca0ef0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
60	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
#0  0x00003fff7eca0ef0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
#1  0x000000005be79044 in my_write_core (sig=11) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/stacktrace.c:477
#2  0x000000005b3c9e58 in handle_fatal_signal (sig=11) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/signal_handler.cc:299
#3  <signal handler called>
#4  internal_fnmatch (pattern=<optimized out>, string=0x5c0feb98 "ib_que", string_end=0x5c0feb9e "", no_leading_period=0, flags=0, ends=0x0, alloca_used=0) at fnmatch_loop.c:62
#5  0x00003fff7e4c1cf0 in __fnmatch (pattern=0xb <Address 0xb out of bounds>, string=0x5c0feb98 "ib_que", flags=<optimized out>) at fnmatch.c:458
#6  0x000000005beb1e20 in InList (linkp=0x2, cp=0x5c0feb98 "ib_que", exact_match=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/dbug/dbug.c:1515
#7  0x000000005beb25bc in _db_keyword_ (cs=0x3fff50000a30, keyword=0x5c0feb98 "ib_que", strict=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/dbug/dbug.c:1717
#8  0x000000005beb12c4 in _db_pargs_ (_line_=994, keyword=0x5c0feb98 "ib_que") at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/dbug/dbug.c:1236
#9  0x000000005b83b674 in que_thr_step (thr=0x10015f61860) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/que/que0que.cc:992
#10 0x000000005b83bc90 in que_run_threads_low (thr=0x10015f61860) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/que/que0que.cc:1114
#11 0x000000005b83bf00 in que_run_threads (thr=0x10015f61860) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/que/que0que.cc:1154
#12 0x000000005b944f4c in srv_task_execute () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2656
#13 0x000000005b9451b0 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2705
#14 0x00003fff7ec97cec in start_thread (arg=0x3fff68fff180) at pthread_create.c:312
#15 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 33 (LWP 27842):
#0  sched_yield () at ../sysdeps/unix/syscall-template.S:82
#1  0x000000005b7fa9b4 in os_thread_yield () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0thread.cc:201
#2  0x000000005b98bbb8 in trx_purge_wait_for_workers_to_complete (purge_sys=0x10015f61370) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/trx/trx0purge.cc:1630
#3  0x000000005b98c150 in trx_purge (n_purge_threads=3, batch_size=300, truncate=false) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/trx/trx0purge.cc:1728
#4  0x000000005b945668 in srv_do_purge (thd=0x3fff60000b00, n_threads=4, n_total_purged=0x3fff6a7fe770) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2814
#5  0x000000005b945e58 in srv_purge_coordinator_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2959
#6  0x00003fff7ec97cec in start_thread (arg=0x3fff6a7ff180) at pthread_create.c:312
#7  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 32 (LWP 27823):
#0  0x00003fff7ebb0738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff7ebb07a0 in io_getevents () from /lib64/libaio.so.1
#2  0x000000005b7e6c6c in LinuxAIOHandler::collect (this=0x3fff737fe620) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:1886
#3  0x000000005b7e726c in LinuxAIOHandler::poll (this=0x3fff737fe620, m1=0x3fff737fe6f8, m2=0x3fff737fe700, request=0x3fff737fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2031
#4  0x000000005b7e7430 in os_aio_linux_handler (global_segment=1, m1=0x3fff737fe6f8, m2=0x3fff737fe700, request=0x3fff737fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2085
#5  0x000000005b7ed6e8 in os_aio_handler (segment=1, m1=0x3fff737fe6f8, m2=0x3fff737fe700, request=0x3fff737fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:5682
#6  0x000000005bb22ff0 in fil_aio_wait (segment=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5423
#7  0x000000005b94ab54 in io_handler_thread (arg=0x5c8ed590 <n+8>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0start.cc:327
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff737ff180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 31 (LWP 27888):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x5c898528 <mysql_bin_log+1880>, mutex=0x5c8984a0 <mysql_bin_log+1744>) at pthread_cond_wait.c:188
#1  0x000000005be80a00 in safe_cond_wait (cond=0x5c898528 <mysql_bin_log+1880>, mp=0x5c898478 <mysql_bin_log+1704>, file=0x5c07ff00 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:491
#2  0x000000005b553d60 in inline_mysql_cond_wait (that=0x5c898528 <mysql_bin_log+1880>, mutex=0x5c898478 <mysql_bin_log+1704>, src_file=0x5c080548 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/log.cc", src_line=4101) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138
#3  0x000000005b56303c in MYSQL_BIN_LOG::reset_logs (this=0x5c897dd0 <mysql_bin_log>, thd=0x3fff14000b00, create_new_log=true, init_state=0x0, init_state_len=0, next_log_number=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/log.cc:4101
#4  0x000000005b0c18c0 in reset_master (thd=0x3fff14000b00, init_state=0x0, init_state_len=0, next_log_number=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_repl.cc:3878
#5  0x000000005b273548 in reload_acl_and_cache (thd=0x3fff14000b00, options=128, tables=0x0, write_to_binlog=0x3fff78116020) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_reload.cc:351
#6  0x000000005b079968 in mysql_execute_command (thd=0x3fff14000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:5372
#7  0x000000005b08220c in mysql_parse (thd=0x3fff14000b00, rawbuf=0x3fff14011118 "RESET MASTER", length=12, parser_state=0x3fff781169e8, is_com_multi=false, is_next_command=false) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:7874
#8  0x000000005b06dbd8 in dispatch_command (command=COM_QUERY, thd=0x3fff14000b00, packet=0x3fff140088a1 "RESET MASTER", packet_length=12, is_com_multi=false, is_next_command=false) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:1812
#9  0x000000005b06c1a4 in do_command (thd=0x3fff14000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:1362
#10 0x000000005b224860 in do_handle_one_connection (connect=0x1001634c8c0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1354
#11 0x000000005b2244bc in handle_one_connection (arg=0x1001634c8c0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1260
#12 0x000000005bdf462c in pfs_spawn_thread (arg=0x10015d3eac0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#13 0x00003fff7ec97cec in start_thread (arg=0x3fff78118180) at pthread_create.c:312
#14 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 30 (LWP 27844):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x10015db6a40, mutex=0x10015db6a18) at pthread_cond_wait.c:188
#1  0x000000005b7fa46c in os_event::wait (this=0x10015db6a00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:166
#2  0x000000005b7f993c in os_event::wait_low (this=0x10015db6a00, reset_sig_count=18) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:336
#3  0x000000005b7f9f98 in os_event_wait_low (event=0x10015db6a00, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:535
#4  0x000000005b93f82c in srv_resume_thread (slot=0x10015db6690, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:927
#5  0x000000005b9451ac in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2703
#6  0x00003fff7ec97cec in start_thread (arg=0x3fff697ff180) at pthread_create.c:312
#7  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 29 (LWP 27826):
#0  0x00003fff7ebb0738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff7ebb07a0 in io_getevents () from /lib64/libaio.so.1
#2  0x000000005b7e6c6c in LinuxAIOHandler::collect (this=0x3fff71ffe620) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:1886
#3  0x000000005b7e726c in LinuxAIOHandler::poll (this=0x3fff71ffe620, m1=0x3fff71ffe6f8, m2=0x3fff71ffe700, request=0x3fff71ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2031
#4  0x000000005b7e7430 in os_aio_linux_handler (global_segment=4, m1=0x3fff71ffe6f8, m2=0x3fff71ffe700, request=0x3fff71ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2085
#5  0x000000005b7ed6e8 in os_aio_handler (segment=4, m1=0x3fff71ffe6f8, m2=0x3fff71ffe700, request=0x3fff71ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:5682
#6  0x000000005bb22ff0 in fil_aio_wait (segment=4) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5423
#7  0x000000005b94ab54 in io_handler_thread (arg=0x5c8ed5a8 <n+32>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0start.cc:327
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff71fff180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 28 (LWP 27858):
#0  0x00003fff7eca4b04 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005b7faa4c in os_thread_sleep (tm=1000000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0thread.cc:220
#2  0x000000005ba46b3c in btr_defragment_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/btr/btr0defragment.cc:759
#3  0x00003fff7ec97cec in start_thread (arg=0x3fff4efff180) at pthread_create.c:312
#4  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 27 (LWP 27843):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x10015db6940, mutex=0x10015db6918) at pthread_cond_wait.c:188
#1  0x000000005b7fa46c in os_event::wait (this=0x10015db6900) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:166
#2  0x000000005b7f993c in os_event::wait_low (this=0x10015db6900, reset_sig_count=23) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:336
#3  0x000000005b7f9f98 in os_event_wait_low (event=0x10015db6900, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:535
#4  0x000000005b93f82c in srv_resume_thread (slot=0x10015db6620, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:927
#5  0x000000005b9451ac in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2703
#6  0x00003fff7ec97cec in start_thread (arg=0x3fff69fff180) at pthread_create.c:312
#7  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 26 (LWP 27828):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x10015db6c40, mutex=0x10015db6c18, abstime=0x3fff70ffe480) at pthread_cond_timedwait.c:199
#1  0x000000005b7f9748 in os_event::timed_wait (this=0x10015db6c00, abstime=0x3fff70ffe480) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:286
#2  0x000000005b7f9b94 in os_event::wait_time_low (this=0x10015db6c00, time_in_usec=991000, reset_sig_count=2) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:413
#3  0x000000005b7f9f40 in os_event_wait_time_low (event=0x10015db6c00, time_in_usec=991000, reset_sig_count=2) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:518
#4  0x000000005ba82dc0 in pc_sleep_if_needed (next_loop_time=1492868215090, sig_count=2) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/buf/buf0flu.cc:2700
#5  0x000000005ba846f8 in buf_flush_page_cleaner_coordinator (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/buf/buf0flu.cc:3202
#6  0x00003fff7ec97cec in start_thread (arg=0x3fff70fff180) at pthread_create.c:312
#7  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 25 (LWP 27889):
#0  0x00003fff7e4f1438 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005bf1b57c in vio_io_wait (vio=0x10015d3eac0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:945
#2  0x000000005bf19180 in vio_socket_io_wait (vio=0x10015d3eac0, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:108
#3  0x000000005bf19404 in vio_read (vio=0x10015d3eac0, buf=0x3fff180088a0 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:184
#4  0x000000005af4f6f8 in my_real_read (net=0x3fff18000dc8, complen=0x3fff780b7100, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:888
#5  0x000000005af50340 in my_net_read_packet_reallen (net=0x3fff18000dc8, read_from_server=1 '\001', reallen=0x3fff780b7188) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1158
#6  0x000000005af502ac in my_net_read_packet (net=0x3fff18000dc8, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1142
#7  0x000000005b06bc04 in do_command (thd=0x3fff18000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:1244
#8  0x000000005b224860 in do_handle_one_connection (connect=0x100162b32a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1354
#9  0x000000005b2244bc in handle_one_connection (arg=0x100162b32a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1260
#10 0x000000005bdf462c in pfs_spawn_thread (arg=0x10015d3ee60) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#11 0x00003fff7ec97cec in start_thread (arg=0x3fff780b8180) at pthread_create.c:312
#12 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 24 (LWP 27841):
#0  0x00003fff7eca4b04 in nanosleep () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005b7faa4c in os_thread_sleep (tm=1000000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0thread.cc:220
#2  0x000000005b9447a4 in srv_master_sleep () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2510
#3  0x000000005b9449c0 in srv_master_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2557
#4  0x00003fff7ec97cec in start_thread (arg=0x3fff6afff180) at pthread_create.c:312
#5  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 23 (LWP 27839):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x10015db6ac0, mutex=0x10015db6a98, abstime=0x3fff6bffe500) at pthread_cond_timedwait.c:199
#1  0x000000005b7f9748 in os_event::timed_wait (this=0x10015db6a80, abstime=0x3fff6bffe500) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:286
#2  0x000000005b7f9b94 in os_event::wait_time_low (this=0x10015db6a80, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:413
#3  0x000000005b7f9f40 in os_event_wait_time_low (event=0x10015db6a80, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:518
#4  0x000000005b942c30 in srv_error_monitor_thread () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1973
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff6bfff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 22 (LWP 27824):
#0  0x00003fff7ebb0738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff7ebb07a0 in io_getevents () from /lib64/libaio.so.1
#2  0x000000005b7e6c6c in LinuxAIOHandler::collect (this=0x3fff72ffe620) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:1886
#3  0x000000005b7e726c in LinuxAIOHandler::poll (this=0x3fff72ffe620, m1=0x3fff72ffe6f8, m2=0x3fff72ffe700, request=0x3fff72ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2031
#4  0x000000005b7e7430 in os_aio_linux_handler (global_segment=2, m1=0x3fff72ffe6f8, m2=0x3fff72ffe700, request=0x3fff72ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2085
#5  0x000000005b7ed6e8 in os_aio_handler (segment=2, m1=0x3fff72ffe6f8, m2=0x3fff72ffe700, request=0x3fff72ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:5682
#6  0x000000005bb22ff0 in fil_aio_wait (segment=2) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5423
#7  0x000000005b94ab54 in io_handler_thread (arg=0x5c8ed598 <n+16>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0start.cc:327
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff72fff180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 21 (LWP 27884):
#0  0x00003fff7e4f1438 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005bf1b57c in vio_io_wait (vio=0x10016357980, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:945
#2  0x000000005bf19180 in vio_socket_io_wait (vio=0x10016357980, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:108
#3  0x000000005bf19404 in vio_read (vio=0x10016357980, buf=0x3fff1c0088a0 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:184
#4  0x000000005af4f6f8 in my_real_read (net=0x3fff1c000dc8, complen=0x3fff781d7100, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:888
#5  0x000000005af50340 in my_net_read_packet_reallen (net=0x3fff1c000dc8, read_from_server=1 '\001', reallen=0x3fff781d7188) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1158
#6  0x000000005af502ac in my_net_read_packet (net=0x3fff1c000dc8, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1142
#7  0x000000005b06bc04 in do_command (thd=0x3fff1c000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:1244
#8  0x000000005b224860 in do_handle_one_connection (connect=0x100162b32a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1354
#9  0x000000005b2244bc in handle_one_connection (arg=0x100162b32a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1260
#10 0x000000005bdf462c in pfs_spawn_thread (arg=0x10016357d20) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#11 0x00003fff7ec97cec in start_thread (arg=0x3fff781d8180) at pthread_create.c:312
#12 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 20 (LWP 27827):
#0  0x00003fff7ebb0738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff7ebb07a0 in io_getevents () from /lib64/libaio.so.1
#2  0x000000005b7e6c6c in LinuxAIOHandler::collect (this=0x3fff717fe620) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:1886
#3  0x000000005b7e726c in LinuxAIOHandler::poll (this=0x3fff717fe620, m1=0x3fff717fe6f8, m2=0x3fff717fe700, request=0x3fff717fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2031
#4  0x000000005b7e7430 in os_aio_linux_handler (global_segment=5, m1=0x3fff717fe6f8, m2=0x3fff717fe700, request=0x3fff717fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2085
#5  0x000000005b7ed6e8 in os_aio_handler (segment=5, m1=0x3fff717fe6f8, m2=0x3fff717fe700, request=0x3fff717fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:5682
#6  0x000000005bb22ff0 in fil_aio_wait (segment=5) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5423
#7  0x000000005b94ab54 in io_handler_thread (arg=0x5c8ed5b0 <n+40>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0start.cc:327
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff717ff180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 19 (LWP 27840):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x10015db6b40, mutex=0x10015db6b18, abstime=0x3fff6b7fe6a0) at pthread_cond_timedwait.c:199
#1  0x000000005b7f9748 in os_event::timed_wait (this=0x10015db6b00, abstime=0x3fff6b7fe6a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:286
#2  0x000000005b7f9b94 in os_event::wait_time_low (this=0x10015db6b00, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:413
#3  0x000000005b7f9f40 in os_event_wait_time_low (event=0x10015db6b00, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:518
#4  0x000000005b942310 in srv_monitor_thread () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1748
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff6b7ff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 18 (LWP 27822):
#0  0x00003fff7ebb0738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff7ebb07a0 in io_getevents () from /lib64/libaio.so.1
#2  0x000000005b7e6c6c in LinuxAIOHandler::collect (this=0x3fff73ffe620) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:1886
#3  0x000000005b7e726c in LinuxAIOHandler::poll (this=0x3fff73ffe620, m1=0x3fff73ffe6f8, m2=0x3fff73ffe700, request=0x3fff73ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2031
#4  0x000000005b7e7430 in os_aio_linux_handler (global_segment=0, m1=0x3fff73ffe6f8, m2=0x3fff73ffe700, request=0x3fff73ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2085
#5  0x000000005b7ed6e8 in os_aio_handler (segment=0, m1=0x3fff73ffe6f8, m2=0x3fff73ffe700, request=0x3fff73ffe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:5682
#6  0x000000005bb22ff0 in fil_aio_wait (segment=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5423
#7  0x000000005b94ab54 in io_handler_thread (arg=0x5c8ed588 <n>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0start.cc:327
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff73fff180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 17 (LWP 27883):
#0  0x00003fff7e4f1438 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005bf1b57c in vio_io_wait (vio=0x10016357710, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:945
#2  0x000000005bf19180 in vio_socket_io_wait (vio=0x10016357710, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:108
#3  0x000000005bf19404 in vio_read (vio=0x10016357710, buf=0x3fff280088a0 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:184
#4  0x000000005af4f6f8 in my_real_read (net=0x3fff28000dc8, complen=0x3fff784a7100, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:888
#5  0x000000005af50340 in my_net_read_packet_reallen (net=0x3fff28000dc8, read_from_server=1 '\001', reallen=0x3fff784a7188) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1158
#6  0x000000005af502ac in my_net_read_packet (net=0x3fff28000dc8, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1142
#7  0x000000005b06bc04 in do_command (thd=0x3fff28000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:1244
#8  0x000000005b224860 in do_handle_one_connection (connect=0x1001634c8c0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1354
#9  0x000000005b2244bc in handle_one_connection (arg=0x1001634c8c0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1260
#10 0x000000005bdf462c in pfs_spawn_thread (arg=0x10016357980) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#11 0x00003fff7ec97cec in start_thread (arg=0x3fff784a8180) at pthread_create.c:312
#12 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 16 (LWP 27825):
#0  0x00003fff7ebb0738 in __io_getevents_0_4 () from /lib64/libaio.so.1
#1  0x00003fff7ebb07a0 in io_getevents () from /lib64/libaio.so.1
#2  0x000000005b7e6c6c in LinuxAIOHandler::collect (this=0x3fff727fe620) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:1886
#3  0x000000005b7e726c in LinuxAIOHandler::poll (this=0x3fff727fe620, m1=0x3fff727fe6f8, m2=0x3fff727fe700, request=0x3fff727fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2031
#4  0x000000005b7e7430 in os_aio_linux_handler (global_segment=3, m1=0x3fff727fe6f8, m2=0x3fff727fe700, request=0x3fff727fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:2085
#5  0x000000005b7ed6e8 in os_aio_handler (segment=3, m1=0x3fff727fe6f8, m2=0x3fff727fe700, request=0x3fff727fe708) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0file.cc:5682
#6  0x000000005bb22ff0 in fil_aio_wait (segment=3) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5423
#7  0x000000005b94ab54 in io_handler_thread (arg=0x5c8ed5a0 <n+24>) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0start.cc:327
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff727ff180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 15 (LWP 27820):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x5d117fa8 <COND_timer>, mutex=0x5d117f20 <LOCK_timer+40>, abstime=0x3fff79f3e720) at pthread_cond_timedwait.c:199
#1  0x000000005be80dd8 in safe_cond_timedwait (cond=0x5d117fa8 <COND_timer>, mp=0x5d117ef8 <LOCK_timer>, abstime=0x3fff79f3e720, file=0x5c246648 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:545
#2  0x000000005be82708 in inline_mysql_cond_timedwait (that=0x5d117fa8 <COND_timer>, mutex=0x5d117ef8 <LOCK_timer>, abstime=0x3fff79f3e720, src_file=0x5c2466b8 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_timer.c", src_line=292) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175
#3  0x000000005be83750 in timer_handler (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_timer.c:292
#4  0x000000005bdf462c in pfs_spawn_thread (arg=0x10015d3c210) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff79f3f180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 14 (LWP 27838):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x10015f16c50, mutex=0x10015f16c28, abstime=0x3fff707fe6d0) at pthread_cond_timedwait.c:199
#1  0x000000005b7f9748 in os_event::timed_wait (this=0x10015f16c10, abstime=0x3fff707fe6d0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:286
#2  0x000000005b7f9b94 in os_event::wait_time_low (this=0x10015f16c10, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:413
#3  0x000000005b7f9f40 in os_event_wait_time_low (event=0x10015f16c10, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:518
#4  0x000000005b7b0f68 in lock_wait_timeout_thread () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/lock/lock0wait.cc:529
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff707ff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 13 (LWP 27887):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x5c86a218 <COND_thread_cache>, mutex=0x5c86c958 <LOCK_thread_cache+40>, abstime=0x3fff78177590) at pthread_cond_timedwait.c:199
#1  0x000000005be80dd8 in safe_cond_timedwait (cond=0x5c86a218 <COND_thread_cache>, mp=0x5c86c930 <LOCK_thread_cache>, abstime=0x3fff78177590, file=0x5bf75c28 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:545
#2  0x000000005af30b98 in inline_mysql_cond_timedwait (that=0x5c86a218 <COND_thread_cache>, mutex=0x5c86c930 <LOCK_thread_cache>, abstime=0x3fff78177590, src_file=0x5bf77c20 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175
#3  0x000000005af38830 in cache_thread (thd=0x3fff20000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:2949
#4  0x000000005af38c38 in one_thread_per_connection_end (thd=0x3fff20000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:3038
#5  0x000000005b224990 in do_handle_one_connection (connect=0x10016333b70) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1373
#6  0x000000005b2244bc in handle_one_connection (arg=0x10016333b70) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1260
#7  0x000000005bdf462c in pfs_spawn_thread (arg=0x10015d3e720) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#8  0x00003fff7ec97cec in start_thread (arg=0x3fff78178180) at pthread_create.c:312
#9  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 12 (LWP 27860):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x3fff4dffe640, mutex=0x3fff4dffe6a0) at pthread_cond_wait.c:188
#1  0x000000005be80a00 in safe_cond_wait (cond=0x3fff4dffe640, mp=0x3fff4dffe678, file=0x5c0aaca0 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1149) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:491
#2  0x000000005b6d4f58 in inline_mysql_cond_wait (that=0x3fff4dffe640, mutex=0x3fff4dffe678, src_file=0x5c0acaf8 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/handler/ha_innodb.cc", src_line=338) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1149
#3  0x000000005b6dcce8 in thd_destructor_proxy () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:338
#4  0x000000005bdf462c in pfs_spawn_thread (arg=0x10015f78240) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff4dfff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 11 (LWP 27817):
#0  0x00003fff7e4f1438 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005af41e20 in handle_connections_sockets () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:6556
#2  0x000000005af40b94 in mysqld_main (argc=146, argv=0x10015babfc0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:6022
#3  0x000000005af2fac8 in main (argc=25, argv=0x3fffc553dda8) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/main.cc:25
 
Thread 10 (LWP 27878):
#0  0x00003fff7e4f1438 in poll () at ../sysdeps/unix/syscall-template.S:81
#1  0x000000005bf1b57c in vio_io_wait (vio=0x10016312490, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:945
#2  0x000000005bf19180 in vio_socket_io_wait (vio=0x10016312490, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:108
#3  0x000000005bf19404 in vio_read (vio=0x10016312490, buf=0x3fff240ebf50 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/vio/viosocket.c:184
#4  0x000000005af4f6f8 in my_real_read (net=0x3fff24000dc8, complen=0x3fff78507100, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:888
#5  0x000000005af50340 in my_net_read_packet_reallen (net=0x3fff24000dc8, read_from_server=1 '\001', reallen=0x3fff78507188) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1158
#6  0x000000005af502ac in my_net_read_packet (net=0x3fff24000dc8, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/net_serv.cc:1142
#7  0x000000005b06bc04 in do_command (thd=0x3fff24000b00) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_parse.cc:1244
#8  0x000000005b224860 in do_handle_one_connection (connect=0x100162b32a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1354
#9  0x000000005b2244bc in handle_one_connection (arg=0x100162b32a0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/sql_connect.cc:1260
#10 0x000000005bdf462c in pfs_spawn_thread (arg=0x10016357710) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#11 0x00003fff7ec97cec in start_thread (arg=0x3fff78508180) at pthread_create.c:312
#12 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 9 (LWP 27872):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x5c86a288 <COND_slave_background>, mutex=0x5c86d2f8 <LOCK_slave_background+40>) at pthread_cond_wait.c:188
#1  0x000000005be80a00 in safe_cond_wait (cond=0x5c86a288 <COND_slave_background>, mp=0x5c86d2d0 <LOCK_slave_background>, file=0x5bf84070 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:491
#2  0x000000005af77a58 in inline_mysql_cond_wait (that=0x5c86a288 <COND_slave_background>, mutex=0x5c86d2d0 <LOCK_slave_background>, src_file=0x5bf84aa8 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/slave.cc", src_line=334) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138
#3  0x000000005af79170 in handle_slave_background (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/slave.cc:334
#4  0x000000005bdf462c in pfs_spawn_thread (arg=0x10016312490) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff78568180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 8 (LWP 27859):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x10015db6cc0, mutex=0x10015db6c98) at pthread_cond_wait.c:188
#1  0x000000005b7fa46c in os_event::wait (this=0x10015db6c80) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:166
#2  0x000000005b7f993c in os_event::wait_low (this=0x10015db6c80, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:336
#3  0x000000005b7f9f98 in os_event_wait_low (event=0x10015db6c80, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:535
#4  0x000000005ba58494 in buf_resize_thread () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/buf/buf0buf.cc:3160
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff4e7ff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 7 (LWP 27868):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x5c898e48 <mysql_bin_log+4216>, mutex=0x5c898dc0 <mysql_bin_log+4080>) at pthread_cond_wait.c:188
#1  0x000000005be80a00 in safe_cond_wait (cond=0x5c898e48 <mysql_bin_log+4216>, mp=0x5c898d98 <mysql_bin_log+4040>, file=0x5c07ff00 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:491
#2  0x000000005b553d60 in inline_mysql_cond_wait (that=0x5c898e48 <mysql_bin_log+4216>, mutex=0x5c898d98 <mysql_bin_log+4040>, src_file=0x5c080548 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/log.cc", src_line=9607) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138
#3  0x000000005b575df4 in binlog_background_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/log.cc:9607
#4  0x000000005bdf462c in pfs_spawn_thread (arg=0x100162622d0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff78628180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 6 (LWP 27870):
#0  0x00003fff7eca5714 in do_sigwait (sig=0x3fff785c7680, set=<optimized out>) at ../sysdeps/unix/sysv/linux/sigwait.c:61
#1  __sigwait (set=<optimized out>, sig=0x3fff785c7680) at ../sysdeps/unix/sysv/linux/sigwait.c:98
#2  0x000000005af39a7c in signal_hand (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/mysqld.cc:3487
#3  0x000000005bdf462c in pfs_spawn_thread (arg=0x10016290420) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#4  0x00003fff7ec97cec in start_thread (arg=0x3fff785c8180) at pthread_create.c:312
#5  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 5 (LWP 27857):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x10015ef8940, mutex=0x10015ef8918, abstime=0x3fff4f7fe4c0) at pthread_cond_timedwait.c:199
#1  0x000000005b7f9748 in os_event::timed_wait (this=0x10015ef8900, abstime=0x3fff4f7fe4c0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:286
#2  0x000000005b7f9b94 in os_event::wait_time_low (this=0x10015ef8900, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:413
#3  0x000000005b7f9f40 in os_event_wait_time_low (event=0x10015ef8900, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:518
#4  0x000000005b9dd0c0 in ib_wqueue_timedwait (wq=0x10015ef9880, wait_in_usecs=5000000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/ut/ut0wqueue.cc:163
#5  0x000000005bb8305c in fts_optimize_thread (arg=0x10015ef9880) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/fts/fts0opt.cc:3046
#6  0x00003fff7ec97cec in start_thread (arg=0x3fff4f7ff180) at pthread_create.c:312
#7  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 4 (LWP 27861):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x5d106748 <COND_checkpoint>, mutex=0x5d1066c0 <LOCK_checkpoint+40>, abstime=0x3fff4d04e620) at pthread_cond_timedwait.c:199
#1  0x000000005be80dd8 in safe_cond_timedwait (cond=0x5d106748 <COND_checkpoint>, mp=0x5d106698 <LOCK_checkpoint>, abstime=0x3fff4d04e620, file=0x5c207178 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/thr_mutex.c:545
#2  0x000000005bc52970 in inline_mysql_cond_timedwait (that=0x5d106748 <COND_checkpoint>, mutex=0x5d106698 <LOCK_checkpoint>, abstime=0x3fff4d04e620, src_file=0x5c2071f8 "/home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175
#3  0x000000005bc530d4 in my_service_thread_sleep (control=0x5c6aa760 <checkpoint_control>, sleep_time=29000000000) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_servicethread.c:115
#4  0x000000005bc41bc8 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/maria/ma_checkpoint.c:708
#5  0x000000005bdf462c in pfs_spawn_thread (arg=0x10016257930) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/perfschema/pfs.cc:1862
#6  0x00003fff7ec97cec in start_thread (arg=0x3fff4d04f180) at pthread_create.c:312
#7  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 3 (LWP 27856):
#0  0x00003fff7ec9fcc0 in __pthread_cond_timedwait (cond=0x10015f2a390, mutex=0x10015f2a368, abstime=0x3fff4fffe6f0) at pthread_cond_timedwait.c:199
#1  0x000000005b7f9748 in os_event::timed_wait (this=0x10015f2a350, abstime=0x3fff4fffe6f0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:286
#2  0x000000005b7f9b94 in os_event::wait_time_low (this=0x10015f2a350, time_in_usec=10000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:413
#3  0x000000005b7f9f40 in os_event_wait_time_low (event=0x10015f2a350, time_in_usec=10000000, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:518
#4  0x000000005bb012a0 in dict_stats_thread () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/dict/dict0stats_bg.cc:418
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff4ffff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 2 (LWP 27854):
#0  0x00003fff7ec9f7d0 in __pthread_cond_wait (cond=0x10015db6bc0, mutex=0x10015db6b98) at pthread_cond_wait.c:188
#1  0x000000005b7fa46c in os_event::wait (this=0x10015db6b80) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:166
#2  0x000000005b7f993c in os_event::wait_low (this=0x10015db6b80, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:336
#3  0x000000005b7f9f98 in os_event_wait_low (event=0x10015db6b80, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/os/os0event.cc:535
#4  0x000000005ba76468 in buf_dump_thread () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/buf/buf0dump.cc:814
#5  0x00003fff7ec97cec in start_thread (arg=0x3fff687ff180) at pthread_create.c:312
#6  0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96
 
Thread 1 (LWP 27845):
#0  0x00003fff7eca0ef0 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60
#1  0x000000005be79044 in my_write_core (sig=11) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/mysys/stacktrace.c:477
#2  0x000000005b3c9e58 in handle_fatal_signal (sig=11) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/sql/signal_handler.cc:299
#3  <signal handler called>
#4  internal_fnmatch (pattern=<optimized out>, string=0x5c0feb98 "ib_que", string_end=0x5c0feb9e "", no_leading_period=0, flags=0, ends=0x0, alloca_used=0) at fnmatch_loop.c:62
#5  0x00003fff7e4c1cf0 in __fnmatch (pattern=0xb <Address 0xb out of bounds>, string=0x5c0feb98 "ib_que", flags=<optimized out>) at fnmatch.c:458
#6  0x000000005beb1e20 in InList (linkp=0x2, cp=0x5c0feb98 "ib_que", exact_match=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/dbug/dbug.c:1515
#7  0x000000005beb25bc in _db_keyword_ (cs=0x3fff50000a30, keyword=0x5c0feb98 "ib_que", strict=0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/dbug/dbug.c:1717
#8  0x000000005beb12c4 in _db_pargs_ (_line_=994, keyword=0x5c0feb98 "ib_que") at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/dbug/dbug.c:1236
#9  0x000000005b83b674 in que_thr_step (thr=0x10015f61860) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/que/que0que.cc:992
#10 0x000000005b83bc90 in que_run_threads_low (thr=0x10015f61860) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/que/que0que.cc:1114
#11 0x000000005b83bf00 in que_run_threads (thr=0x10015f61860) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/que/que0que.cc:1154
#12 0x000000005b944f4c in srv_task_execute () at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2656
#13 0x000000005b9451b0 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp06-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2705
#14 0x00003fff7ec97cec in start_thread (arg=0x3fff68fff180) at pthread_create.c:312
#15 0x00003fff7e501140 in clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96



 Comments   
Comment by Alice Sherepa [ 2017-09-16 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9619/steps/test_3/logs/stdio

binlog_encryption.rpl_binlog_errors 'innodb,stmt' w2 [ fail ]
        Test ended at 2017-09-16 03:34:46
 
CURRENT_TEST: binlog_encryption.rpl_binlog_errors
mysqltest: In included file "./extra/rpl_tests/rpl_binlog_errors.inc": 
included from /mnt/buildbot/build/mariadb-10.2.9/mysql-test/suite/binlog_encryption/rpl_binlog_errors.test at line 3:
At line 186: query 'RESET MASTER' failed: 2013: Lost connection to MySQL server during query
 
The result from queries just before the failure was:
< snip >
SET GLOBAL debug_dbug=@old_debug;
DELETE FROM t2;
RESET MASTER;
###################### TEST #6
SET GLOBAL debug_dbug="+d,error_unique_log_filename";
SET AUTOCOMMIT=0;
INSERT INTO t2 VALUES ('muse');
LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/bug_46166.data' INTO TABLE t2;
INSERT INTO t2 VALUES ('muse');
COMMIT;
ERROR HY000: Can't generate a unique log-filename master-bin.(1-999)
 
# assert: must show three entries
SELECT count(*) FROM t2;
count(*)
3
SET AUTOCOMMIT= 1;
SET GLOBAL debug_dbug=@old_debug;
DELETE FROM t2;
RESET MASTER;

2017-09-16  3:34:40 2859125568 [Warning] Aborted connection 13 to db: 'unconnected' user: 'root' host: 'localhost' (Got an error writing communication packets)
2017-09-16  3:34:40 2858822464 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-09-16  3:34:41 2858822464 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-09-16  3:34:41 2858822464 [ERROR] mysqld: Error writing file 'master-bin' (errno: 0 "Internal error/check (Not system error)")
2017-09-16  3:34:43 2858822464 [ERROR] Can't generate a unique log-filename master-bin.(1-999)
 
2017-09-16  3:34:43 2858822464 [ERROR] mysqld: Error writing file 'master-bin' (errno: 11 "Resource temporarily unavailable")
170916  3:34:43 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.2.9-MariaDB-debug-log
stack_bottom = 0xa94fd28c thread_stack 0x49000
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(my_print_stacktrace+0x32)[0x8cf57ce]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(handle_fatal_signal+0x3c7)[0x85a2ef3]
[0xb772e400]
/lib/i386-linux-gnu/libc.so.6(+0xbd4a5)[0xb72994a5]
/lib/i386-linux-gnu/libc.so.6(fnmatch+0x69)[0xb729aaf9]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8d17932]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_db_keyword_+0x79)[0x8d17e0a]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_db_pargs_+0x65)[0x8d17220]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(free_root+0x51)[0x8ce32c2]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_Z9reset_thdP3THD+0x51)[0x82fdc1d]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x88a272b]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a0cd6e]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899afcd]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899b1d8]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899b3ae]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a8925b]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a59bab]
/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a5a166]
/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb74b6d4c]
/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb72c6ace]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): 
Connection ID (thread ID): 1
Status: NOT_KILLED

Generated at Thu Feb 08 08:00:06 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.