[MDEV-11119] Server crashes on STOP SLAVE, rpl.rpl_parallel_retry fails in buildbot - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Won't Fix
Affects Version/s: 10.1(EOL), 10.2(EOL)
Fix Version/s: N/A
Component/s: Replication, Tests
Labels:
None

Description

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/6136/steps/test_2/logs/stdio

rpl.rpl_parallel_retry 'mix,xtradb'      w2 [ fail ]

        Test ended at 2016-10-23 04:22:51

CURRENT_TEST: rpl.rpl_parallel_retry

mysqltest: In included file "./include/stop_slave.inc":

included from /mnt/buildbot/build/mariadb-10.1.19/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 377:

At line 82: query 'STOP SLAVE' failed: 2013: Lost connection to MySQL server during query

The result from queries just before the failure was:

161023  4:22:50 [ERROR] mysqld got signal 11 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed,

something is definitely wrong and this may fail.

Server version: 10.1.19-MariaDB-debug

key_buffer_size=1048576

read_buffer_size=131072

max_used_connections=7

max_threads=153

thread_count=2

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61817 K  bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0xa65063c0

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0xa726a660 thread_stack 0x48400

2016-10-23  4:22:50 2807312192 [Note] Error reading relay log event: slave SQL thread was killed

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(my_print_stacktrace+0x32)[0x8b2c2d5]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(handle_fatal_signal+0x3b7)[0x84d5c43]

[0xb77b7400]

/lib/i386-linux-gnu/libc.so.6(+0x7b678)[0xb72e0678]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x8b49b06]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(_db_keyword_+0x79)[0x8b4a002]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(my_write+0xf6)[0x8b2a6d6]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x8b078b0]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x8b0a6e7]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(my_b_flush_io_cache+0x25e)[0x8b0afea]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(_Z20flush_relay_log_infoP14Relay_log_info+0x1f4)[0x8213e15]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(_ZN14Relay_log_info9stmt_doneEyP3THDP14rpl_group_info+0x1e4)[0x83bdcb8]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(_ZN9Log_event13do_update_posEP14rpl_group_info+0xb5)[0x85c1515]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(_ZN9Log_event10update_posEP14rpl_group_info+0x1f)[0x821625f]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x820b84e]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(_Z39apply_event_and_update_pos_for_parallelP9Log_eventP3THDP14rpl_group_info+0xdd)[0x820bc22]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x843ae7c]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x843d026]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld(handle_rpl_parallel_thread+0xb06)[0x843de0e]

/mnt/buildbot/build/mariadb-10.1.19/sql/mysqld[0x86a5924]

/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb753fd4c]

/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb734face]

Trying to get some variables.

Some pointers may be invalid and cause the dump to abort.

Query (0x0):

Connection ID (thread ID): 1339

Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off

According to cross-reference, it only happened twice in buildbot so far. The second occurrence was in April, on bb-10.1-serg (logs not available).

Attachments

Issue Links

relates to

MDEV-7069 Fix buildbot failures in main server trees

Stalled

MDEV-24110 rpl.rpl_parallel (rpl.rpl_parallel_stop_on_con_kill) failed in bb with Lost connection after STOP SLAVE

Confirmed

Activity

Ascending order - Click to sort in descending order

Elena Stepanova added a comment - 2017-05-09 11:33

10.2 variation:

rpl.rpl_parallel_retry 'innodb,row'      w2 [ fail ]

        Test ended at 2017-05-06 12:14:59

CURRENT_TEST: rpl.rpl_parallel_retry

mysqltest: In included file "./include/wait_for_slave_param.inc":

included from ./include/wait_for_slave_io_to_start.inc at line 40:

included from ./include/wait_for_slave_to_start.inc at line 26:

included from ./include/start_slave.inc at line 35:

included from /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 379:

At line 119: query 'let $_show_slave_status_value= query_get_value("SHOW SLAVE STATUS", $slave_param, 1)' failed: 2013: Lost connection to MySQL server during query

The result from queries just before the failure was:

< snip >

SET debug_sync='now WAIT_FOR t3_waiting';

SET debug_sync='now SIGNAL t1_start';

SET GLOBAL debug_dbug=@old_dbug;

SET debug_sync='RESET';

retries

SELECT * FROM t3 ORDER BY a;

a	b

1	NULL

3	NULL

4	4

5	NULL

6	NULL

7	7

connection server_1;

SET binlog_format=@old_format;

connection server_2;

include/stop_slave.inc

SET GLOBAL slave_parallel_threads=@old_parallel_threads;

include/start_slave.inc

More results from queries before failure can be found in /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/2/log/rpl_parallel_retry.log

Server [mysqld.2 - pid: 10609, winpid: 10609, exit: 256] failed during test run

Server log from this test:

----------SERVER LOG START-----------

2017-05-06 12:14:55 70367135957376 [Note] Deleted Master_info file '/dev/shm/var_auto_fqAr/2/mysqld.2/data/master.info'.

2017-05-06 12:14:55 70367135957376 [Note] Deleted Master_info file '/dev/shm/var_auto_fqAr/2/mysqld.2/data/relay-log.info'.

2017-05-06 12:14:55 70367135957376 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'

2017-05-06 12:14:55 70367135957376 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='4'.

2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 4

2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4

2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 484

2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860

2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860

2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 484, relay log './slave-relay-bin.000002' position: 784

2017-05-06 12:14:55 70366319837568 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 860

2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860

2017-05-06 12:14:55 70366319837568 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860

2017-05-06 12:14:55 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 860, relay log './slave-relay-bin.000003' position: 556

2017-05-06 12:14:55 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:55 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1359

2017-05-06 12:14:55 70366319837568 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1359

2017-05-06 12:14:55 70366318657920 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1359

2017-05-06 12:14:55 70367257723264 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1359, relay log './slave-relay-bin.000004' position: 1055

2017-05-06 12:14:55 70367257723264 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:55 70367257723264 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996

2017-05-06 12:14:55 70366318657920 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1996

2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1996

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000005' position: 1193

2017-05-06 12:14:56 70367135170944 [ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up. Consider raising the value of the slave_transaction_retries variable.

2017-05-06 12:14:56 70367135170944 [ERROR] Slave SQL: Deadlock found when trying to get lock; try restarting transaction, Gtid 0-11-100, Internal MariaDB error code: 1213

2017-05-06 12:14:56 70367135170944 [Warning] Slave: Deadlock found when trying to get lock; try restarting transaction Error_code: 1213

2017-05-06 12:14:56 70367135170944 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'master-bin.000001' position 1996

2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996

2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 2633

2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 2633

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000006' position: 556

2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 3028

2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 3028

2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 3028

2017-05-06 12:14:56 70367135170944 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 3028, relay log './slave-relay-bin.000007' position: 951

2017-05-06 12:14:56 70367135170944 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:56 70367135170944 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34004

2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 34004

2017-05-06 12:14:56 70367135170944 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 34004

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34004, relay log './slave-relay-bin.000014' position: 539

2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34641

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34641, relay log './slave-relay-bin.000015' position: 1193

2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 36559

2017-05-06 12:14:56 70367135170944 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 36559

2017-05-06 12:14:56 70367135957376 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'

2017-05-06 12:14:56 70367135957376 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'.

2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 36559

2017-05-06 12:14:56 70367135564160 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 36559, relay log './slave-relay-bin.000001' position: 4

170506 12:14:56 [ERROR] mysqld got signal 11 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed,

something is definitely wrong and this may fail.

Server version: 10.2.6-MariaDB-debug-log

key_buffer_size=1048576

read_buffer_size=131072

max_used_connections=7

max_threads=153

thread_count=21

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63063 K  bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x3fff8c000b00

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0x3fffa136e608 thread_stack 0x49000

2017-05-06 12:14:56 70367135564160 [Note] Error reading relay log event: slave SQL thread was killed

2017-05-06 12:14:56 70367135564160 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37580

2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37580

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(my_print_stacktrace-0xb176e8)[0x221e28f0]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(handle_fatal_signal-0x14ebac8)[0x21752890]

linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fffb1fd0478]

/opt/at8.0/lib64/power8/libc.so.6(+0x1111e0)[0x3fffb17311e0]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0xf8279c)[0x21b6279c]

/opt/at8.0/lib64/power8/libc.so.6(fnmatch-0xe6f70)[0x3fffb17329c8]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0x163b120)[0x2221b120]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_db_keyword_-0xae26ac)[0x2221b88c]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_db_pargs_-0xae380c)[0x2221a5f4]

2017-05-06 12:14:56 70367135564160 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 37580

2017-05-06 12:14:56 70366319837568 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 37580, relay log './slave-relay-bin.000002' position: 1577

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_ZN3THD11enter_stageEPK17PSI_stage_info_v1PKcS4_j-0x191ef9c)[0x212d9a14]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_Z19close_thread_tablesP3THD-0x18a68dc)[0x2135d54c]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_Z9reset_thdP3THD-0x1865d5c)[0x213a2dac]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0xe7fb84)[0x21a5fb84]

mysys/stacktrace.c:267(my_print_stacktrace)[0x21c5270c]

dbug/dbug.c:1515(InList)[0x21bb858c]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0xfd8880)[0x21bb8880]

sql/sql_base.cc:714(close_thread_tables(THD*))[0x21bb8ae8]

sql/sql_class.cc:4510(reset_thd(THD*))[0x21cbca10]

/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0x10dcc08)[0x21cbcc08]

/opt/at8.0/lib64/power8/libpthread.so.0(+0xb9e0)[0x3fffb1f9b9e0]

handler/ha_innodb.cc:1751(innobase_reset_background_thd(THD*))[0x3fffb17718d0]

Trying to get some variables.

Some pointers may be invalid and cause the dump to abort.

Query (0x0):

Connection ID (thread ID): 4

Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains

information that should help you find out what is causing the crash.

Writing a core file

----------SERVER LOG END-------------

 - saving '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/2/log/rpl.rpl_parallel_retry-innodb,row/' to '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/log/rpl.rpl_parallel_retry-innodb,row/'

 - found 'core.10610' (0/5)

Trying 'dbx' to get a backtrace

Trying 'gdb' to get a backtrace from coredump /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/log/rpl.rpl_parallel_retry-innodb,row/mysqld.2/data/core.10610

Core generated by '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld'

Output from gdb follows. The first stack trace is from the failing thread.

The following stack traces are from all threads (so the failing one is

duplicated).

--------------------------

[New LWP 10687]

[New LWP 10689]

[New LWP 10700]

[New LWP 10701]

[New LWP 10702]

[New LWP 10712]

[New LWP 10704]

[New LWP 10777]

[New LWP 14308]

[New LWP 10714]

[New LWP 15645]

[New LWP 10662]

[New LWP 10723]

[New LWP 10666]

[New LWP 12430]

[New LWP 10682]

[New LWP 10686]

[New LWP 10610]

[New LWP 10663]

[New LWP 10665]

[New LWP 10667]

[New LWP 10683]

[New LWP 10668]

[New LWP 10664]

[New LWP 15644]

[New LWP 10685]

[New LWP 10684]

[New LWP 10661]

[New LWP 10726]

[New LWP 14279]

[New LWP 10716]

[New LWP 10776]

[New LWP 10703]

[New LWP 10705]

[New LWP 10698]

[New LWP 10688]

warning: Could not load shared library symbols for linux-vdso64.so.1.

Do you need "set solib-search-path" or "set sysroot"?

warning: File "/opt/at8.0/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".

To enable execution of this file add

	add-auto-load-safe-path /opt/at8.0/lib64/power8/libthread_db-1.0.so

line to your configuration file "/home/buildbot/.gdbinit".

To completely disable this security protection add

	set auto-load safe-path /

line to your configuration file "/home/buildbot/.gdbinit".

For more information about this security protection see the

"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:

	info "(gdb)Auto-loading safe path"

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

warning: File "/opt/at8.0/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

Core was generated by `/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld --default'.

Program terminated with signal 11, Segmentation fault.

#0  0x00003fffb1fa4f00 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60

60	../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.

#0  0x00003fffb1fa4f00 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60

#1  0x00000000221e2a88 in my_write_core (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/stacktrace.c:477

#2  0x0000000021752e18 in handle_fatal_signal (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/signal_handler.cc:299

#3  <signal handler called>

#4  internal_fnmatch (pattern=<optimized out>, string=string@entry=0x222db6a0 "THD::enter_stage", string_end=0x222db6b0 "", no_leading_period=no_leading_period@entry=0, flags=flags@entry=0, ends=ends@entry=0x0, alloca_used=alloca_used@entry=0) at fnmatch_loop.c:62

#5  0x00003fffb17329c8 in __fnmatch (pattern=0xd9 <Address 0xd9 out of bounds>, string=0x222db6a0 "THD::enter_stage", flags=<optimized out>) at fnmatch.c:458

#6  0x000000002221b120 in InList (linkp=0xd0, cp=0x222db6a0 "THD::enter_stage", exact_match=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1515

#7  0x000000002221b88c in _db_keyword_ (cs=0x3fff8c000a30, keyword=0x222db6a0 "THD::enter_stage", strict=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1717

#8  0x000000002221a5f4 in _db_pargs_ (_line_=2175, keyword=0x222db6a0 "THD::enter_stage") at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1236

#9  0x00000000212d9a14 in THD::enter_stage (this=0x3fff8c000b00, stage=0x22a011d0 <stage_closing_tables>, calling_func=0x222f72e8 <close_thread_tables(THD*)::__FUNCTION__> "close_thread_tables", calling_file=0x222f55f8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc", calling_line=714) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.h:2175

#10 0x000000002135d54c in close_thread_tables (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc:714

#11 0x00000000213a2dac in reset_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.cc:4510

#12 0x0000000021a5fb84 in innobase_reset_background_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:1751

#13 0x0000000021c5270c in row_purge_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/row/row0purge.cc:1100

#14 0x0000000021bb858c in que_thr_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1052

#15 0x0000000021bb8880 in que_run_threads_low (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1114

#16 0x0000000021bb8ae8 in que_run_threads (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1154

#17 0x0000000021cbca10 in srv_task_execute () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2667

#18 0x0000000021cbcc08 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2712

#19 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa136f180) at pthread_create.c:312

#20 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 36 (LWP 10688):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db70f0, mutex=0x10033db70c8) at pthread_cond_wait.c:188

#1  0x0000000021b7845c in os_event::wait (this=0x10033db70b0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166

#2  0x0000000021b779b4 in os_event::wait_low (this=0x10033db70b0, reset_sig_count=3183) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336

#3  0x0000000021b77fc8 in os_event_wait_low (event=0x10033db70b0, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535

#4  0x0000000021cb73a8 in srv_resume_thread (slot=0x10033db6dd0, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:954

#5  0x0000000021cbcc04 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2710

#6  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0b6f180) at pthread_create.c:312

#7  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 35 (LWP 10698):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db7370, mutex=0x10033db7348) at pthread_cond_wait.c:188

#1  0x0000000021b7845c in os_event::wait (this=0x10033db7330) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166

#2  0x0000000021b779b4 in os_event::wait_low (this=0x10033db7330, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336

#3  0x0000000021b77fc8 in os_event_wait_low (event=0x10033db7330, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535

#4  0x0000000021de7f68 in buf_dump_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0dump.cc:815

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9b7ff180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 34 (LWP 10705):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x235a4e90 <COND_checkpoint>, mutex=0x235a4e08 <LOCK_checkpoint+40>, abstime=0x3fff6fffe570) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x235a4e90 <COND_checkpoint>, mp=0x235a4de0 <LOCK_checkpoint>, abstime=0x3fff6fffe570, file=0x2256da08 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x0000000021fbef9c in inline_mysql_cond_timedwait (that=0x235a4e90 <COND_checkpoint>, mutex=0x235a4de0 <LOCK_checkpoint>, abstime=0x3fff6fffe570, src_file=0x2256da88 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x0000000021fbf6f8 in my_service_thread_sleep (control=0x22a3a728 <checkpoint_control>, sleep_time=30000000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/maria/ma_servicethread.c:115

#4  0x0000000021fae41c in ma_checkpoint_background (arg=0x1e) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/maria/ma_checkpoint.c:708

#5  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034258330) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#6  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6ffff180) at pthread_create.c:312

#7  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 33 (LWP 10703):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db7470, mutex=0x10033db7448) at pthread_cond_wait.c:188

#1  0x0000000021b7845c in os_event::wait (this=0x10033db7430) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166

#2  0x0000000021b779b4 in os_event::wait_low (this=0x10033db7430, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336

#3  0x0000000021b77fc8 in os_event_wait_low (event=0x10033db7430, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535

#4  0x0000000021dcacac in buf_resize_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0buf.cc:3063

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff997ff180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 32 (LWP 10776):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fffa0367470) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa0367470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa0367470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x00000000212c60d0 in cache_thread (thd=0x3fff4c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949

#4  0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff4c000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038

#5  0x00000000215b0300 in do_handle_one_connection (connect=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373

#6  0x00000000215afe1c in handle_one_connection (arg=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#7  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3eec0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0368180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 31 (LWP 10716):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x22d08888 <COND_slave_background>, mutex=0x22d0b8f8 <LOCK_slave_background+40>) at pthread_cond_wait.c:188

#1  0x00000000221ea3c8 in safe_cond_wait (cond=0x22d08888 <COND_slave_background>, mp=0x22d0b8d0 <LOCK_slave_background>, file=0x222e98d8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491

#2  0x000000002130441c in inline_mysql_cond_wait (that=0x22d08888 <COND_slave_background>, mutex=0x22d0b8d0 <LOCK_slave_background>, src_file=0x222ea310 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc", src_line=334) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138

#3  0x0000000021305b48 in handle_slave_background (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:334

#4  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034312a50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa7788180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 30 (LWP 14279):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fff6f677470) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f677470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f677470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x00000000212c60d0 in cache_thread (thd=0x3fff34215bf0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949

#4  0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff34215bf0, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038

#5  0x00000000215b0300 in do_handle_one_connection (connect=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373

#6  0x00000000215afe1c in handle_one_connection (arg=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#7  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3f600) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6f678180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 29 (LWP 10726):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fffa76c7470) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa76c7470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa76c7470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x00000000212c60d0 in cache_thread (thd=0x3fff50000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949

#4  0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff50000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038

#5  0x00000000215b0300 in do_handle_one_connection (connect=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373

#6  0x00000000215afe1c in handle_one_connection (arg=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#7  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034357b50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa76c8180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 28 (LWP 10661):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x235b6728 <COND_timer>, mutex=0x235b66a0 <LOCK_timer+40>, abstime=0x3fffad18e680) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x235b6728 <COND_timer>, mp=0x235b6678 <LOCK_timer>, abstime=0x3fffad18e680, file=0x225ad0f0 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x00000000221ec068 in inline_mysql_cond_timedwait (that=0x235b6728 <COND_timer>, mutex=0x235b6678 <LOCK_timer>, abstime=0x3fffad18e680, src_file=0x225ad160 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_timer.c", src_line=292) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x00000000221ed094 in timer_handler (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_timer.c:292

#4  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3c950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffad18f180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 27 (LWP 10684):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033db72f0, mutex=0x10033db72c8, abstime=0x3fffa2b6e5d0) at pthread_cond_timedwait.c:199

#1  0x0000000021b777c8 in os_event::timed_wait (this=0x10033db72b0, abstime=0x3fffa2b6e5d0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286

#2  0x0000000021b77c04 in os_event::wait_time_low (this=0x10033db72b0, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413

#3  0x0000000021b77f78 in os_event_wait_time_low (event=0x10033db72b0, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518

#4  0x0000000021cb9e54 in srv_monitor_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1775

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa2b6f180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 26 (LWP 10685):

#0  0x00003fffb1fa8984 in .__libc_nanosleep () at ../sysdeps/unix/syscall-template.S:81

#1  0x0000000021b78ab0 in os_thread_sleep (tm=1000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0thread.cc:245

#2  0x0000000021cbc260 in srv_master_sleep () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2521

#3  0x0000000021cbc484 in srv_master_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2568

#4  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa236f180) at pthread_create.c:312

#5  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 25 (LWP 15644):

#0  0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81

#1  0x0000000022280430 in vio_io_wait (vio=0x3fff34146d20, event=VIO_IO_EVENT_READ, timeout=120000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:945

#2  0x000000002227e098 in vio_socket_io_wait (vio=0x3fff34146d20, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:108

#3  0x000000002227e314 in vio_read (vio=0x3fff34146d20, buf=0x3fff3413eaf0 "1", size=16384) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:184

#4  0x000000002227e5e8 in vio_read_buff (vio=0x3fff34146d20, buf=0x3fff342e6910 "", size=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:232

#5  0x00000000212dc8c0 in my_real_read (net=0x3fff34074c40, complen=0x3fffa01e7300, header=0 '\000') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:888

#6  0x00000000212dd550 in my_net_read_packet_reallen (net=0x3fff34074c40, read_from_server=0 '\000', reallen=0x3fffa01e7568) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1158

#7  0x00000000216ed060 in cli_safe_read_reallen (mysql=0x3fff34074c40, reallen=0x3fffa01e7568) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql-common/client.c:587

#8  0x0000000021311660 in read_event (mysql=0x3fff34074c40, mi=0x10034351c50, suppress_warnings=0x3fffa01e7541, network_read_len=0x3fffa01e7568) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:3344

#9  0x0000000021314964 in handle_slave_io (arg=0x10034351c50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:4456

#10 0x000000002215e7b8 in pfs_spawn_thread (arg=0x3fff34146d20) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#11 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa01e8180) at pthread_create.c:312

#12 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 24 (LWP 10664):

#0  0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1

#1  0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1

#2  0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa657e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886

#3  0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa657e370, m1=0x3fffa657e4d0, m2=0x3fffa657e4d8, request=0x3fffa657e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031

#4  0x0000000021b65d38 in os_aio_linux_handler (global_segment=2, m1=0x3fffa657e4d0, m2=0x3fffa657e4d8, request=0x3fffa657e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085

#5  0x0000000021b6bea0 in os_aio_handler (segment=2, m1=0x3fffa657e4d0, m2=0x3fffa657e4d8, request=0x3fffa657e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683

#6  0x0000000021e93564 in fil_aio_wait (segment=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480

#7  0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcd8 <n+16>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa657f180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 23 (LWP 10668):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033db73f0, mutex=0x10033db73c8, abstime=0x3fffa457e370) at pthread_cond_timedwait.c:199

#1  0x0000000021b777c8 in os_event::timed_wait (this=0x10033db73b0, abstime=0x3fffa457e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286

#2  0x0000000021b77c04 in os_event::wait_time_low (this=0x10033db73b0, time_in_usec=983000, reset_sig_count=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413

#3  0x0000000021b77f78 in os_event_wait_time_low (event=0x10033db73b0, time_in_usec=983000, reset_sig_count=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518

#4  0x0000000021df4690 in pc_sleep_if_needed (next_loop_time=1494087297614, sig_count=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0flu.cc:2711

#5  0x0000000021df5f90 in buf_flush_page_cleaner_coordinator (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0flu.cc:3213

#6  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa457f180) at pthread_create.c:312

#7  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 22 (LWP 10683):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033db7270, mutex=0x10033db7248, abstime=0x3fffa336e470) at pthread_cond_timedwait.c:199

#1  0x0000000021b777c8 in os_event::timed_wait (this=0x10033db7230, abstime=0x3fffa336e470) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286

#2  0x0000000021b77c04 in os_event::wait_time_low (this=0x10033db7230, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413

#3  0x0000000021b77f78 in os_event_wait_time_low (event=0x10033db7230, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518

#4  0x0000000021cba780 in srv_error_monitor_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2000

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa336f180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 21 (LWP 10667):

#0  0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1

#1  0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1

#2  0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa4d7e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886

#3  0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa4d7e370, m1=0x3fffa4d7e4d0, m2=0x3fffa4d7e4d8, request=0x3fffa4d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031

#4  0x0000000021b65d38 in os_aio_linux_handler (global_segment=5, m1=0x3fffa4d7e4d0, m2=0x3fffa4d7e4d8, request=0x3fffa4d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085

#5  0x0000000021b6bea0 in os_aio_handler (segment=5, m1=0x3fffa4d7e4d0, m2=0x3fffa4d7e4d8, request=0x3fffa4d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683

#6  0x0000000021e93564 in fil_aio_wait (segment=5) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480

#7  0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcf0 <n+40>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa4d7f180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 20 (LWP 10665):

#0  0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1

#1  0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1

#2  0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa5d7e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886

#3  0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa5d7e370, m1=0x3fffa5d7e4d0, m2=0x3fffa5d7e4d8, request=0x3fffa5d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031

#4  0x0000000021b65d38 in os_aio_linux_handler (global_segment=3, m1=0x3fffa5d7e4d0, m2=0x3fffa5d7e4d8, request=0x3fffa5d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085

#5  0x0000000021b6bea0 in os_aio_handler (segment=3, m1=0x3fffa5d7e4d0, m2=0x3fffa5d7e4d8, request=0x3fffa5d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683

#6  0x0000000021e93564 in fil_aio_wait (segment=3) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480

#7  0x0000000021cc2454 in io_handler_thread (arg=0x22d8bce0 <n+24>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa5d7f180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 19 (LWP 10663):

#0  0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1

#1  0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1

#2  0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa6d7e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886

#3  0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa6d7e370, m1=0x3fffa6d7e4d0, m2=0x3fffa6d7e4d8, request=0x3fffa6d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031

#4  0x0000000021b65d38 in os_aio_linux_handler (global_segment=1, m1=0x3fffa6d7e4d0, m2=0x3fffa6d7e4d8, request=0x3fffa6d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085

#5  0x0000000021b6bea0 in os_aio_handler (segment=1, m1=0x3fffa6d7e4d0, m2=0x3fffa6d7e4d8, request=0x3fffa6d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683

#6  0x0000000021e93564 in fil_aio_wait (segment=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480

#7  0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcd0 <n+8>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa6d7f180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 18 (LWP 10610):

#0  0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81

#1  0x00000000212cf5a8 in handle_connections_sockets () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:6556

#2  0x00000000212ce35c in mysqld_main (argc=155, argv=0x10033ba8750) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:6022

#3  0x00000000212bd328 in main (argc=24, argv=0x3fffc21e2338) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/main.cc:25

Thread 17 (LWP 10686):

#0  .__sched_yield () at ../sysdeps/unix/syscall-template.S:82

#1  0x0000000021b78a20 in os_thread_yield () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0thread.cc:226

#2  0x0000000021d01b7c in trx_purge_wait_for_workers_to_complete (purge_sys=0x10033f61b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/trx/trx0purge.cc:1618

#3  0x0000000021d02028 in trx_purge (n_purge_threads=3, batch_size=300, truncate=false) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/trx/trx0purge.cc:1700

#4  0x0000000021cbd0bc in srv_do_purge (thd=0x3fff88000b00, n_threads=4, n_total_purged=0x3fffa1b6e720) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2821

#5  0x0000000021cbd8ac in srv_purge_coordinator_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2966

#6  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa1b6f180) at pthread_create.c:312

#7  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 16 (LWP 10682):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033f17420, mutex=0x10033f173f8, abstime=0x3fffa3b6e600) at pthread_cond_timedwait.c:199

#1  0x0000000021b777c8 in os_event::timed_wait (this=0x10033f173e0, abstime=0x3fffa3b6e600) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286

#2  0x0000000021b77c04 in os_event::wait_time_low (this=0x10033f173e0, time_in_usec=1000000, reset_sig_count=46) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413

#3  0x0000000021b77f78 in os_event_wait_time_low (event=0x10033f173e0, time_in_usec=1000000, reset_sig_count=46) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518

#4  0x0000000021b30940 in lock_wait_timeout_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/lock/lock0wait.cc:529

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa3b6f180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 15 (LWP 12430):

#0  0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81

#1  0x0000000022280430 in vio_io_wait (vio=0x10033d3eec0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:945

#2  0x000000002227e098 in vio_socket_io_wait (vio=0x10033d3eec0, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:108

#3  0x000000002227e314 in vio_read (vio=0x10033d3eec0, buf=0x3fff34160ae0 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:184

#4  0x00000000212dc8c0 in my_real_read (net=0x3fff340faa08, complen=0x3fffa0246f90, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:888

#5  0x00000000212dd550 in my_net_read_packet_reallen (net=0x3fff340faa08, read_from_server=1 '\001', reallen=0x3fffa0247058) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1158

#6  0x00000000212dd4c4 in my_net_read_packet (net=0x3fff340faa08, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1142

#7  0x00000000213f6bb8 in do_command (thd=0x3fff340fa740) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_parse.cc:1244

#8  0x00000000215b01c8 in do_handle_one_connection (connect=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1354

#9  0x00000000215afe1c in handle_one_connection (arg=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#10 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034357b50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#11 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0248180) at pthread_create.c:312

#12 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 14 (LWP 10666):

#0  0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1

#1  0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1

#2  0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa557e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886

#3  0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa557e370, m1=0x3fffa557e4d0, m2=0x3fffa557e4d8, request=0x3fffa557e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031

#4  0x0000000021b65d38 in os_aio_linux_handler (global_segment=4, m1=0x3fffa557e4d0, m2=0x3fffa557e4d8, request=0x3fffa557e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085

#5  0x0000000021b6bea0 in os_aio_handler (segment=4, m1=0x3fffa557e4d0, m2=0x3fffa557e4d8, request=0x3fffa557e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683

#6  0x0000000021e93564 in fil_aio_wait (segment=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480

#7  0x0000000021cc2454 in io_handler_thread (arg=0x22d8bce8 <n+32>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa557f180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 13 (LWP 10723):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fffa7727470) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa7727470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa7727470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x00000000212c60d0 in cache_thread (thd=0x3fff5c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949

#4  0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff5c000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038

#5  0x00000000215b0300 in do_handle_one_connection (connect=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373

#6  0x00000000215afe1c in handle_one_connection (arg=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#7  0x000000002215e7b8 in pfs_spawn_thread (arg=0x100343578e0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa7728180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 12 (LWP 10662):

#0  0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1

#1  0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1

#2  0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa757e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886

#3  0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa757e370, m1=0x3fffa757e4d0, m2=0x3fffa757e4d8, request=0x3fffa757e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031

#4  0x0000000021b65d38 in os_aio_linux_handler (global_segment=0, m1=0x3fffa757e4d0, m2=0x3fffa757e4d8, request=0x3fffa757e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085

#5  0x0000000021b6bea0 in os_aio_handler (segment=0, m1=0x3fffa757e4d0, m2=0x3fffa757e4d8, request=0x3fffa757e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683

#6  0x0000000021e93564 in fil_aio_wait (segment=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480

#7  0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcc8 <n>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa757f180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 11 (LWP 15645):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10034354590, mutex=0x10034353e30) at pthread_cond_wait.c:188

#1  0x00000000221ea3c8 in safe_cond_wait (cond=0x10034354590, mp=0x10034353e08, file=0x223e5a18 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491

#2  0x00000000218d7d78 in inline_mysql_cond_wait (that=0x10034354590, mutex=0x10034353e08, src_file=0x223e6060 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc", src_line=8054) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138

#3  0x00000000218f460c in MYSQL_BIN_LOG::wait_for_update_relay_log (this=0x10034353e00, thd=0x3fff341e4860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc:8054

#4  0x000000002131d590 in next_event (rgi=0x3fff34044c00, event_size=0x3fff6f7f68a8) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:7150

#5  0x0000000021312b00 in exec_relay_log_event (thd=0x3fff341e4860, rli=0x10034353928, serial_rgi=0x3fff34044c00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:3864

#6  0x0000000021316db8 in handle_slave_sql (arg=0x10034351c50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:5147

#7  0x000000002215e7b8 in pfs_spawn_thread (arg=0x3fff3402f320) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6f7f8180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 10 (LWP 10714):

#0  0x00003fffb1fa9554 in do_sigwait (sig=0x3fffac0575e0, set=<optimized out>) at ../sysdeps/unix/sysv/linux/sigwait.c:61

#1  __sigwait (set=<optimized out>, sig=0x3fffac0575e0) at ../sysdeps/unix/sysv/linux/sigwait.c:98

#2  0x00000000212c7304 in signal_hand (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3487

#3  0x000000002215e7b8 in pfs_spawn_thread (arg=0x100342909e0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#4  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffac058180) at pthread_create.c:312

#5  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 9 (LWP 14308):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fff6f617470) at pthread_cond_timedwait.c:199

#1  0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f617470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545

#2  0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f617470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175

#3  0x00000000212c60d0 in cache_thread (thd=0x3fff34137e70) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949

#4  0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff34137e70, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038

#5  0x00000000215b0300 in do_handle_one_connection (connect=0x1003434ca90) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373

#6  0x00000000215afe1c in handle_one_connection (arg=0x1003434ca90) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#7  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3f9a0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#8  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6f618180) at pthread_create.c:312

#9  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 8 (LWP 10777):

#0  0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81

#1  0x0000000022280430 in vio_io_wait (vio=0x10033d3f9a0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:945

#2  0x000000002227e098 in vio_socket_io_wait (vio=0x10033d3f9a0, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:108

#3  0x000000002227e314 in vio_read (vio=0x10033d3f9a0, buf=0x3fff4000ca70 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:184

#4  0x00000000212dc8c0 in my_real_read (net=0x3fff40000dc8, complen=0x3fffa0306f90, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:888

#5  0x00000000212dd550 in my_net_read_packet_reallen (net=0x3fff40000dc8, read_from_server=1 '\001', reallen=0x3fffa0307058) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1158

#6  0x00000000212dd4c4 in my_net_read_packet (net=0x3fff40000dc8, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1142

#7  0x00000000213f6bb8 in do_command (thd=0x3fff40000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_parse.cc:1244

#8  0x00000000215b01c8 in do_handle_one_connection (connect=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1354

#9  0x00000000215afe1c in handle_one_connection (arg=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260

#10 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3f260) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#11 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0308180) at pthread_create.c:312

#12 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 7 (LWP 10704):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x3fff98ffe5a0, mutex=0x3fff98ffe600) at pthread_cond_wait.c:188

#1  0x00000000221ea3c8 in safe_cond_wait (cond=0x3fff98ffe5a0, mp=0x3fff98ffe5d8, file=0x22410710 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1149) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491

#2  0x0000000021a57268 in inline_mysql_cond_wait (that=0x3fff98ffe5a0, mutex=0x3fff98ffe5d8, src_file=0x224125d8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc", src_line=338) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1149

#3  0x0000000021a5ecac in thd_destructor_proxy () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:338

#4  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033f79ca0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff98fff180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 6 (LWP 10712):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x22d37448 <mysql_bin_log+4216>, mutex=0x22d373c0 <mysql_bin_log+4080>) at pthread_cond_wait.c:188

#1  0x00000000221ea3c8 in safe_cond_wait (cond=0x22d37448 <mysql_bin_log+4216>, mp=0x22d37398 <mysql_bin_log+4040>, file=0x223e5a18 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491

#2  0x00000000218d7d78 in inline_mysql_cond_wait (that=0x22d37448 <mysql_bin_log+4216>, mutex=0x22d37398 <mysql_bin_log+4040>, src_file=0x223e6060 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc", src_line=9607) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138

#3  0x00000000218f9ae0 in binlog_background_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc:9607

#4  0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034262b20) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fffac0b8180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 5 (LWP 10702):

#0  0x00003fffb1fa8984 in .__libc_nanosleep () at ../sysdeps/unix/syscall-template.S:81

#1  0x0000000021b78ab0 in os_thread_sleep (tm=1000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0thread.cc:245

#2  0x0000000021db9630 in btr_defragment_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/btr/btr0defragment.cc:759

#3  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff99fff180) at pthread_create.c:312

#4  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 4 (LWP 10701):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033f79670, mutex=0x10033f79648, abstime=0x3fff9a7fe3b0) at pthread_cond_timedwait.c:199

#1  0x0000000021b777c8 in os_event::timed_wait (this=0x10033f79630, abstime=0x3fff9a7fe3b0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286

#2  0x0000000021b77c04 in os_event::wait_time_low (this=0x10033f79630, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413

#3  0x0000000021b77f78 in os_event_wait_time_low (event=0x10033f79630, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518

#4  0x0000000021d51608 in ib_wqueue_timedwait (wq=0x10033f79520, wait_in_usecs=5000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/ut/ut0wqueue.cc:163

#5  0x0000000021ef1e30 in fts_optimize_thread (arg=0x10033f79520) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fts/fts0opt.cc:3041

#6  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9a7ff180) at pthread_create.c:312

#7  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 3 (LWP 10700):

#0  0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033f2ab80, mutex=0x10033f2ab58, abstime=0x3fff9affe630) at pthread_cond_timedwait.c:199

#1  0x0000000021b777c8 in os_event::timed_wait (this=0x10033f2ab40, abstime=0x3fff9affe630) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286

#2  0x0000000021b77c04 in os_event::wait_time_low (this=0x10033f2ab40, time_in_usec=10000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413

#3  0x0000000021b77f78 in os_event_wait_time_low (event=0x10033f2ab40, time_in_usec=10000000, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518

#4  0x0000000021e7119c in dict_stats_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/dict/dict0stats_bg.cc:420

#5  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9afff180) at pthread_create.c:312

#6  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 2 (LWP 10689):

#0  0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db7170, mutex=0x10033db7148) at pthread_cond_wait.c:188

#1  0x0000000021b7845c in os_event::wait (this=0x10033db7130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166

#2  0x0000000021b779b4 in os_event::wait_low (this=0x10033db7130, reset_sig_count=2626) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336

#3  0x0000000021b77fc8 in os_event_wait_low (event=0x10033db7130, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535

#4  0x0000000021cb73a8 in srv_resume_thread (slot=0x10033db6e08, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:954

#5  0x0000000021cbcc04 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2710

#6  0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9bfff180) at pthread_create.c:312

#7  0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Thread 1 (LWP 10687):

#0  0x00003fffb1fa4f00 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60

#1  0x00000000221e2a88 in my_write_core (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/stacktrace.c:477

#2  0x0000000021752e18 in handle_fatal_signal (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/signal_handler.cc:299

#3  <signal handler called>

#4  internal_fnmatch (pattern=<optimized out>, string=string@entry=0x222db6a0 "THD::enter_stage", string_end=0x222db6b0 "", no_leading_period=no_leading_period@entry=0, flags=flags@entry=0, ends=ends@entry=0x0, alloca_used=alloca_used@entry=0) at fnmatch_loop.c:62

#5  0x00003fffb17329c8 in __fnmatch (pattern=0xd9 <Address 0xd9 out of bounds>, string=0x222db6a0 "THD::enter_stage", flags=<optimized out>) at fnmatch.c:458

#6  0x000000002221b120 in InList (linkp=0xd0, cp=0x222db6a0 "THD::enter_stage", exact_match=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1515

#7  0x000000002221b88c in _db_keyword_ (cs=0x3fff8c000a30, keyword=0x222db6a0 "THD::enter_stage", strict=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1717

#8  0x000000002221a5f4 in _db_pargs_ (_line_=2175, keyword=0x222db6a0 "THD::enter_stage") at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1236

#9  0x00000000212d9a14 in THD::enter_stage (this=0x3fff8c000b00, stage=0x22a011d0 <stage_closing_tables>, calling_func=0x222f72e8 <close_thread_tables(THD*)::__FUNCTION__> "close_thread_tables", calling_file=0x222f55f8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc", calling_line=714) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.h:2175

#10 0x000000002135d54c in close_thread_tables (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc:714

#11 0x00000000213a2dac in reset_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.cc:4510

#12 0x0000000021a5fb84 in innobase_reset_background_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:1751

#13 0x0000000021c5270c in row_purge_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/row/row0purge.cc:1100

#14 0x0000000021bb858c in que_thr_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1052

#15 0x0000000021bb8880 in que_run_threads_low (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1114

#16 0x0000000021bb8ae8 in que_run_threads (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1154

#17 0x0000000021cbca10 in srv_task_execute () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2667

#18 0x0000000021cbcc08 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2712

#19 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa136f180) at pthread_create.c:312

#20 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Elena Stepanova added a comment - 2017-05-09 11:33 10.2 variation: rpl.rpl_parallel_retry 'innodb,row' w2 [ fail ] Test ended at 2017-05-06 12:14:59 CURRENT_TEST: rpl.rpl_parallel_retry mysqltest: In included file "./include/wait_for_slave_param.inc": included from ./include/wait_for_slave_io_to_start.inc at line 40: included from ./include/wait_for_slave_to_start.inc at line 26: included from ./include/start_slave.inc at line 35: included from /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 379: At line 119: query 'let $_show_slave_status_value= query_get_value("SHOW SLAVE STATUS", $slave_param, 1)' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > SET debug_sync='now WAIT_FOR t3_waiting'; SET debug_sync='now SIGNAL t1_start'; SET GLOBAL debug_dbug=@old_dbug; SET debug_sync='RESET'; retries 1 SELECT * FROM t3 ORDER BY a; a b 1 NULL 3 NULL 4 4 5 NULL 6 NULL 7 7 connection server_1; SET binlog_format=@old_format; connection server_2; include/stop_slave.inc SET GLOBAL slave_parallel_threads=@old_parallel_threads; include/start_slave.inc More results from queries before failure can be found in /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/2/log/rpl_parallel_retry.log Server [mysqld.2 - pid: 10609, winpid: 10609, exit: 256] failed during test run Server log from this test: ----------SERVER LOG START----------- 2017-05-06 12:14:55 70367135957376 [Note] Deleted Master_info file '/dev/shm/var_auto_fqAr/2/mysqld.2/data/master.info'. 2017-05-06 12:14:55 70367135957376 [Note] Deleted Master_info file '/dev/shm/var_auto_fqAr/2/mysqld.2/data/relay-log.info'. 2017-05-06 12:14:55 70367135957376 [Note] Master connection name: '' Master_info_file: 'master.info' Relay_info_file: 'relay-log.info' 2017-05-06 12:14:55 70367135957376 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='4'. 2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 4 2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4 2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 484 2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860 2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860 2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 484, relay log './slave-relay-bin.000002' position: 784 2017-05-06 12:14:55 70366319837568 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:55 70366319837568 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 860 2017-05-06 12:14:55 70367135170944 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860 2017-05-06 12:14:55 70366319837568 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860 2017-05-06 12:14:55 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 860, relay log './slave-relay-bin.000003' position: 556 2017-05-06 12:14:55 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:55 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1359 2017-05-06 12:14:55 70366319837568 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1359 2017-05-06 12:14:55 70366318657920 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1359 2017-05-06 12:14:55 70367257723264 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1359, relay log './slave-relay-bin.000004' position: 1055 2017-05-06 12:14:55 70367257723264 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:55 70367257723264 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996 2017-05-06 12:14:55 70366318657920 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1996 2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1996 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000005' position: 1193 2017-05-06 12:14:56 70367135170944 [ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up. Consider raising the value of the slave_transaction_retries variable. 2017-05-06 12:14:56 70367135170944 [ERROR] Slave SQL: Deadlock found when trying to get lock; try restarting transaction, Gtid 0-11-100, Internal MariaDB error code: 1213 2017-05-06 12:14:56 70367135170944 [Warning] Slave: Deadlock found when trying to get lock; try restarting transaction Error_code: 1213 2017-05-06 12:14:56 70367135170944 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'master-bin.000001' position 1996 2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996 2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 2633 2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 2633 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000006' position: 556 2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 3028 2017-05-06 12:14:56 70367257723264 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 3028 2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 3028 2017-05-06 12:14:56 70367135170944 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 3028, relay log './slave-relay-bin.000007' position: 951 2017-05-06 12:14:56 70367135170944 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:56 70367135170944 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34004 2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 34004 2017-05-06 12:14:56 70367135170944 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 34004 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34004, relay log './slave-relay-bin.000014' position: 539 2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34641 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34641, relay log './slave-relay-bin.000015' position: 1193 2017-05-06 12:14:56 70366318657920 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:56 70366318657920 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 36559 2017-05-06 12:14:56 70367135170944 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 36559 2017-05-06 12:14:56 70367135957376 [Note] Master connection name: '' Master_info_file: 'master.info' Relay_info_file: 'relay-log.info' 2017-05-06 12:14:56 70367135957376 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'. 2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 36559 2017-05-06 12:14:56 70367135564160 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 36559, relay log './slave-relay-bin.000001' position: 4 170506 12:14:56 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.2.6-MariaDB-debug-log key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=7 max_threads=153 thread_count=21 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63063 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x3fff8c000b00 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x3fffa136e608 thread_stack 0x49000 2017-05-06 12:14:56 70367135564160 [Note] Error reading relay log event: slave SQL thread was killed 2017-05-06 12:14:56 70367135564160 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37580 2017-05-06 12:14:56 70366318657920 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37580 /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(my_print_stacktrace-0xb176e8)[0x221e28f0] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(handle_fatal_signal-0x14ebac8)[0x21752890] linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fffb1fd0478] /opt/at8.0/lib64/power8/libc.so.6(+0x1111e0)[0x3fffb17311e0] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0xf8279c)[0x21b6279c] /opt/at8.0/lib64/power8/libc.so.6(fnmatch-0xe6f70)[0x3fffb17329c8] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0x163b120)[0x2221b120] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_db_keyword_-0xae26ac)[0x2221b88c] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_db_pargs_-0xae380c)[0x2221a5f4] 2017-05-06 12:14:56 70367135564160 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 37580 2017-05-06 12:14:56 70366319837568 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 37580, relay log './slave-relay-bin.000002' position: 1577 /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_ZN3THD11enter_stageEPK17PSI_stage_info_v1PKcS4_j-0x191ef9c)[0x212d9a14] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_Z19close_thread_tablesP3THD-0x18a68dc)[0x2135d54c] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(_Z9reset_thdP3THD-0x1865d5c)[0x213a2dac] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0xe7fb84)[0x21a5fb84] mysys/stacktrace.c:267(my_print_stacktrace)[0x21c5270c] dbug/dbug.c:1515(InList)[0x21bb858c] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0xfd8880)[0x21bb8880] sql/sql_base.cc:714(close_thread_tables(THD*))[0x21bb8ae8] sql/sql_class.cc:4510(reset_thd(THD*))[0x21cbca10] /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld(+0x10dcc08)[0x21cbcc08] /opt/at8.0/lib64/power8/libpthread.so.0(+0xb9e0)[0x3fffb1f9b9e0] handler/ha_innodb.cc:1751(innobase_reset_background_thd(THD*))[0x3fffb17718d0] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x0): Connection ID (thread ID): 4 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file ----------SERVER LOG END------------- - saving '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/2/log/rpl.rpl_parallel_retry-innodb,row/' to '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/log/rpl.rpl_parallel_retry-innodb,row/' - found 'core.10610' (0/5) Trying 'dbx' to get a backtrace Trying 'gdb' to get a backtrace from coredump /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysql-test/var/log/rpl.rpl_parallel_retry-innodb,row/mysqld.2/data/core.10610 Core generated by '/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -------------------------- [New LWP 10687] [New LWP 10689] [New LWP 10700] [New LWP 10701] [New LWP 10702] [New LWP 10712] [New LWP 10704] [New LWP 10777] [New LWP 14308] [New LWP 10714] [New LWP 15645] [New LWP 10662] [New LWP 10723] [New LWP 10666] [New LWP 12430] [New LWP 10682] [New LWP 10686] [New LWP 10610] [New LWP 10663] [New LWP 10665] [New LWP 10667] [New LWP 10683] [New LWP 10668] [New LWP 10664] [New LWP 15644] [New LWP 10685] [New LWP 10684] [New LWP 10661] [New LWP 10726] [New LWP 14279] [New LWP 10716] [New LWP 10776] [New LWP 10703] [New LWP 10705] [New LWP 10698] [New LWP 10688] warning: Could not load shared library symbols for linux-vdso64.so.1. Do you need "set solib-search-path" or "set sysroot"? warning: File "/opt/at8.0/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py". To enable execution of this file add add-auto-load-safe-path /opt/at8.0/lib64/power8/libthread_db-1.0.so line to your configuration file "/home/buildbot/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/home/buildbot/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. warning: File "/opt/at8.0/lib64/power8/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py". warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Core was generated by `/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld --default'. Program terminated with signal 11, Segmentation fault. #0 0x00003fffb1fa4f00 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60 60 ../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory. #0 0x00003fffb1fa4f00 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60 #1 0x00000000221e2a88 in my_write_core (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/stacktrace.c:477 #2 0x0000000021752e18 in handle_fatal_signal (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/signal_handler.cc:299 #3 <signal handler called> #4 internal_fnmatch (pattern=<optimized out>, string=string@entry=0x222db6a0 "THD::enter_stage", string_end=0x222db6b0 "", no_leading_period=no_leading_period@entry=0, flags=flags@entry=0, ends=ends@entry=0x0, alloca_used=alloca_used@entry=0) at fnmatch_loop.c:62 #5 0x00003fffb17329c8 in __fnmatch (pattern=0xd9 <Address 0xd9 out of bounds>, string=0x222db6a0 "THD::enter_stage", flags=<optimized out>) at fnmatch.c:458 #6 0x000000002221b120 in InList (linkp=0xd0, cp=0x222db6a0 "THD::enter_stage", exact_match=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1515 #7 0x000000002221b88c in _db_keyword_ (cs=0x3fff8c000a30, keyword=0x222db6a0 "THD::enter_stage", strict=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1717 #8 0x000000002221a5f4 in _db_pargs_ (_line_=2175, keyword=0x222db6a0 "THD::enter_stage") at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1236 #9 0x00000000212d9a14 in THD::enter_stage (this=0x3fff8c000b00, stage=0x22a011d0 <stage_closing_tables>, calling_func=0x222f72e8 <close_thread_tables(THD*)::__FUNCTION__> "close_thread_tables", calling_file=0x222f55f8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc", calling_line=714) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.h:2175 #10 0x000000002135d54c in close_thread_tables (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc:714 #11 0x00000000213a2dac in reset_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.cc:4510 #12 0x0000000021a5fb84 in innobase_reset_background_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:1751 #13 0x0000000021c5270c in row_purge_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/row/row0purge.cc:1100 #14 0x0000000021bb858c in que_thr_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1052 #15 0x0000000021bb8880 in que_run_threads_low (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1114 #16 0x0000000021bb8ae8 in que_run_threads (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1154 #17 0x0000000021cbca10 in srv_task_execute () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2667 #18 0x0000000021cbcc08 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2712 #19 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa136f180) at pthread_create.c:312 #20 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 36 (LWP 10688): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db70f0, mutex=0x10033db70c8) at pthread_cond_wait.c:188 #1 0x0000000021b7845c in os_event::wait (this=0x10033db70b0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166 #2 0x0000000021b779b4 in os_event::wait_low (this=0x10033db70b0, reset_sig_count=3183) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336 #3 0x0000000021b77fc8 in os_event_wait_low (event=0x10033db70b0, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535 #4 0x0000000021cb73a8 in srv_resume_thread (slot=0x10033db6dd0, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:954 #5 0x0000000021cbcc04 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2710 #6 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0b6f180) at pthread_create.c:312 #7 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 35 (LWP 10698): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db7370, mutex=0x10033db7348) at pthread_cond_wait.c:188 #1 0x0000000021b7845c in os_event::wait (this=0x10033db7330) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166 #2 0x0000000021b779b4 in os_event::wait_low (this=0x10033db7330, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336 #3 0x0000000021b77fc8 in os_event_wait_low (event=0x10033db7330, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535 #4 0x0000000021de7f68 in buf_dump_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0dump.cc:815 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9b7ff180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 34 (LWP 10705): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x235a4e90 <COND_checkpoint>, mutex=0x235a4e08 <LOCK_checkpoint+40>, abstime=0x3fff6fffe570) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x235a4e90 <COND_checkpoint>, mp=0x235a4de0 <LOCK_checkpoint>, abstime=0x3fff6fffe570, file=0x2256da08 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x0000000021fbef9c in inline_mysql_cond_timedwait (that=0x235a4e90 <COND_checkpoint>, mutex=0x235a4de0 <LOCK_checkpoint>, abstime=0x3fff6fffe570, src_file=0x2256da88 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x0000000021fbf6f8 in my_service_thread_sleep (control=0x22a3a728 <checkpoint_control>, sleep_time=30000000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/maria/ma_servicethread.c:115 #4 0x0000000021fae41c in ma_checkpoint_background (arg=0x1e) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/maria/ma_checkpoint.c:708 #5 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034258330) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #6 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6ffff180) at pthread_create.c:312 #7 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 33 (LWP 10703): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db7470, mutex=0x10033db7448) at pthread_cond_wait.c:188 #1 0x0000000021b7845c in os_event::wait (this=0x10033db7430) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166 #2 0x0000000021b779b4 in os_event::wait_low (this=0x10033db7430, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336 #3 0x0000000021b77fc8 in os_event_wait_low (event=0x10033db7430, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535 #4 0x0000000021dcacac in buf_resize_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0buf.cc:3063 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff997ff180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 32 (LWP 10776): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fffa0367470) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa0367470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa0367470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x00000000212c60d0 in cache_thread (thd=0x3fff4c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949 #4 0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff4c000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038 #5 0x00000000215b0300 in do_handle_one_connection (connect=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373 #6 0x00000000215afe1c in handle_one_connection (arg=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #7 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3eec0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0368180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 31 (LWP 10716): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x22d08888 <COND_slave_background>, mutex=0x22d0b8f8 <LOCK_slave_background+40>) at pthread_cond_wait.c:188 #1 0x00000000221ea3c8 in safe_cond_wait (cond=0x22d08888 <COND_slave_background>, mp=0x22d0b8d0 <LOCK_slave_background>, file=0x222e98d8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491 #2 0x000000002130441c in inline_mysql_cond_wait (that=0x22d08888 <COND_slave_background>, mutex=0x22d0b8d0 <LOCK_slave_background>, src_file=0x222ea310 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc", src_line=334) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138 #3 0x0000000021305b48 in handle_slave_background (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:334 #4 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034312a50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa7788180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 30 (LWP 14279): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fff6f677470) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f677470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f677470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x00000000212c60d0 in cache_thread (thd=0x3fff34215bf0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949 #4 0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff34215bf0, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038 #5 0x00000000215b0300 in do_handle_one_connection (connect=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373 #6 0x00000000215afe1c in handle_one_connection (arg=0x100342b3860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #7 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3f600) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6f678180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 29 (LWP 10726): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fffa76c7470) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa76c7470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa76c7470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x00000000212c60d0 in cache_thread (thd=0x3fff50000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949 #4 0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff50000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038 #5 0x00000000215b0300 in do_handle_one_connection (connect=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373 #6 0x00000000215afe1c in handle_one_connection (arg=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #7 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034357b50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa76c8180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 28 (LWP 10661): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x235b6728 <COND_timer>, mutex=0x235b66a0 <LOCK_timer+40>, abstime=0x3fffad18e680) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x235b6728 <COND_timer>, mp=0x235b6678 <LOCK_timer>, abstime=0x3fffad18e680, file=0x225ad0f0 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x00000000221ec068 in inline_mysql_cond_timedwait (that=0x235b6728 <COND_timer>, mutex=0x235b6678 <LOCK_timer>, abstime=0x3fffad18e680, src_file=0x225ad160 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_timer.c", src_line=292) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x00000000221ed094 in timer_handler (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_timer.c:292 #4 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3c950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffad18f180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 27 (LWP 10684): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033db72f0, mutex=0x10033db72c8, abstime=0x3fffa2b6e5d0) at pthread_cond_timedwait.c:199 #1 0x0000000021b777c8 in os_event::timed_wait (this=0x10033db72b0, abstime=0x3fffa2b6e5d0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286 #2 0x0000000021b77c04 in os_event::wait_time_low (this=0x10033db72b0, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413 #3 0x0000000021b77f78 in os_event_wait_time_low (event=0x10033db72b0, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518 #4 0x0000000021cb9e54 in srv_monitor_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:1775 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa2b6f180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 26 (LWP 10685): #0 0x00003fffb1fa8984 in .__libc_nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x0000000021b78ab0 in os_thread_sleep (tm=1000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0thread.cc:245 #2 0x0000000021cbc260 in srv_master_sleep () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2521 #3 0x0000000021cbc484 in srv_master_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2568 #4 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa236f180) at pthread_create.c:312 #5 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 25 (LWP 15644): #0 0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x0000000022280430 in vio_io_wait (vio=0x3fff34146d20, event=VIO_IO_EVENT_READ, timeout=120000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:945 #2 0x000000002227e098 in vio_socket_io_wait (vio=0x3fff34146d20, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:108 #3 0x000000002227e314 in vio_read (vio=0x3fff34146d20, buf=0x3fff3413eaf0 "1", size=16384) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:184 #4 0x000000002227e5e8 in vio_read_buff (vio=0x3fff34146d20, buf=0x3fff342e6910 "", size=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:232 #5 0x00000000212dc8c0 in my_real_read (net=0x3fff34074c40, complen=0x3fffa01e7300, header=0 '\000') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:888 #6 0x00000000212dd550 in my_net_read_packet_reallen (net=0x3fff34074c40, read_from_server=0 '\000', reallen=0x3fffa01e7568) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1158 #7 0x00000000216ed060 in cli_safe_read_reallen (mysql=0x3fff34074c40, reallen=0x3fffa01e7568) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql-common/client.c:587 #8 0x0000000021311660 in read_event (mysql=0x3fff34074c40, mi=0x10034351c50, suppress_warnings=0x3fffa01e7541, network_read_len=0x3fffa01e7568) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:3344 #9 0x0000000021314964 in handle_slave_io (arg=0x10034351c50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:4456 #10 0x000000002215e7b8 in pfs_spawn_thread (arg=0x3fff34146d20) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #11 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa01e8180) at pthread_create.c:312 #12 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 24 (LWP 10664): #0 0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1 #2 0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa657e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886 #3 0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa657e370, m1=0x3fffa657e4d0, m2=0x3fffa657e4d8, request=0x3fffa657e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031 #4 0x0000000021b65d38 in os_aio_linux_handler (global_segment=2, m1=0x3fffa657e4d0, m2=0x3fffa657e4d8, request=0x3fffa657e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085 #5 0x0000000021b6bea0 in os_aio_handler (segment=2, m1=0x3fffa657e4d0, m2=0x3fffa657e4d8, request=0x3fffa657e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683 #6 0x0000000021e93564 in fil_aio_wait (segment=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480 #7 0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcd8 <n+16>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa657f180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 23 (LWP 10668): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033db73f0, mutex=0x10033db73c8, abstime=0x3fffa457e370) at pthread_cond_timedwait.c:199 #1 0x0000000021b777c8 in os_event::timed_wait (this=0x10033db73b0, abstime=0x3fffa457e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286 #2 0x0000000021b77c04 in os_event::wait_time_low (this=0x10033db73b0, time_in_usec=983000, reset_sig_count=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413 #3 0x0000000021b77f78 in os_event_wait_time_low (event=0x10033db73b0, time_in_usec=983000, reset_sig_count=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518 #4 0x0000000021df4690 in pc_sleep_if_needed (next_loop_time=1494087297614, sig_count=2) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0flu.cc:2711 #5 0x0000000021df5f90 in buf_flush_page_cleaner_coordinator (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/buf/buf0flu.cc:3213 #6 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa457f180) at pthread_create.c:312 #7 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 22 (LWP 10683): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033db7270, mutex=0x10033db7248, abstime=0x3fffa336e470) at pthread_cond_timedwait.c:199 #1 0x0000000021b777c8 in os_event::timed_wait (this=0x10033db7230, abstime=0x3fffa336e470) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286 #2 0x0000000021b77c04 in os_event::wait_time_low (this=0x10033db7230, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413 #3 0x0000000021b77f78 in os_event_wait_time_low (event=0x10033db7230, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518 #4 0x0000000021cba780 in srv_error_monitor_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2000 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa336f180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 21 (LWP 10667): #0 0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1 #2 0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa4d7e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886 #3 0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa4d7e370, m1=0x3fffa4d7e4d0, m2=0x3fffa4d7e4d8, request=0x3fffa4d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031 #4 0x0000000021b65d38 in os_aio_linux_handler (global_segment=5, m1=0x3fffa4d7e4d0, m2=0x3fffa4d7e4d8, request=0x3fffa4d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085 #5 0x0000000021b6bea0 in os_aio_handler (segment=5, m1=0x3fffa4d7e4d0, m2=0x3fffa4d7e4d8, request=0x3fffa4d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683 #6 0x0000000021e93564 in fil_aio_wait (segment=5) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480 #7 0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcf0 <n+40>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa4d7f180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 20 (LWP 10665): #0 0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1 #2 0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa5d7e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886 #3 0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa5d7e370, m1=0x3fffa5d7e4d0, m2=0x3fffa5d7e4d8, request=0x3fffa5d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031 #4 0x0000000021b65d38 in os_aio_linux_handler (global_segment=3, m1=0x3fffa5d7e4d0, m2=0x3fffa5d7e4d8, request=0x3fffa5d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085 #5 0x0000000021b6bea0 in os_aio_handler (segment=3, m1=0x3fffa5d7e4d0, m2=0x3fffa5d7e4d8, request=0x3fffa5d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683 #6 0x0000000021e93564 in fil_aio_wait (segment=3) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480 #7 0x0000000021cc2454 in io_handler_thread (arg=0x22d8bce0 <n+24>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa5d7f180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 19 (LWP 10663): #0 0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1 #2 0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa6d7e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886 #3 0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa6d7e370, m1=0x3fffa6d7e4d0, m2=0x3fffa6d7e4d8, request=0x3fffa6d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031 #4 0x0000000021b65d38 in os_aio_linux_handler (global_segment=1, m1=0x3fffa6d7e4d0, m2=0x3fffa6d7e4d8, request=0x3fffa6d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085 #5 0x0000000021b6bea0 in os_aio_handler (segment=1, m1=0x3fffa6d7e4d0, m2=0x3fffa6d7e4d8, request=0x3fffa6d7e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683 #6 0x0000000021e93564 in fil_aio_wait (segment=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480 #7 0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcd0 <n+8>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa6d7f180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 18 (LWP 10610): #0 0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00000000212cf5a8 in handle_connections_sockets () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:6556 #2 0x00000000212ce35c in mysqld_main (argc=155, argv=0x10033ba8750) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:6022 #3 0x00000000212bd328 in main (argc=24, argv=0x3fffc21e2338) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/main.cc:25 Thread 17 (LWP 10686): #0 .__sched_yield () at ../sysdeps/unix/syscall-template.S:82 #1 0x0000000021b78a20 in os_thread_yield () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0thread.cc:226 #2 0x0000000021d01b7c in trx_purge_wait_for_workers_to_complete (purge_sys=0x10033f61b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/trx/trx0purge.cc:1618 #3 0x0000000021d02028 in trx_purge (n_purge_threads=3, batch_size=300, truncate=false) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/trx/trx0purge.cc:1700 #4 0x0000000021cbd0bc in srv_do_purge (thd=0x3fff88000b00, n_threads=4, n_total_purged=0x3fffa1b6e720) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2821 #5 0x0000000021cbd8ac in srv_purge_coordinator_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2966 #6 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa1b6f180) at pthread_create.c:312 #7 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 16 (LWP 10682): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033f17420, mutex=0x10033f173f8, abstime=0x3fffa3b6e600) at pthread_cond_timedwait.c:199 #1 0x0000000021b777c8 in os_event::timed_wait (this=0x10033f173e0, abstime=0x3fffa3b6e600) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286 #2 0x0000000021b77c04 in os_event::wait_time_low (this=0x10033f173e0, time_in_usec=1000000, reset_sig_count=46) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413 #3 0x0000000021b77f78 in os_event_wait_time_low (event=0x10033f173e0, time_in_usec=1000000, reset_sig_count=46) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518 #4 0x0000000021b30940 in lock_wait_timeout_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/lock/lock0wait.cc:529 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa3b6f180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 15 (LWP 12430): #0 0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x0000000022280430 in vio_io_wait (vio=0x10033d3eec0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:945 #2 0x000000002227e098 in vio_socket_io_wait (vio=0x10033d3eec0, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:108 #3 0x000000002227e314 in vio_read (vio=0x10033d3eec0, buf=0x3fff34160ae0 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:184 #4 0x00000000212dc8c0 in my_real_read (net=0x3fff340faa08, complen=0x3fffa0246f90, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:888 #5 0x00000000212dd550 in my_net_read_packet_reallen (net=0x3fff340faa08, read_from_server=1 '\001', reallen=0x3fffa0247058) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1158 #6 0x00000000212dd4c4 in my_net_read_packet (net=0x3fff340faa08, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1142 #7 0x00000000213f6bb8 in do_command (thd=0x3fff340fa740) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_parse.cc:1244 #8 0x00000000215b01c8 in do_handle_one_connection (connect=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1354 #9 0x00000000215afe1c in handle_one_connection (arg=0x1003433d8c0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #10 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034357b50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #11 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0248180) at pthread_create.c:312 #12 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 14 (LWP 10666): #0 0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1 #2 0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa557e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886 #3 0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa557e370, m1=0x3fffa557e4d0, m2=0x3fffa557e4d8, request=0x3fffa557e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031 #4 0x0000000021b65d38 in os_aio_linux_handler (global_segment=4, m1=0x3fffa557e4d0, m2=0x3fffa557e4d8, request=0x3fffa557e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085 #5 0x0000000021b6bea0 in os_aio_handler (segment=4, m1=0x3fffa557e4d0, m2=0x3fffa557e4d8, request=0x3fffa557e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683 #6 0x0000000021e93564 in fil_aio_wait (segment=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480 #7 0x0000000021cc2454 in io_handler_thread (arg=0x22d8bce8 <n+32>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa557f180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 13 (LWP 10723): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fffa7727470) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa7727470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fffa7727470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x00000000212c60d0 in cache_thread (thd=0x3fff5c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949 #4 0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff5c000b00, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038 #5 0x00000000215b0300 in do_handle_one_connection (connect=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373 #6 0x00000000215afe1c in handle_one_connection (arg=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #7 0x000000002215e7b8 in pfs_spawn_thread (arg=0x100343578e0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa7728180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 12 (LWP 10662): #0 0x00003fffb1eb0998 in .__io_getevents_0_4 () from /lib64/libaio.so.1 #1 0x00003fffb1eb09f8 in .io_getevents () from /lib64/libaio.so.1 #2 0x0000000021b65584 in LinuxAIOHandler::collect (this=0x3fffa757e370) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:1886 #3 0x0000000021b65b7c in LinuxAIOHandler::poll (this=0x3fffa757e370, m1=0x3fffa757e4d0, m2=0x3fffa757e4d8, request=0x3fffa757e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2031 #4 0x0000000021b65d38 in os_aio_linux_handler (global_segment=0, m1=0x3fffa757e4d0, m2=0x3fffa757e4d8, request=0x3fffa757e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:2085 #5 0x0000000021b6bea0 in os_aio_handler (segment=0, m1=0x3fffa757e4d0, m2=0x3fffa757e4d8, request=0x3fffa757e508) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0file.cc:5683 #6 0x0000000021e93564 in fil_aio_wait (segment=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fil/fil0fil.cc:5480 #7 0x0000000021cc2454 in io_handler_thread (arg=0x22d8bcc8 <n>) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0start.cc:327 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa757f180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 11 (LWP 15645): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10034354590, mutex=0x10034353e30) at pthread_cond_wait.c:188 #1 0x00000000221ea3c8 in safe_cond_wait (cond=0x10034354590, mp=0x10034353e08, file=0x223e5a18 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491 #2 0x00000000218d7d78 in inline_mysql_cond_wait (that=0x10034354590, mutex=0x10034353e08, src_file=0x223e6060 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc", src_line=8054) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138 #3 0x00000000218f460c in MYSQL_BIN_LOG::wait_for_update_relay_log (this=0x10034353e00, thd=0x3fff341e4860) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc:8054 #4 0x000000002131d590 in next_event (rgi=0x3fff34044c00, event_size=0x3fff6f7f68a8) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:7150 #5 0x0000000021312b00 in exec_relay_log_event (thd=0x3fff341e4860, rli=0x10034353928, serial_rgi=0x3fff34044c00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:3864 #6 0x0000000021316db8 in handle_slave_sql (arg=0x10034351c50) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/slave.cc:5147 #7 0x000000002215e7b8 in pfs_spawn_thread (arg=0x3fff3402f320) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6f7f8180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 10 (LWP 10714): #0 0x00003fffb1fa9554 in do_sigwait (sig=0x3fffac0575e0, set=<optimized out>) at ../sysdeps/unix/sysv/linux/sigwait.c:61 #1 __sigwait (set=<optimized out>, sig=0x3fffac0575e0) at ../sysdeps/unix/sysv/linux/sigwait.c:98 #2 0x00000000212c7304 in signal_hand (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3487 #3 0x000000002215e7b8 in pfs_spawn_thread (arg=0x100342909e0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #4 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffac058180) at pthread_create.c:312 #5 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 9 (LWP 14308): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mutex=0x22d0af58 <LOCK_thread_cache+40>, abstime=0x3fff6f617470) at pthread_cond_timedwait.c:199 #1 0x00000000221ea798 in safe_cond_timedwait (cond=0x22d08818 <COND_thread_cache>, mp=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f617470, file=0x222db528 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1175) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:545 #2 0x00000000212be47c in inline_mysql_cond_timedwait (that=0x22d08818 <COND_thread_cache>, mutex=0x22d0af30 <LOCK_thread_cache>, abstime=0x3fff6f617470, src_file=0x222dd520 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc", src_line=2950) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1175 #3 0x00000000212c60d0 in cache_thread (thd=0x3fff34137e70) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:2949 #4 0x00000000212c64e8 in one_thread_per_connection_end (thd=0x3fff34137e70, put_in_cache=true) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/mysqld.cc:3038 #5 0x00000000215b0300 in do_handle_one_connection (connect=0x1003434ca90) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1373 #6 0x00000000215afe1c in handle_one_connection (arg=0x1003434ca90) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #7 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3f9a0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #8 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff6f618180) at pthread_create.c:312 #9 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 8 (LWP 10777): #0 0x00003fffb1761d10 in .__GI___poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x0000000022280430 in vio_io_wait (vio=0x10033d3f9a0, event=VIO_IO_EVENT_READ, timeout=28800000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:945 #2 0x000000002227e098 in vio_socket_io_wait (vio=0x10033d3f9a0, event=VIO_IO_EVENT_READ) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:108 #3 0x000000002227e314 in vio_read (vio=0x10033d3f9a0, buf=0x3fff4000ca70 "\001", size=4) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/vio/viosocket.c:184 #4 0x00000000212dc8c0 in my_real_read (net=0x3fff40000dc8, complen=0x3fffa0306f90, header=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:888 #5 0x00000000212dd550 in my_net_read_packet_reallen (net=0x3fff40000dc8, read_from_server=1 '\001', reallen=0x3fffa0307058) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1158 #6 0x00000000212dd4c4 in my_net_read_packet (net=0x3fff40000dc8, read_from_server=1 '\001') at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/net_serv.cc:1142 #7 0x00000000213f6bb8 in do_command (thd=0x3fff40000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_parse.cc:1244 #8 0x00000000215b01c8 in do_handle_one_connection (connect=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1354 #9 0x00000000215afe1c in handle_one_connection (arg=0x10034334130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_connect.cc:1260 #10 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033d3f260) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #11 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa0308180) at pthread_create.c:312 #12 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 7 (LWP 10704): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x3fff98ffe5a0, mutex=0x3fff98ffe600) at pthread_cond_wait.c:188 #1 0x00000000221ea3c8 in safe_cond_wait (cond=0x3fff98ffe5a0, mp=0x3fff98ffe5d8, file=0x22410710 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1149) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491 #2 0x0000000021a57268 in inline_mysql_cond_wait (that=0x3fff98ffe5a0, mutex=0x3fff98ffe5d8, src_file=0x224125d8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc", src_line=338) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1149 #3 0x0000000021a5ecac in thd_destructor_proxy () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:338 #4 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10033f79ca0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff98fff180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 6 (LWP 10712): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x22d37448 <mysql_bin_log+4216>, mutex=0x22d373c0 <mysql_bin_log+4080>) at pthread_cond_wait.c:188 #1 0x00000000221ea3c8 in safe_cond_wait (cond=0x22d37448 <mysql_bin_log+4216>, mp=0x22d37398 <mysql_bin_log+4040>, file=0x223e5a18 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h", line=1138) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/thr_mutex.c:491 #2 0x00000000218d7d78 in inline_mysql_cond_wait (that=0x22d37448 <mysql_bin_log+4216>, mutex=0x22d37398 <mysql_bin_log+4040>, src_file=0x223e6060 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc", src_line=9607) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/include/mysql/psi/mysql_thread.h:1138 #3 0x00000000218f9ae0 in binlog_background_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/log.cc:9607 #4 0x000000002215e7b8 in pfs_spawn_thread (arg=0x10034262b20) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/perfschema/pfs.cc:1862 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffac0b8180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 5 (LWP 10702): #0 0x00003fffb1fa8984 in .__libc_nanosleep () at ../sysdeps/unix/syscall-template.S:81 #1 0x0000000021b78ab0 in os_thread_sleep (tm=1000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0thread.cc:245 #2 0x0000000021db9630 in btr_defragment_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/btr/btr0defragment.cc:759 #3 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff99fff180) at pthread_create.c:312 #4 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 4 (LWP 10701): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033f79670, mutex=0x10033f79648, abstime=0x3fff9a7fe3b0) at pthread_cond_timedwait.c:199 #1 0x0000000021b777c8 in os_event::timed_wait (this=0x10033f79630, abstime=0x3fff9a7fe3b0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286 #2 0x0000000021b77c04 in os_event::wait_time_low (this=0x10033f79630, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413 #3 0x0000000021b77f78 in os_event_wait_time_low (event=0x10033f79630, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518 #4 0x0000000021d51608 in ib_wqueue_timedwait (wq=0x10033f79520, wait_in_usecs=5000000) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/ut/ut0wqueue.cc:163 #5 0x0000000021ef1e30 in fts_optimize_thread (arg=0x10033f79520) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/fts/fts0opt.cc:3041 #6 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9a7ff180) at pthread_create.c:312 #7 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 3 (LWP 10700): #0 0x00003fffb1fa3960 in __pthread_cond_timedwait (cond=0x10033f2ab80, mutex=0x10033f2ab58, abstime=0x3fff9affe630) at pthread_cond_timedwait.c:199 #1 0x0000000021b777c8 in os_event::timed_wait (this=0x10033f2ab40, abstime=0x3fff9affe630) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:286 #2 0x0000000021b77c04 in os_event::wait_time_low (this=0x10033f2ab40, time_in_usec=10000000, reset_sig_count=1) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:413 #3 0x0000000021b77f78 in os_event_wait_time_low (event=0x10033f2ab40, time_in_usec=10000000, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:518 #4 0x0000000021e7119c in dict_stats_thread () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/dict/dict0stats_bg.cc:420 #5 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9afff180) at pthread_create.c:312 #6 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 2 (LWP 10689): #0 0x00003fffb1fa3470 in __pthread_cond_wait (cond=0x10033db7170, mutex=0x10033db7148) at pthread_cond_wait.c:188 #1 0x0000000021b7845c in os_event::wait (this=0x10033db7130) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:166 #2 0x0000000021b779b4 in os_event::wait_low (this=0x10033db7130, reset_sig_count=2626) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:336 #3 0x0000000021b77fc8 in os_event_wait_low (event=0x10033db7130, reset_sig_count=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/os/os0event.cc:535 #4 0x0000000021cb73a8 in srv_resume_thread (slot=0x10033db6e08, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:954 #5 0x0000000021cbcc04 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2710 #6 0x00003fffb1f9b9e0 in start_thread (arg=0x3fff9bfff180) at pthread_create.c:312 #7 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96 Thread 1 (LWP 10687): #0 0x00003fffb1fa4f00 in __pthread_kill (threadid=<optimized out>, signo=<optimized out>) at ../sysdeps/unix/sysv/linux/pthread_kill.c:60 #1 0x00000000221e2a88 in my_write_core (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/mysys/stacktrace.c:477 #2 0x0000000021752e18 in handle_fatal_signal (sig=11) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/signal_handler.cc:299 #3 <signal handler called> #4 internal_fnmatch (pattern=<optimized out>, string=string@entry=0x222db6a0 "THD::enter_stage", string_end=0x222db6b0 "", no_leading_period=no_leading_period@entry=0, flags=flags@entry=0, ends=ends@entry=0x0, alloca_used=alloca_used@entry=0) at fnmatch_loop.c:62 #5 0x00003fffb17329c8 in __fnmatch (pattern=0xd9 <Address 0xd9 out of bounds>, string=0x222db6a0 "THD::enter_stage", flags=<optimized out>) at fnmatch.c:458 #6 0x000000002221b120 in InList (linkp=0xd0, cp=0x222db6a0 "THD::enter_stage", exact_match=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1515 #7 0x000000002221b88c in _db_keyword_ (cs=0x3fff8c000a30, keyword=0x222db6a0 "THD::enter_stage", strict=0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1717 #8 0x000000002221a5f4 in _db_pargs_ (_line_=2175, keyword=0x222db6a0 "THD::enter_stage") at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/dbug/dbug.c:1236 #9 0x00000000212d9a14 in THD::enter_stage (this=0x3fff8c000b00, stage=0x22a011d0 <stage_closing_tables>, calling_func=0x222f72e8 <close_thread_tables(THD*)::__FUNCTION__> "close_thread_tables", calling_file=0x222f55f8 "/home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc", calling_line=714) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.h:2175 #10 0x000000002135d54c in close_thread_tables (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_base.cc:714 #11 0x00000000213a2dac in reset_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/sql/sql_class.cc:4510 #12 0x0000000021a5fb84 in innobase_reset_background_thd (thd=0x3fff8c000b00) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/handler/ha_innodb.cc:1751 #13 0x0000000021c5270c in row_purge_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/row/row0purge.cc:1100 #14 0x0000000021bb858c in que_thr_step (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1052 #15 0x0000000021bb8880 in que_run_threads_low (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1114 #16 0x0000000021bb8ae8 in que_run_threads (thr=0x10033f62348) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/que/que0que.cc:1154 #17 0x0000000021cbca10 in srv_task_execute () at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2667 #18 0x0000000021cbcc08 in srv_worker_thread (arg=0x0) at /home/buildbot/maria-slave/power8-vlp03-bintar-debug/build/storage/innobase/srv/srv0srv.cc:2712 #19 0x00003fffb1f9b9e0 in start_thread (arg=0x3fffa136f180) at pthread_create.c:312 #20 0x00003fffb17718d0 in .__clone () at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:96

Alice Sherepa added a comment - 2017-08-17 07:41

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9232/steps/test_2/logs/stdio

rpl.rpl_parallel_retry 'innodb,row'      w2 [ fail ]

        Test ended at 2017-08-16 23:55:13

CURRENT_TEST: rpl.rpl_parallel_retry

mysqltest: In included file "./include/wait_for_slave_param.inc":

included from ./include/wait_for_slave_io_to_start.inc at line 40:

included from ./include/wait_for_slave_to_start.inc at line 26:

included from ./include/start_slave.inc at line 35:

included from /mnt/buildbot/build/mariadb-10.2.8/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 379:

At line 75: query 'let $_slave_check_configured= query_get_value("SHOW SLAVE STATUS", Slave_IO_Running, 1)' failed: 2013: Lost connection to MySQL server during query

The result from queries just before the failure was:

< snip >

SET debug_sync='now WAIT_FOR t3_waiting';

SET debug_sync='now SIGNAL t1_start';

SET GLOBAL debug_dbug=@old_dbug;

SET debug_sync='RESET';

retries

SELECT * FROM t3 ORDER BY a;

a	b

1	NULL

3	NULL

4	4

5	NULL

6	NULL

7	7

connection server_1;

SET binlog_format=@old_format;

connection server_2;

include/stop_slave.inc

SET GLOBAL slave_parallel_threads=@old_parallel_threads;

include/start_slave.inc

More results from queries before failure can be found in /mnt/buildbot/build/mariadb-10.2.8/mysql-test/var/2/log/rpl_parallel_retry.log

Server [mysqld.2 - pid: 5675, winpid: 5675, exit: 256] failed during test run

Server log from this test:

----------SERVER LOG START-----------

2017-08-16 23:55:09 2833644352 [Note] Deleted Master_info file '/run/shm/var_auto_xu3H/2/mysqld.2/data/master.info'.

2017-08-16 23:55:09 2833644352 [Note] Deleted Master_info file '/run/shm/var_auto_xu3H/2/mysqld.2/data/relay-log.info'.

2017-08-16 23:55:09 2833644352 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'

2017-08-16 23:55:09 2833644352 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='4'.

2017-08-16 23:55:09 2834250560 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 4

2017-08-16 23:55:09 2823203648 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4

2017-08-16 23:55:10 2823203648 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 649

2017-08-16 23:55:10 2834250560 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860

2017-08-16 23:55:10 2823203648 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860

2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 649, relay log './slave-relay-bin.000002' position: 949

2017-08-16 23:55:10 2834250560 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 860

2017-08-16 23:55:10 2823203648 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860

2017-08-16 23:55:10 2834250560 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860

2017-08-16 23:55:10 2821716800 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 860, relay log './slave-relay-bin.000003' position: 556

2017-08-16 23:55:10 2821716800 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:10 2821716800 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1359

2017-08-16 23:55:10 2834250560 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1359

2017-08-16 23:55:10 2821716800 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1359

2017-08-16 23:55:10 2821110592 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1359, relay log './slave-relay-bin.000004' position: 1055

2017-08-16 23:55:10 2821110592 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:10 2821110592 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996

2017-08-16 23:55:10 2821716800 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1996

2017-08-16 23:55:10 2821110592 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1996

2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000005' position: 1193

2017-08-16 23:55:10 2896620352 [ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up. Consider raising the value of the slave_transaction_retries variable.

2017-08-16 23:55:10 2896620352 [ERROR] Slave SQL: Deadlock found when trying to get lock; try restarting transaction, Gtid 0-11-100, Internal MariaDB error code: 1213

2017-08-16 23:55:10 2896620352 [Warning] Slave: Deadlock found when trying to get lock; try restarting transaction Error_code: 1213

2017-08-16 23:55:10 2896620352 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'master-bin.000001' position 1996

2017-08-16 23:55:10 2834250560 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996

2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 2633

2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 2633

2017-08-16 23:55:11 2896620352 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000006' position: 556

2017-08-16 23:55:11 2896620352 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:11 2896620352 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 3028

2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 3028

2017-08-16 23:55:11 2896620352 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 3028

2017-08-16 23:55:11 2821110592 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 3028, relay log './slave-relay-bin.000007' position: 951

2017-08-16 23:55:11 2821110592 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:11 2821110592 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34004

2017-08-16 23:55:11 2896620352 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 34004

2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 34004

2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34004, relay log './slave-relay-bin.000014' position: 539

2017-08-16 23:55:11 2823506752 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34641

2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34641, relay log './slave-relay-bin.000015' position: 1193

2017-08-16 23:55:11 2823506752 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 36559

2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 36559

2017-08-16 23:55:11 2833644352 [Note] Master connection name: ''  Master_info_file: 'master.info'  Relay_info_file: 'relay-log.info'

2017-08-16 23:55:11 2833644352 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'.

2017-08-16 23:55:11 2834250560 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 36559

2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 36559, relay log './slave-relay-bin.000001' position: 4

170816 23:55:11 [ERROR] mysqld got signal 11 ;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help

diagnose the problem, but since we have already crashed,

something is definitely wrong and this may fail.

Server version: 10.2.8-MariaDB-debug-log

key_buffer_size=1048576

read_buffer_size=131072

max_used_connections=7

max_threads=153

thread_count=21

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61867 K  bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0xb03006f8

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

stack_bottom = 0xa78fe28c thread_stack 0x49000

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(my_print_stacktrace+0x32)[0x8cf1f92]

2017-08-16 23:55:11 2823506752 [Note] Error reading relay log event: slave SQL thread was killed

2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37580

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(handle_fatal_signal+0x3c7)[0x85a0d37]

2017-08-16 23:55:11 2834250560 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37580

[0xb775f400]

/lib/i386-linux-gnu/libc.so.6(+0xbd4a5)[0xb72ca4a5]

/lib/i386-linux-gnu/libc.so.6(fnmatch+0x69)[0xb72cbaf9]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8d140ea]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(_db_keyword_+0x79)[0x8d145c2]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(_db_pargs_+0x65)[0x8d139d8]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x899748e]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8997910]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8997ae6]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8a560d3]

/mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8a56254]

2017-08-16 23:55:12 2823506752 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 37580

/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb74e7d4c]

/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb72f7ace]

Trying to get some variables.

Some pointers may be invalid and cause the dump to abort.

Query (0x0):

Connection ID (thread ID): 3

Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains

information that should help you find out what is causing the crash.

Writing a core file

----------SERVER LOG END-------------

Alice Sherepa added a comment - 2017-08-17 07:41 http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9232/steps/test_2/logs/stdio rpl.rpl_parallel_retry 'innodb,row' w2 [ fail ] Test ended at 2017-08-16 23:55:13 CURRENT_TEST: rpl.rpl_parallel_retry mysqltest: In included file "./include/wait_for_slave_param.inc": included from ./include/wait_for_slave_io_to_start.inc at line 40: included from ./include/wait_for_slave_to_start.inc at line 26: included from ./include/start_slave.inc at line 35: included from /mnt/buildbot/build/mariadb-10.2.8/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 379: At line 75: query 'let $_slave_check_configured= query_get_value("SHOW SLAVE STATUS", Slave_IO_Running, 1)' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > SET debug_sync='now WAIT_FOR t3_waiting'; SET debug_sync='now SIGNAL t1_start'; SET GLOBAL debug_dbug=@old_dbug; SET debug_sync='RESET'; retries 1 SELECT * FROM t3 ORDER BY a; a b 1 NULL 3 NULL 4 4 5 NULL 6 NULL 7 7 connection server_1; SET binlog_format=@old_format; connection server_2; include/stop_slave.inc SET GLOBAL slave_parallel_threads=@old_parallel_threads; include/start_slave.inc More results from queries before failure can be found in /mnt/buildbot/build/mariadb-10.2.8/mysql-test/var/2/log/rpl_parallel_retry.log Server [mysqld.2 - pid: 5675, winpid: 5675, exit: 256] failed during test run Server log from this test: ----------SERVER LOG START----------- 2017-08-16 23:55:09 2833644352 [Note] Deleted Master_info file '/run/shm/var_auto_xu3H/2/mysqld.2/data/master.info'. 2017-08-16 23:55:09 2833644352 [Note] Deleted Master_info file '/run/shm/var_auto_xu3H/2/mysqld.2/data/relay-log.info'. 2017-08-16 23:55:09 2833644352 [Note] Master connection name: '' Master_info_file: 'master.info' Relay_info_file: 'relay-log.info' 2017-08-16 23:55:09 2833644352 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='4'. 2017-08-16 23:55:09 2834250560 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 4 2017-08-16 23:55:09 2823203648 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4 2017-08-16 23:55:10 2823203648 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 649 2017-08-16 23:55:10 2834250560 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860 2017-08-16 23:55:10 2823203648 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860 2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 649, relay log './slave-relay-bin.000002' position: 949 2017-08-16 23:55:10 2834250560 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 860 2017-08-16 23:55:10 2823203648 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 860 2017-08-16 23:55:10 2834250560 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 860 2017-08-16 23:55:10 2821716800 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 860, relay log './slave-relay-bin.000003' position: 556 2017-08-16 23:55:10 2821716800 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:10 2821716800 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1359 2017-08-16 23:55:10 2834250560 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1359 2017-08-16 23:55:10 2821716800 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1359 2017-08-16 23:55:10 2821110592 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1359, relay log './slave-relay-bin.000004' position: 1055 2017-08-16 23:55:10 2821110592 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:10 2821110592 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996 2017-08-16 23:55:10 2821716800 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 1996 2017-08-16 23:55:10 2821110592 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 1996 2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000005' position: 1193 2017-08-16 23:55:10 2896620352 [ERROR] Slave worker thread retried transaction 10 time(s) in vain, giving up. Consider raising the value of the slave_transaction_retries variable. 2017-08-16 23:55:10 2896620352 [ERROR] Slave SQL: Deadlock found when trying to get lock; try restarting transaction, Gtid 0-11-100, Internal MariaDB error code: 1213 2017-08-16 23:55:10 2896620352 [Warning] Slave: Deadlock found when trying to get lock; try restarting transaction Error_code: 1213 2017-08-16 23:55:10 2896620352 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'master-bin.000001' position 1996 2017-08-16 23:55:10 2834250560 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:10 2834250560 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 1996 2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 2633 2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 2633 2017-08-16 23:55:11 2896620352 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 1996, relay log './slave-relay-bin.000006' position: 556 2017-08-16 23:55:11 2896620352 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:11 2896620352 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 3028 2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 3028 2017-08-16 23:55:11 2896620352 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 3028 2017-08-16 23:55:11 2821110592 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 3028, relay log './slave-relay-bin.000007' position: 951 2017-08-16 23:55:11 2821110592 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:11 2821110592 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34004 2017-08-16 23:55:11 2896620352 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 34004 2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 34004 2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34004, relay log './slave-relay-bin.000014' position: 539 2017-08-16 23:55:11 2823506752 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 34641 2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 34641, relay log './slave-relay-bin.000015' position: 1193 2017-08-16 23:55:11 2823506752 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 36559 2017-08-16 23:55:11 2821110592 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 36559 2017-08-16 23:55:11 2833644352 [Note] Master connection name: '' Master_info_file: 'master.info' Relay_info_file: 'relay-log.info' 2017-08-16 23:55:11 2833644352 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'. New state master_host='127.0.0.1', master_port='16020', master_log_file='master-bin.000001', master_log_pos='36559'. 2017-08-16 23:55:11 2834250560 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 36559 2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 36559, relay log './slave-relay-bin.000001' position: 4 170816 23:55:11 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.2.8-MariaDB-debug-log key_buffer_size=1048576 read_buffer_size=131072 max_used_connections=7 max_threads=153 thread_count=21 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61867 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0xb03006f8 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0xa78fe28c thread_stack 0x49000 /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(my_print_stacktrace+0x32)[0x8cf1f92] 2017-08-16 23:55:11 2823506752 [Note] Error reading relay log event: slave SQL thread was killed 2017-08-16 23:55:11 2823506752 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37580 /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(handle_fatal_signal+0x3c7)[0x85a0d37] 2017-08-16 23:55:11 2834250560 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37580 [0xb775f400] /lib/i386-linux-gnu/libc.so.6(+0xbd4a5)[0xb72ca4a5] /lib/i386-linux-gnu/libc.so.6(fnmatch+0x69)[0xb72cbaf9] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8d140ea] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(_db_keyword_+0x79)[0x8d145c2] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld(_db_pargs_+0x65)[0x8d139d8] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x899748e] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8997910] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8997ae6] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8a560d3] /mnt/buildbot/build/mariadb-10.2.8/sql/mysqld[0x8a56254] 2017-08-16 23:55:12 2823506752 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16020',replication started in log 'master-bin.000001' at position 37580 /lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb74e7d4c] /lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb72f7ace] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0x0): Connection ID (thread ID): 3 Status: NOT_KILLED Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. Writing a core file ----------SERVER LOG END-------------

Alice Sherepa added a comment - 2017-09-13 07:45

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9567/steps/test_3/logs/stdio

rpl.rpl_parallel_retry 'innodb,row'      w3 [ fail ]

        Test ended at 2017-09-13 01:56:02

CURRENT_TEST: rpl.rpl_parallel_retry

mysqltest: In included file "./include/end_include_file.inc":

included from ./include/wait_for_slave_to_stop.inc at line 39:

included from ./include/stop_slave.inc at line 84:

included from ./include/rpl_for_each_slave.inc at line 33:

included from ./include/rpl_stop_slaves.inc at line 31:

included from ./include/rpl_end.inc at line 81:

included from /mnt/buildbot/build/mariadb-10.2.9/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 385:

At line 25: query 'let $_tmp= `SELECT SUBSTRING('$_include_file_enabled_warnings', 1, 1)`' failed: 2013: Lost connection to MySQL server during query

The result from queries just before the failure was:

< snip >

retries

SELECT * FROM t3 ORDER BY a;

a	b

1	NULL

3	NULL

4	4

5	NULL

6	NULL

7	7

connection server_1;

SET binlog_format=@old_format;

connection server_2;

include/stop_slave.inc

SET GLOBAL slave_parallel_threads=@old_parallel_threads;

include/start_slave.inc

connection server_1;

DROP TABLE t1, t2, t3, t4;

DROP function foo;

include/rpl_end.inc

2017-09-13  1:56:01 2806430528 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16040', master_log_file='master-bin.000001', master_log_pos='36559'. New state master_host='127.0.0.1', master_port='16040', master_log_file='master-bin.000001', master_log_pos='36559'.

2017-09-13  1:56:01 2808769344 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16040',replication started in log 'master-bin.000001' at position 36559

2017-09-13  1:56:01 2768968512 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 36559, relay log './slave-relay-bin.000001' position: 4

170913  1:56:01 [ERROR] mysqld got signal 11 ;

stack_bottom = 0xa86f228c thread_stack 0x49000

2017-09-13  1:56:01 2768968512 [Note] Error reading relay log event: slave SQL thread was killed

2017-09-13  1:56:01 2768968512 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37580

2017-09-13  1:56:01 2808769344 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37580

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(my_print_stacktrace+0x32)[0x8cf5896]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(handle_fatal_signal+0x3c7)[0x85a2edf]

[0xb7724400]

/lib/i386-linux-gnu/libc.so.6(+0xbd4a5)[0xb728f4a5]

2017-09-13  1:56:01 2768665408 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16040',replication started in log 'master-bin.000001' at position 37580

2017-09-13  1:56:01 2768968512 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 37580, relay log './slave-relay-bin.000002' position: 1577

/lib/i386-linux-gnu/libc.so.6(fnmatch+0x69)[0xb7290af9]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8d179fa]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_db_keyword_+0x79)[0x8d17ed2]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_db_pargs_+0x65)[0x8d172e8]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_ZN3THD11enter_stageEPK17PSI_stage_info_v1PKcS4_j+0x28)[0x827049a]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(set_thd_stage_info+0x6b)[0x82f19ef]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(set_thd_proc_info+0x4d)[0x82f197b]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x88a2789]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a0ce92]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899afbd]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899b1c8]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899b39e]

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a59819]

2017-09-13  1:56:01 2768968512 [Note] Error reading relay log event: slave SQL thread was killed

2017-09-13  1:56:01 2768968512 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37871

2017-09-13  1:56:01 2768665408 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37871

/mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a5999a]

/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb74acd4c]

/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb72bcace]

Alice Sherepa added a comment - 2017-09-13 07:45 http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9567/steps/test_3/logs/stdio rpl.rpl_parallel_retry 'innodb,row' w3 [ fail ] Test ended at 2017-09-13 01:56:02 CURRENT_TEST: rpl.rpl_parallel_retry mysqltest: In included file "./include/end_include_file.inc": included from ./include/wait_for_slave_to_stop.inc at line 39: included from ./include/stop_slave.inc at line 84: included from ./include/rpl_for_each_slave.inc at line 33: included from ./include/rpl_stop_slaves.inc at line 31: included from ./include/rpl_end.inc at line 81: included from /mnt/buildbot/build/mariadb-10.2.9/mysql-test/suite/rpl/t/rpl_parallel_retry.test at line 385: At line 25: query 'let $_tmp= `SELECT SUBSTRING('$_include_file_enabled_warnings', 1, 1)`' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > retries 1 SELECT * FROM t3 ORDER BY a; a b 1 NULL 3 NULL 4 4 5 NULL 6 NULL 7 7 connection server_1; SET binlog_format=@old_format; connection server_2; include/stop_slave.inc SET GLOBAL slave_parallel_threads=@old_parallel_threads; include/start_slave.inc connection server_1; DROP TABLE t1, t2, t3, t4; DROP function foo; include/rpl_end.inc 2017-09-13 1:56:01 2806430528 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16040', master_log_file='master-bin.000001', master_log_pos='36559'. New state master_host='127.0.0.1', master_port='16040', master_log_file='master-bin.000001', master_log_pos='36559'. 2017-09-13 1:56:01 2808769344 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16040',replication started in log 'master-bin.000001' at position 36559 2017-09-13 1:56:01 2768968512 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 36559, relay log './slave-relay-bin.000001' position: 4 170913 1:56:01 [ERROR] mysqld got signal 11 ; stack_bottom = 0xa86f228c thread_stack 0x49000 2017-09-13 1:56:01 2768968512 [Note] Error reading relay log event: slave SQL thread was killed 2017-09-13 1:56:01 2768968512 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37580 2017-09-13 1:56:01 2808769344 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37580 /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(my_print_stacktrace+0x32)[0x8cf5896] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(handle_fatal_signal+0x3c7)[0x85a2edf] [0xb7724400] /lib/i386-linux-gnu/libc.so.6(+0xbd4a5)[0xb728f4a5] 2017-09-13 1:56:01 2768665408 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16040',replication started in log 'master-bin.000001' at position 37580 2017-09-13 1:56:01 2768968512 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 37580, relay log './slave-relay-bin.000002' position: 1577 /lib/i386-linux-gnu/libc.so.6(fnmatch+0x69)[0xb7290af9] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8d179fa] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_db_keyword_+0x79)[0x8d17ed2] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_db_pargs_+0x65)[0x8d172e8] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(_ZN3THD11enter_stageEPK17PSI_stage_info_v1PKcS4_j+0x28)[0x827049a] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(set_thd_stage_info+0x6b)[0x82f19ef] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld(set_thd_proc_info+0x4d)[0x82f197b] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x88a2789] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a0ce92] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899afbd] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899b1c8] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x899b39e] /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a59819] 2017-09-13 1:56:01 2768968512 [Note] Error reading relay log event: slave SQL thread was killed 2017-09-13 1:56:01 2768968512 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 37871 2017-09-13 1:56:01 2768665408 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 37871 /mnt/buildbot/build/mariadb-10.2.9/sql/mysqld[0x8a5999a] /lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb74acd4c] /lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb72bcace]

People

Assignee:: Andrei Elkin

Reporter:: Elena Stepanova

Votes:: 1 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 2016-10-23 21:46

Updated:: 2023-11-29 17:34

Resolved:: 2023-11-29 17:34

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server