[MDEV-10653] SHOW SLAVE STATUS Can Deadlock an Errored Slave Created: 2016-08-24  Updated: 2023-12-19  Resolved: 2023-12-11

Status: Closed
Project: MariaDB Server
Component/s: Replication, Tests
Affects Version/s: 10.0, 10.1, 10.2, 10.3, 10.4
Fix Version/s: 10.4.33, 10.5.24, 10.6.17, 10.11.7, 11.0.5, 11.1.4, 11.2.3

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Brandon Nesterenko
Resolution: Fixed Votes: 1
Labels: hang

Issue Links:
Duplicate
is duplicated by MDEV-14277 binlog_encryption.rpl_parallel faile... Closed
is duplicated by MDEV-25450 rpl.rpl_parallel_gco_wait_kill failed... Closed
is duplicated by MDEV-31894 Optimize Check for Parallel Replicati... Closed
Relates
relates to MDEV-24086 binlog_encryption.rpl_parallel_stop_o... Open
relates to MDEV-7069 Fix buildbot failures in main server ... Stalled
Sprint: 5.5.55

 Description   

Note: this JIRA was formerly titled rpl.rpl_parallel, binlog_encryption.rpl_parallel fails in buildbot with timeout in include.

A replication parallel worker thread can deadlock with another
connection running SHOW SLAVE STATUS. That is, if the replication
worker thread is in do_gco_wait() and is killed, it will already
hold the LOCK_parallel_entry, and during error reporting, try to
grab the err_lock. SHOW SLAVE STATUS, however, grabs these locks in
reverse order. It will initially grab the err_lock, and then try to
grab LOCK_parallel_entry. This leads to a deadlock when both threads
have grabbed their first lock without the second.

This lead to test errors in rpl.rpl_parallel and binlog_encryption.rpl_parallel.

http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/9240/steps/test/logs/stdio

rpl.rpl_parallel 'mix,xtradb'            w2 [ fail ]
 
        Test ended at 2016-08-23 22:06:29
 
 
 
CURRENT_TEST: rpl.rpl_parallel
 
mysqltest: In included file "./include/wait_for_slave_param.inc": 
included from ./include/wait_for_slave_sql_to_start.inc at line 32:
 
included from ./include/wait_for_slave_to_start.inc at line 27:
 
included from ./include/start_slave.inc at line 35:
 
included from /mnt/data/buildot/maria-slave/work-opensuse-amd64/build/mysql-test/suite/rpl/t/rpl_parallel.test at line 355:
 
At line 115: Timeout in include/wait_for_slave_param.inc
 
 
 
The result from queries just before the failure was:
 
< snip >
 
'group_commit_waiting_for_prior SIGNAL slave_queued3',
 
''))
 
master-bin.000002	1417	Xid	1	1444	COMMIT /* xid=370 */
 
master-bin.000002	1444	Gtid	1	1482	BEGIN GTID 0-1-15
 
master-bin.000002	1482	Query	1	1571	use `test`; INSERT INTO t2 VALUES (20)
 
master-bin.000002	1571	Query	1	1660	use `test`; INSERT INTO t1 VALUES (20)
 
master-bin.000002	1660	Query	1	1749	use `test`; INSERT INTO t2 VALUES (21)
 
master-bin.000002	1749	Query	1	1842	use `test`; INSERT INTO t3 VALUES (20, 20)
 
master-bin.000002	1842	Xid	1	1869	COMMIT /* xid=433 */
 
master-bin.000002	1869	Gtid	1	1907	BEGIN GTID 0-1-16
 
master-bin.000002	1907	Query	1	1999	use `test`; INSERT INTO t3 VALUES(21, 21)
 
master-bin.000002	1999	Xid	1	2026	COMMIT /* xid=438 */
 
master-bin.000002	2026	Gtid	1	2064	BEGIN GTID 0-1-17
 
master-bin.000002	2064	Query	1	2156	use `test`; INSERT INTO t3 VALUES(22, 22)
 
master-bin.000002	2156	Xid	1	2183	COMMIT /* xid=439 */
 
 
 
**** SHOW RELAYLOG EVENTS on server_1 ****
 
relaylog_name = 'No such row'
 
SHOW RELAYLOG EVENTS IN 'No such row';
 
Log_name	Pos	Event_type	Server_id	End_log_pos	Info
 
 
 
More results from queries before failure can be found in /mnt/data/buildot/maria-slave/work-opensuse-amd64/build/mysql-test/var/2/log/rpl_parallel.log

Happens not very often, but quite regularly.



 Comments   
Comment by Elena Stepanova [ 2016-10-23 ]

Another variation:
http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/6138/steps/test_2/logs/stdio

rpl.rpl_parallel 'mix,xtradb'            w4 [ fail ]  timeout after 900 seconds
 
        Test ended at 2016-10-23 20:25:37
 
 
 
Test case timeout after 900 seconds
 
 
 
== /mnt/buildbot/build/mariadb-10.0.28/mysql-test/var/4/log/rpl_parallel.log == 
64	64
 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]
 
 
 
 == /mnt/buildbot/build/mariadb-10.0.28/mysql-test/var/4/tmp/analyze-timeout-mysqld.2.err ==
 
mysqltest: Could not open connection 'default': 2013 Lost connection to MySQL server at 'reading initial communication packet', system error: 104 "Connection reset by peer"
 
 
 
 == /mnt/buildbot/build/mariadb-10.0.28/mysql-test/var/4/tmp/analyze-timeout-mysqld.1.err ==
 
mysqltest: Could not open connection 'default': 2013 Lost connection to MySQL server at 'reading initial communication packet', system error: 104 "Connection reset by peer"

Comment by Elena Stepanova [ 2016-12-09 ]

binlog_encryption.rpl_parallel which inherited the logic from rpl.rpl_parallel is affected too.

Comment by Elena Stepanova [ 2017-01-05 ]

A fix for the valgrind failure pushed into 10.0 https://github.com/MariaDB/server/commit/670b85804ca6d3b0e640a7ddbb5b6e494bec7c2b .
It won't, however, help with the other one.

Comment by Elena Stepanova [ 2017-02-20 ]

Recent occurrences of the timeout on rpl.rpl_parallel:

http://buildbot.askmonty.org/buildbot/builders/p8-trusty-bintar-debug/builds/560
2017-02-17, 10.0, p8-trusty-bintar-debug, failed twice with timeouts of different sorts, the suite aborted because to many tests were failing
This builder is most prone to timeouts. Sometimes it behaves, then the test takes there 150-200 seconds, but sometimes it's really bad, then various tests are failing. I don't think it's specific to this particular test.

2017-01-08, 10.0, fulltest2 normal run, unfortunately logs aren't available anymore. 

rpl.rpl_parallel 'stmt,xtradb'           w1 [ fail ]  timeout after 900 seconds
 
        Test ended at 2017-01-08 19:34:12
 
 
 
Test case timeout after 900 seconds
 
 
 
== /mnt/buildbot/build/mariadb-10.0.29/mysql-test/var/1/log/rpl_parallel.log == 
64	64
 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]
 
 
 
 == /mnt/buildbot/build/mariadb-10.0.29/mysql-test/var/1/tmp/analyze-timeout-mysqld.1.err ==
 
mysqltest: Could not open connection 'default': 2013 Lost connection to MySQL server at 'reading initial communication packet', system error: 104 "Connection reset by peer"

There have been other recorded failures in that test run.
Normally the test takes only ~15 seconds there, so it's very suspicious.

2017-01-01, bb-10.0-gal-sachin, bld-dan-debug, logs not available.

rpl.rpl_parallel 'innodb_plugin,stmt'    w1 [ fail ]  timeout after 3600 seconds
 
        Test ended at 2017-01-01 07:14:51
 
 
 
Test case timeout after 3600 seconds
 
 
 
== /opt/buildbot-slave/mariadb/dan_demeter1/build/mysql-test/var/1/log/rpl_parallel.log == 
64	64
 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]

There have been many errors in that builds, but they are usual ipv6, not timeouts.
Normally the test takes 45-60 seconds there, so it's also suspicious.

2016-09-09, 10.2, kvm-qa-linux, logs not available

rpl.rpl_parallel 'stmt,xtradb'           w3 [ fail ]  timeout after 900 seconds
 
        Test ended at 2016-09-08 18:06:47
 
 
 
Test case timeout after 900 seconds
 
 
 
== /home/buildbot/build/mysql-test/var/3/log/rpl_parallel.log == 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
connection server_2;
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]

It was the only failure in the test run.
It's jessie-based builder, and it's not slow, so it's also suspicious.

The rest is similar, the timeout most often occurs on fulltest2 and on p8.
Recently test runs on p8 bintars were switched to using --mem, it seems to be working well so far, no more timeouts (although maybe it's just the lucky time).
fulltest2 failures need to be investigated.

Comment by Elena Stepanova [ 2017-02-21 ]

So, the valgrind issue was fixed by increasing the timeout, the p8 issue was hopefully fixed by switching to --mem, but for fulltest2 and dan builders we have nothing to work with, as no logs are available. It's quite possible that there is a hidden problem in there. We'll have to wait for another occasion.

Comment by Alice Sherepa [ 2017-09-08 ]

rpl.rpl_parallel failed on 10.2 (kvm-fulltest2) http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/9511/steps/test_2/logs/stdio

rpl.rpl_parallel 'innodb,row'            w1 [ fail ]
 
        Test ended at 2017-09-08 04:52:35
 
 
 
CURRENT_TEST: rpl.rpl_parallel
 
mysqltest: In included file "./include/wait_for_slave_param.inc": 
included from ./include/wait_for_slave_sql_error.inc at line 41:
 
included from ./extra/rpl_tests/rpl_parallel.inc at line 1476:
 
included from /mnt/buildbot/build/mariadb-10.2.9/mysql-test/suite/rpl/t/rpl_parallel.test at line 2:
 
At line 119: Timeout in include/wait_for_slave_param.inc
 
 
 
The result from queries just before the failure was:
 
< snip >
 
master-bin.000004	4	Format_desc	1	256	Server ver: 10.2.9-MariaDB-debug-log, Binlog ver: 4
 
master-bin.000004	256	Gtid_list	1	347	[1-1-3,0-1-71,3-1-1,2-1-2]
 
master-bin.000004	347	Binlog_checkpoint	1	391	master-bin.000003
 
master-bin.000004	391	Binlog_checkpoint	1	435	master-bin.000004
 
master-bin.000004	435	Gtid	1	477	BEGIN GTID 0-1-72
 
master-bin.000004	477	Annotate_rows	1	526	INSERT INTO t2 VALUES (33)
 
master-bin.000004	526	Table_map	1	571	table_id: 35 (test.t2)
 
master-bin.000004	571	Write_rows_v1	1	609	table_id: 35 flags: STMT_END_F
 
master-bin.000004	609	Xid	1	640	COMMIT /* xid=991 */
 
master-bin.000004	640	Gtid	1	682	BEGIN GTID 0-1-73
 
master-bin.000004	682	Annotate_rows	1	731	INSERT INTO t2 VALUES (34)
 
master-bin.000004	731	Table_map	1	776	table_id: 35 (test.t2)
 
master-bin.000004	776	Write_rows_v1	1	814	table_id: 35 flags: STMT_END_F
 
master-bin.000004	814	Xid	1	845	COMMIT /* xid=992 */
 
 
 
**** SHOW RELAYLOG EVENTS on server_1 ****
 
relaylog_name = 'No such row'
 
SHOW RELAYLOG EVENTS IN 'No such row';
 
Log_name	Pos	Event_type	Server_id	End_log_pos	Info
 
connection server_2;
 
 
 
More results from queries before failure can be found in /mnt/buildbot/build/mariadb-10.2.9/mysql-test/var/1/log/rpl_parallel.log
 
 
 
 - saving '/mnt/buildbot/build/mariadb-10.2.9/mysql-test/var/1/log/rpl.rpl_parallel-innodb,row/' to '/mnt/buildbot/build/mariadb-10.2.9/mysql-test/var/log/rpl.rpl_parallel-innodb,row/'


rpl.rpl_parallel 'innodb,mix'            w4 [ fail ]  timeout after 900 seconds
 
        Test ended at 2017-09-08 04:55:55
 
 
 
Test case timeout after 900 seconds
 
 
 
== /mnt/buildbot/build/mariadb-10.2.9/mysql-test/var/4/log/rpl_parallel.log == 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
connection server_2;
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]

Comment by Alice Sherepa [ 2017-11-06 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/10260/steps/test_2/logs/stdio

binlog_encryption.rpl_parallel 'innodb,stmt' w4 [ fail ]
 
        Test ended at 2017-11-03 18:33:15
 
 
 
CURRENT_TEST: binlog_encryption.rpl_parallel
 
mysqltest: In included file "./include/wait_for_slave_param.inc": 
included from ./include/wait_for_slave_sql_error.inc at line 41:
 
included from ./extra/rpl_tests/rpl_parallel.inc at line 1476:
 
included from /mnt/buildbot/build/mariadb-10.2.11/mysql-test/suite/binlog_encryption/rpl_parallel.test at line 2:
 
At line 119: Timeout in include/wait_for_slave_param.inc
 
 
 
The result from queries just before the failure was:
 
< snip >
 
binlog_name = 'master-bin.000004'
 
SHOW BINLOG EVENTS IN 'master-bin.000004';
 
Log_name	Pos	Event_type	Server_id	End_log_pos	Info
 
master-bin.000004	4	Format_desc	1	256	Server ver: 10.2.11-MariaDB-debug-log, Binlog ver: 4
 
master-bin.000004	256	Start_encryption	1	296	
master-bin.000004	296	Gtid_list	1	387	[1-1-3,0-1-71,3-1-1,2-1-2]
 
master-bin.000004	387	Binlog_checkpoint	1	431	master-bin.000003
 
master-bin.000004	431	Binlog_checkpoint	1	475	master-bin.000004
 
master-bin.000004	475	Gtid	1	517	BEGIN GTID 0-1-72
 
master-bin.000004	517	Query	1	610	use `test`; INSERT INTO t2 VALUES (33)
 
master-bin.000004	610	Xid	1	641	COMMIT /* xid=9090 */
 
master-bin.000004	641	Gtid	1	683	BEGIN GTID 0-1-73
 
master-bin.000004	683	Query	1	776	use `test`; INSERT INTO t2 VALUES (34)
 
master-bin.000004	776	Xid	1	807	COMMIT /* xid=9091 */
 
 
 
**** SHOW RELAYLOG EVENTS on server_1 ****
 
relaylog_name = 'No such row'
 
SHOW RELAYLOG EVENTS IN 'No such row';
 
Log_name	Pos	Event_type	Server_id	End_log_pos	Info
 
connection server_2;
 
 
 
More results from queries before failure can be found in /mnt/buildbot/build/mariadb-10.2.11/mysql-test/var/4/log/rpl_parallel.log
 
 
 
 - saving '/mnt/buildbot/build/mariadb-10.2.11/mysql-test/var/4/log/binlog_encryption.rpl_parallel-innodb,stmt/' to '/mnt/buildbot/build/mariadb-10.2.11/mysql-test/var/log/binlog_encryption.rpl_parallel-innodb,stmt/'
 
 
 
Retrying test binlog_encryption.rpl_parallel, attempt(2/3)...
 
 
 
worker[4] > Restart  - not started
 
worker[4] > Restart  - not started
 
***Warnings generated in error logs during shutdown after running tests: binlog_encryption.rpl_relayrotate binlog_encryption.rpl_parallel
 
 
 
2017-11-03 18:28:01 2808769344 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it accesses a non-transactional table after accessing a transactional table within the same transaction Statement: INSERT INTO t1 VALUES (20)
 
2017-11-03 18:28:02 2821716800 [Warning] Slave: Connection was killed Error_code: 1927
 
2017-11-03 18:28:02 2768603968 [Warning] Slave: Commit failed due to failure of an earlier commit on which this one depends Error_code: 1964
 
2017-11-03 18:28:02 2767190848 [Warning] Slave: Connection was killed Error_code: 1927
 
2017-11-03 18:28:02 2825857856 [Warning] Slave: Commit failed due to failure of an earlier commit on which this one depends Error_code: 1964
 
2017-11-03 18:28:03 2821716800 [Warning] Slave: Connection was killed Error_code: 1927
 
2017-11-03 18:28:03 2821110592 [Warning] Slave: Commit failed due to failure of an earlier commit on which this one depends Error_code: 1964
 
2017-11-03 18:28:04 2896620352 [Warning] Slave: Connection was killed Error_code: 1927
 
2017-11-03 18:28:04 2825857856 [Warning] Slave: Connection was killed Error_code: 1927
 
2017-11-03 18:28:05 2821413696 [Warning] Slave: Duplicate entry '111' for key 'PRIMARY' Error_code: 1062
 
2017-11-03 18:28:07 2823813952 [Warning] Slave: Duplicate entry '32' for key 'PRIMARY' Error_code: 1062
 
2017-11-03 18:28:07 2769210176 [Warning] Slave: Duplicate entry '32' for key 'PRIMARY' Error_code: 1062

Comment by Elena Stepanova [ 2019-07-07 ]

Still happens:
http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos6-amd64-debug/builds/1945

Comment by Alice Sherepa [ 2020-09-08 ]

recent occurrence on 10.1 http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-centos6-amd64-debug/builds/3048/steps/mtr/logs/stdio
on 10.4 http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos74-amd64-debug/builds/4757/steps/mtr_centos_mix/logs/stdio
on 10.2 http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos74-amd64-debug/builds/4846/steps/mtr/logs/stdio

Comment by Andrei Elkin [ 2020-09-15 ]

Verified: still fails. The most recent 

rpl.rpl_parallel 'innodb,stmt'           w1 [ fail ]
 
        Test ended at 2020-09-09 11:28:14
 
 
 
CURRENT_TEST: rpl.rpl_parallel
 
mysqltest: In included file "./include/wait_for_slave_param.inc": 
included from ./include/wait_for_slave_sql_to_start.inc at line 32:

Comment by Alice Sherepa [ 2020-12-18 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/26116/steps/mtr_nm/logs/stdio

rpl.rpl_parallel 'innodb,mix'            w4 [ fail ]  timeout after 900 seconds
 
        Test ended at 2020-12-17 07:40:10
 
 
 
Test case timeout after 900 seconds
 
 
 
== /mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/4/log/rpl_parallel.log == 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
connection server_2;
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]
 
 
 
 == /mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/4/tmp/analyze-timeout-mysqld.2.err ==
 
mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/tmp/4/mysqld.' (111)
 
 
 
 == /mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/4/tmp/analyze-timeout-mysqld.1.err ==
 
mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/tmp/4/mysqld.' (111)
 
 
 
 
 
 - found 'core' (0/0)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/4/log/rpl.rpl_parallel-innodb,mix/mysqld.2/data/core
 
Core generated by '/mnt/buildbot/build/mariadb-10.4.18/sql/mysqld'
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
[New LWP 15867]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
 
Core was generated by `/mnt/buildbot/build/mariadb-10.4.18/sql/mysqld --defaults-group-suffix=.2 --def'.
 
Program terminated with signal SIGABRT, Aborted.
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
[Current thread is 1 (Thread 0xb6e8a7c0 (LWP 15847))]
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76cace1 in pthread_kill () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116341a in my_write_core (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/stacktrace.c:386
 
#3  0x808e0d0a in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/signal_handler.cc:343
 
#4  <signal handler called>
 
#5  0xb776ac31 in __kernel_vsyscall ()
 
#6  0xb70f9f8f in poll () from /lib/i386-linux-gnu/libc.so.6
 
#7  0x8045b79a in handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:6481
 
#8  0x8045aa7e in mysqld_main (argc=158, argv=0x8337c330) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:5917
 
#9  0x8044e48b in main (argc=25, argv=0xbfcc57e4) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/main.cc:25
 
 
 
Thread 39 (Thread 0xb17ccb40 (LWP 15867)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x824bdba0 <COND_checkpoint>, mp=0x824bdb20 <LOCK_checkpoint>, abstime=0xb17cc19c, file=0x8149f498 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x80bab811 in inline_mysql_cond_timedwait (that=0x824bdba0 <COND_checkpoint>, mutex=0x824bdb20 <LOCK_checkpoint>, abstime=0xb17cc19c, src_file=0x8149f4e8 "/home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80babd76 in my_service_thread_sleep (control=0x81ade4d8 <checkpoint_control>, sleep_time=30000000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_servicethread.c:115
 
#5  0x80b9f504 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_checkpoint.c:707
 
#6  0x80b426ad in pfs_spawn_thread (arg=0x8352a028) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 38 (Thread 0xae5fcb40 (LWP 15880)):
 
#0  0xb7736570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77365e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80d3eb67 in LinuxAIOHandler::collect (this=0xae5fc0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80d3f273 in LinuxAIOHandler::poll (this=0xae5fc0f0, m1=0xae5fc154, m2=0xae5fc158, request=0xae5fc178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80d3f3d7 in os_aio_linux_handler (global_segment=3, m1=0xae5fc154, m2=0xae5fc158, request=0xae5fc178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80d449c9 in os_aio_handler (segment=3, m1=0xae5fc154, m2=0xae5fc158, request=0xae5fc178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x81000431 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e6a88e in io_handler_thread (arg=0x824f1bac <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 37 (Thread 0xaddfbb40 (LWP 15882)):
 
#0  0xb7736570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77365e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80d3eb67 in LinuxAIOHandler::collect (this=0xaddfb0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80d3f273 in LinuxAIOHandler::poll (this=0xaddfb0f0, m1=0xaddfb154, m2=0xaddfb158, request=0xaddfb178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80d3f3d7 in os_aio_linux_handler (global_segment=4, m1=0xaddfb154, m2=0xaddfb158, request=0xaddfb178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80d449c9 in os_aio_handler (segment=4, m1=0xaddfb154, m2=0xaddfb158, request=0xaddfb178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x81000431 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e6a88e in io_handler_thread (arg=0x824f1bb0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 36 (Thread 0xacdf9b40 (LWP 15884)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4ea8c in os_event::timed_wait (this=0x8355c258, abstime=0xacdf9078) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80d4ed9d in os_event::wait_time_low (this=0x8355c258, time_in_usec=1000000, reset_sig_count=904) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80d4ef73 in os_event_wait_time_low (event=0x8355c258, time_in_usec=1000000, reset_sig_count=904) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80f7b803 in pc_sleep_if_needed (next_loop_time=2526506, sig_count=904, cur_time=2525506) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0flu.cc:2628
 
#6  0x80f7ca1b in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0flu.cc:3087
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 35 (Thread 0xab9f9b40 (LWP 15887)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4ea8c in os_event::timed_wait (this=0x8368dc68, abstime=0xab9f9218) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80d4ed9d in os_event::wait_time_low (this=0x8368dc68, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80d4ef73 in os_event_wait_time_low (event=0x8368dc68, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80d0bfc2 in lock_wait_timeout_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/lock/lock0wait.cc:522
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 34 (Thread 0xab1f8b40 (LWP 15888)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4ea8c in os_event::timed_wait (this=0x8355c138, abstime=0xab1f8118) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80d4ed9d in os_event::wait_time_low (this=0x8355c138, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80d4ef73 in os_event_wait_time_low (event=0x8355c138, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80e65900 in srv_error_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:1854
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 33 (Thread 0xac1fab40 (LWP 15893)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76cd8aa in nanosleep () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f5f6 in os_thread_sleep (tm=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0thread.cc:231
 
#3  0x80e6718a in srv_master_sleep () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2341
 
#4  0x80e67307 in srv_master_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2381
 
#5  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#6  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 32 (Thread 0xa91f4b40 (LWP 15894)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f1fe in os_event::wait (this=0x8355bfb8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80d4ec5f in os_event::wait_low (this=0x8355bfb8, reset_sig_count=46) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80d4efa5 in os_event_wait_low (event=0x8355bfb8, reset_sig_count=46) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e63306 in srv_resume_thread (slot=0x81ae7124 <srv_sys+228>, sig_count=46, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e68003 in srv_purge_coordinator_suspend (slot=0x81ae7124 <srv_sys+228>, rseg_history_len=67) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2657
 
#7  0x80e682f5 in srv_purge_coordinator_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2732
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 31 (Thread 0xa85ffb40 (LWP 15895)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f1fe in os_event::wait (this=0x8355c018) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80d4ec5f in os_event::wait_low (this=0x8355c018, reset_sig_count=188) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80d4efa5 in os_event_wait_low (event=0x8355c018, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e63306 in srv_resume_thread (slot=0x81ae7140 <srv_sys+256>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e67b2e in srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2517
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 30 (Thread 0xa7dfeb40 (LWP 15896)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f1fe in os_event::wait (this=0x8355c0d8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80d4ec5f in os_event::wait_low (this=0x8355c0d8, reset_sig_count=174) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80d4efa5 in os_event_wait_low (event=0x8355c0d8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e63306 in srv_resume_thread (slot=0x81ae7178 <srv_sys+312>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e67b2e in srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2517
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 29 (Thread 0xb0bc0b40 (LWP 15902)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811680af in safe_cond_wait (cond=0x81cb425c <mysql_bin_log+3324>, mp=0x81cb41f8 <mysql_bin_log+3224>, file=0x81439ccc "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x80a03955 in inline_mysql_cond_wait (that=0x81cb425c <mysql_bin_log+3324>, mutex=0x81cb41f8 <mysql_bin_log+3224>, src_file=0x8143a9dc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc", src_line=10061) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x80a20f1d in binlog_background_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc:10061
 
#5  0x80b426ad in pfs_spawn_thread (arg=0x834d69b8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 28 (Thread 0xac5f8b40 (LWP 15912)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811680af in safe_cond_wait (cond=0x81c8bae0 <COND_slave_background>, mp=0x81c8e140 <LOCK_slave_background>, file=0x81231248 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x8048129e in inline_mysql_cond_wait (that=0x81c8bae0 <COND_slave_background>, mutex=0x81c8e140 <LOCK_slave_background>, src_file=0x81231ecc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc", src_line=522) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x80482c86 in handle_slave_background (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:522
 
#5  0x80b426ad in pfs_spawn_thread (arg=0x8373ee68) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 27 (Thread 0xac5aeb40 (LWP 15918)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76ccd32 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0xb76c69e5 in pthread_mutex_lock () from /lib/i386-linux-gnu/libpthread.so.0
 
#3  0x81167902 in safe_mutex_lock (mp=0xaff3e5b8, my_flags=0, file=0x812c7df4 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc", line=2412) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:293
 
#4  0x807835af in inline_mysql_mutex_lock (that=0xaff3e5b8, src_file=0x812c7df4 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc", src_line=2412) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:702
 
#5  0x8078a48a in rpl_parallel::workers_idle (this=0x83809758) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:2412
 
#6  0x8048bfac in send_show_master_info_data (thd=0xa8609d48, mi=0x838051c0, full=false, gtid_pos=0xac5ad3c8) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:3343
 
#7  0x80489e94 in show_master_info (thd=0xa8609d48, mi=0x838051c0, full=false) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:2990
 
#8  0x805914e8 in mysql_execute_command (thd=0xa8609d48) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_parse.cc:4144
 
#9  0x8059ecc5 in mysql_parse (thd=0xa8609d48, rawbuf=0xa8623248 "SHOW SLAVE STATUS", length=17, parser_state=0xac5adb94, is_com_multi=false, is_next_command=false) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_parse.cc:7936
 
#10 0x8058a17a in dispatch_command (command=COM_QUERY, thd=0xa8609d48, packet=0xa861b199 "", packet_length=17, is_com_multi=false, is_next_command=false) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_parse.cc:1840
 
#11 0x805887ad in do_command (thd=0xa8609d48) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_parse.cc:1358
 
#12 0x8070432c in do_handle_one_connection (connect=0x8380b960) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1412
 
#13 0x8070404f in handle_one_connection (arg=0x8380b960) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#14 0x80b426ad in pfs_spawn_thread (arg=0x8380c8f8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#15 0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#16 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 26 (Thread 0xac564b40 (LWP 15933)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x81c8baa0 <COND_thread_cache>, mp=0x81c8db40 <LOCK_thread_cache>, abstime=0xac56416c, file=0x81221024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8044ed89 in inline_mysql_cond_timedwait (that=0x81c8baa0 <COND_thread_cache>, mutex=0x81c8db40 <LOCK_thread_cache>, abstime=0xac56416c, src_file=0x8122311c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80453ad6 in cache_thread (thd=0xa88059b8) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80453d64 in one_thread_per_connection_end (thd=0xa88059b8, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x8070439f in do_handle_one_connection (connect=0x8380bb28) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x8070404f in handle_one_connection (arg=0x8380bb28) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80b426ad in pfs_spawn_thread (arg=0x8380cbd0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 25 (Thread 0xa895fb40 (LWP 15967)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x81c8baa0 <COND_thread_cache>, mp=0x81c8db40 <LOCK_thread_cache>, abstime=0xa895f16c, file=0x81221024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8044ed89 in inline_mysql_cond_timedwait (that=0x81c8baa0 <COND_thread_cache>, mutex=0x81c8db40 <LOCK_thread_cache>, abstime=0xa895f16c, src_file=0x8122311c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80453ad6 in cache_thread (thd=0xac406c10) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80453d64 in one_thread_per_connection_end (thd=0xac406c10, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x8070439f in do_handle_one_connection (connect=0x8380bb28) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x8070404f in handle_one_connection (arg=0x8380bb28) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80b426ad in pfs_spawn_thread (arg=0x834aecf0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 24 (Thread 0xa4df8b40 (LWP 15970)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x81c8baa0 <COND_thread_cache>, mp=0x81c8db40 <LOCK_thread_cache>, abstime=0xa4df816c, file=0x81221024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8044ed89 in inline_mysql_cond_timedwait (that=0x81c8baa0 <COND_thread_cache>, mutex=0x81c8db40 <LOCK_thread_cache>, abstime=0xa4df816c, src_file=0x8122311c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80453ad6 in cache_thread (thd=0xb0a0cde0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80453d64 in one_thread_per_connection_end (thd=0xb0a0cde0, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x8070439f in do_handle_one_connection (connect=0x834d6b68) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x8070404f in handle_one_connection (arg=0x834d6b68) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80b426ad in pfs_spawn_thread (arg=0x834aeeb8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 23 (Thread 0xa4b5eb40 (LWP 16386)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811680af in safe_cond_wait (cond=0x8380765c, mp=0x838070b4, file=0x81439ccc "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x80a03955 in inline_mysql_cond_wait (that=0x8380765c, mutex=0x838070b4, src_file=0x8143a9dc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc", src_line=8484) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x80a1ca62 in MYSQL_BIN_LOG::wait_for_update_relay_log (this=0x838070ac, thd=0xaff152b8) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc:8484
 
#5  0x80498876 in next_event (rgi=0xaff0bf00, event_size=0xa4b5db60) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:7761
 
#6  0x8048e447 in exec_relay_log_event (thd=0xaff152b8, rli=0x83806c30, serial_rgi=0xaff0bf00) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:4267
 
#7  0x80492697 in handle_slave_sql (arg=0x838051c0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:5612
 
#8  0x80b426ad in pfs_spawn_thread (arg=0xa87d3290) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 22 (Thread 0xa4d1ab40 (LWP 16387)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811680af in safe_cond_wait (cond=0xaff07b40, mp=0xaff3e5b8, file=0x812c7980 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x80783899 in inline_mysql_cond_wait (that=0xaff07b40, mutex=0xaff3e5b8, src_file=0x812c7df4 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc", src_line=381) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x80784fa7 in do_gco_wait (rgi=0xaff1aad8, gco=0xaff07b40, did_enter_cond=0xa4d1a1df, old_stage=0xa4d1a234) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:381
 
#5  0x80787292 in handle_rpl_parallel_thread (arg=0xaff35a38) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:1236
 
#6  0x80b426ad in pfs_spawn_thread (arg=0xaff43a08) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 21 (Thread 0xad5fab40 (LWP 15883)):
 
#0  0xb7736570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77365e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80d3eb67 in LinuxAIOHandler::collect (this=0xad5fa0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80d3f273 in LinuxAIOHandler::poll (this=0xad5fa0f0, m1=0xad5fa154, m2=0xad5fa158, request=0xad5fa178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80d3f3d7 in os_aio_linux_handler (global_segment=5, m1=0xad5fa154, m2=0xad5fa158, request=0xad5fa178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80d449c9 in os_aio_handler (segment=5, m1=0xad5fa154, m2=0xad5fa158, request=0xad5fa178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x81000431 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e6a88e in io_handler_thread (arg=0x824f1bb4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 20 (Thread 0xafdffb40 (LWP 15877)):
 
#0  0xb7736570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77365e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80d3eb67 in LinuxAIOHandler::collect (this=0xafdff0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80d3f273 in LinuxAIOHandler::poll (this=0xafdff0f0, m1=0xafdff154, m2=0xafdff158, request=0xafdff178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80d3f3d7 in os_aio_linux_handler (global_segment=0, m1=0xafdff154, m2=0xafdff158, request=0xafdff178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80d449c9 in os_aio_handler (segment=0, m1=0xafdff154, m2=0xafdff158, request=0xafdff178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x81000431 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e6a88e in io_handler_thread (arg=0x824f1ba0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 19 (Thread 0xb0b76b40 (LWP 15905)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb7049e8c in sigwaitinfo () from /lib/i386-linux-gnu/libc.so.6
 
#2  0x8044e512 in my_sigwait (set=0xb0b761d0, sig=0xb0b761a0, code=0xb0b761a4) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/my_pthread.h:196
 
#3  0x804548bd in signal_hand (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:3251
 
#4  0x80b426ad in pfs_spawn_thread (arg=0x83737b78) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#5  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#6  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 18 (Thread 0xa75fdb40 (LWP 15897)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f1fe in os_event::wait (this=0x8355c078) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80d4ec5f in os_event::wait_low (this=0x8355c078, reset_sig_count=121) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80d4efa5 in os_event_wait_low (event=0x8355c078, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e63306 in srv_resume_thread (slot=0x81ae715c <srv_sys+284>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e67b2e in srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2517
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 17 (Thread 0xaedfdb40 (LWP 15879)):
 
#0  0xb7736570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77365e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80d3eb67 in LinuxAIOHandler::collect (this=0xaedfd0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80d3f273 in LinuxAIOHandler::poll (this=0xaedfd0f0, m1=0xaedfd154, m2=0xaedfd158, request=0xaedfd178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80d3f3d7 in os_aio_linux_handler (global_segment=2, m1=0xaedfd154, m2=0xaedfd158, request=0xaedfd178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80d449c9 in os_aio_handler (segment=2, m1=0xaedfd154, m2=0xaedfd158, request=0xaedfd178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x81000431 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e6a88e in io_handler_thread (arg=0x824f1ba8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 16 (Thread 0xaa1f6b40 (LWP 15890)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4ea8c in os_event::timed_wait (this=0x836b6af8, abstime=0xaa1f6238) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80d4ed9d in os_event::wait_time_low (this=0x836b6af8, time_in_usec=10000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80d4ef73 in os_event_wait_time_low (event=0x836b6af8, time_in_usec=10000000, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80fe78eb in dict_stats_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/dict/dict0stats_bg.cc:476
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 15 (Thread 0xaa9f7b40 (LWP 15889)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4ea8c in os_event::timed_wait (this=0x8355c198, abstime=0xaa9f7208) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80d4ed9d in os_event::wait_time_low (this=0x8355c198, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80d4ef73 in os_event_wait_time_low (event=0x8355c198, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80e65340 in srv_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:1690
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 14 (Thread 0xa6dfcb40 (LWP 15898)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f1fe in os_event::wait (this=0x8355c1f8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80d4ec5f in os_event::wait_low (this=0x8355c1f8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80d4efa5 in os_event_wait_low (event=0x8355c1f8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80f6e156 in buf_dump_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0dump.cc:832
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 13 (Thread 0xa5dfab40 (LWP 15900)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f1fe in os_event::wait (this=0x8355c2b8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80d4ec5f in os_event::wait_low (this=0x8355c2b8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80d4efa5 in os_event_wait_low (event=0x8355c2b8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80f5393d in buf_resize_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0buf.cc:3142
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 12 (Thread 0xa55f9b40 (LWP 15901)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811680af in safe_cond_wait (cond=0xa55f91c4, mp=0xa55f91f8, file=0x814ad660 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1185) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x80c3a4ad in inline_mysql_cond_wait (that=0xa55f91c4, mutex=0xa55f91f8, src_file=0x814af4ec "/home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/handler/ha_innodb.cc", src_line=311) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1185
 
#4  0x80c3fdb8 in thd_destructor_proxy () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/handler/ha_innodb.cc:311
 
#5  0x80b426ad in pfs_spawn_thread (arg=0x836e1930) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 11 (Thread 0xa4acab40 (LWP 16032)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x81c8baa0 <COND_thread_cache>, mp=0x81c8db40 <LOCK_thread_cache>, abstime=0xa4aca16c, file=0x81221024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8044ed89 in inline_mysql_cond_timedwait (that=0x81c8baa0 <COND_thread_cache>, mutex=0x81c8db40 <LOCK_thread_cache>, abstime=0xa4aca16c, src_file=0x8122311c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80453ad6 in cache_thread (thd=0xb2216dd0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80453d64 in one_thread_per_connection_end (thd=0xb2216dd0, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x8070439f in do_handle_one_connection (connect=0x83835720) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x8070404f in handle_one_connection (arg=0x83835720) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80b426ad in pfs_spawn_thread (arg=0x83835960) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 10 (Thread 0xa4a80b40 (LWP 16076)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x81c8baa0 <COND_thread_cache>, mp=0x81c8db40 <LOCK_thread_cache>, abstime=0xa4a8016c, file=0x81221024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8044ed89 in inline_mysql_cond_timedwait (that=0x81c8baa0 <COND_thread_cache>, mutex=0x81c8db40 <LOCK_thread_cache>, abstime=0xa4a8016c, src_file=0x8122311c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80453ad6 in cache_thread (thd=0xac452f70) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80453d64 in one_thread_per_connection_end (thd=0xac452f70, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x8070439f in do_handle_one_connection (connect=0x83836498) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x8070404f in handle_one_connection (arg=0x83836498) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80b426ad in pfs_spawn_thread (arg=0x83834600) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 9 (Thread 0xa4d64b40 (LWP 16389)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76ccd32 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0xb76c69e5 in pthread_mutex_lock () from /lib/i386-linux-gnu/libpthread.so.0
 
#3  0x81167902 in safe_mutex_lock (mp=0x83806c34, my_flags=0, file=0x8144a5c8 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_reporting.cc", line=42) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:293
 
#4  0x80a4dc43 in inline_mysql_mutex_lock (that=0x83806c34, src_file=0x8144a5c8 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_reporting.cc", src_line=42) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:702
 
#5  0x80a4dde5 in Slave_reporting_capability::report (this=0x83806c30, level=ERROR_LEVEL, err_code=1927, extra_info=0xaff07aa5 "Gtid 0-1-33", msg=0x81234071 "%s") at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_reporting.cc:42
 
#6  0x80491145 in slave_output_error_info (rgi=0xaff07470, thd=0xb2202670) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:5222
 
#7  0x80784f78 in do_gco_wait (rgi=0xaff07470, gco=0xaff07b40, did_enter_cond=0xa4d641df, old_stage=0xa4d64234) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:370
 
#8  0x80787292 in handle_rpl_parallel_thread (arg=0xaff35d00) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:1236
 
#9  0x80b426ad in pfs_spawn_thread (arg=0xaff0b470) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#10 0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#11 0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 8 (Thread 0xa89a9b40 (LWP 16390)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811680af in safe_cond_wait (cond=0xaff07b40, mp=0xaff3e5b8, file=0x812c7980 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x80783899 in inline_mysql_cond_wait (that=0xaff07b40, mutex=0xaff3e5b8, src_file=0x812c7df4 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc", src_line=381) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x80784fa7 in do_gco_wait (rgi=0xaff3d2a8, gco=0xaff07b40, did_enter_cond=0xa89a91df, old_stage=0xa89a9234) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:381
 
#5  0x80787292 in handle_rpl_parallel_thread (arg=0xaff35e64) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:1236
 
#6  0x80b426ad in pfs_spawn_thread (arg=0xaff36010) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 7 (Thread 0xa4cd0b40 (LWP 16388)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76ccd32 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0xb76c69e5 in pthread_mutex_lock () from /lib/i386-linux-gnu/libpthread.so.0
 
#3  0x81167902 in safe_mutex_lock (mp=0xaff3e5b8, my_flags=0, file=0x8129c608 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_rli.cc", line=2450) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:293
 
#4  0x806ed8db in inline_mysql_mutex_lock (that=0xaff3e5b8, src_file=0x8129c608 "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_rli.cc", src_line=2450) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:702
 
#5  0x806f513c in rpl_group_info::mark_start_commit (this=0xaff09480) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_rli.cc:2450
 
#6  0x807874dd in handle_rpl_parallel_thread (arg=0xaff35b9c) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/rpl_parallel.cc:1324
 
#7  0x80b426ad in pfs_spawn_thread (arg=0xaff19368) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 6 (Thread 0xa89f3b40 (LWP 16385)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76ccd32 in __lll_lock_wait () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0xb76c69e5 in pthread_mutex_lock () from /lib/i386-linux-gnu/libpthread.so.0
 
#3  0x81167902 in safe_mutex_lock (mp=0x838069c4, my_flags=0, file=0x81231ecc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc", line=6336) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:293
 
#4  0x80481063 in inline_mysql_mutex_lock (that=0x838069c4, src_file=0x81231ecc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc", src_line=6336) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:702
 
#5  0x80494abb in queue_event (mi=0x838051c0, buf=0xa8847711 "", event_len=40) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:6336
 
#6  0x80490754 in handle_slave_io (arg=0x838051c0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:4992
 
#7  0x80b426ad in pfs_spawn_thread (arg=0xa87d3290) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 5 (Thread 0xa65fbb40 (LWP 15899)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76cd8aa in nanosleep () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4f5f6 in os_thread_sleep (tm=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0thread.cc:231
 
#3  0x80f41f1f in btr_defragment_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/btr/btr0defragment.cc:700
 
#4  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#5  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 4 (Thread 0xa99f5b40 (LWP 15891)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80d4ea8c in os_event::timed_wait (this=0x83673cb0, abstime=0xa99f50f8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80d4ed9d in os_event::wait_time_low (this=0x83673cb0, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80d4ef73 in os_event_wait_time_low (event=0x83673cb0, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80ed5357 in ib_wqueue_timedwait (wq=0x83673b90, wait_in_usecs=5000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/ut/ut0wqueue.cc:156
 
#6  0x8104bb62 in fts_optimize_thread (arg=0x83673b90) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fts/fts0opt.cc:2830
 
#7  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 3 (Thread 0xaf5feb40 (LWP 15878)):
 
#0  0xb7736570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77365e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80d3eb67 in LinuxAIOHandler::collect (this=0xaf5fe0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80d3f273 in LinuxAIOHandler::poll (this=0xaf5fe0f0, m1=0xaf5fe154, m2=0xaf5fe158, request=0xaf5fe178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80d3f3d7 in os_aio_linux_handler (global_segment=1, m1=0xaf5fe154, m2=0xaf5fe158, request=0xaf5fe178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80d449c9 in os_aio_handler (segment=1, m1=0xaf5fe154, m2=0xaf5fe158, request=0xaf5fe178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x81000431 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e6a88e in io_handler_thread (arg=0x824f1ba4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 2 (Thread 0xb2befb40 (LWP 15866)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76c9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116835a in safe_cond_timedwait (cond=0x825065a0 <COND_timer>, mp=0x82506520 <LOCK_timer>, abstime=0xb2bef250, file=0x81633854 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8116930c in inline_mysql_cond_timedwait (that=0x825065a0 <COND_timer>, mutex=0x82506520 <LOCK_timer>, abstime=0xb2bef250, src_file=0x816338a4 "/home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x81169eeb in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_timer.c:292
 
#5  0x80b426ad in pfs_spawn_thread (arg=0x834a0b08) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76c42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb710416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 1 (Thread 0xb6e8a7c0 (LWP 15847)):
 
#0  0xb776ac31 in __kernel_vsyscall ()
 
#1  0xb76cace1 in pthread_kill () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8116341a in my_write_core (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/stacktrace.c:386
 
#3  0x808e0d0a in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/signal_handler.cc:343
 
#4  <signal handler called>
 
#5  0xb776ac31 in __kernel_vsyscall ()
 
#6  0xb70f9f8f in poll () from /lib/i386-linux-gnu/libc.so.6
 
#7  0x8045b79a in handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:6481
 
#8  0x8045aa7e in mysqld_main (argc=158, argv=0x8337c330) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:5917
 
#9  0x8044e48b in main (argc=25, argv=0xbfcc57e4) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/main.cc:25
 
 
 
 - deleting it, already saved 0
 
 - found 'core' (0/0)
 
 
 
Trying 'dbx' to get a backtrace
 
 
 
Trying 'gdb' to get a backtrace from coredump /mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/4/log/rpl.rpl_parallel-innodb,mix/mysqld.1/data/core
 
Core generated by '/mnt/buildbot/build/mariadb-10.4.18/sql/mysqld'
 
Output from gdb follows. The first stack trace is from the failing thread.
 
The following stack traces are from all threads (so the failing one is
 
duplicated).
 
--------------------------
 
[New LWP 15843]
 
[New LWP 15852]
 
[New LWP 15873]
 
[New LWP 15864]
 
[New LWP 15855]
 
[New LWP 15863]
 
[New LWP 16110]
 
[New LWP 15876]
 
[New LWP 15958]
 
[New LWP 15874]
 
[New LWP 15875]
 
[New LWP 15881]
 
[New LWP 15938]
 
[New LWP 15885]
 
[New LWP 15853]
 
[New LWP 15934]
 
[New LWP 15917]
 
[New LWP 15854]
 
[New LWP 15892]
 
[New LWP 15859]
 
[New LWP 15988]
 
[New LWP 15862]
 
[New LWP 15966]
 
[New LWP 15868]
 
[New LWP 16127]
 
[New LWP 15861]
 
[New LWP 16114]
 
[New LWP 15872]
 
[New LWP 15871]
 
[New LWP 15870]
 
[New LWP 15869]
 
[New LWP 15865]
 
[New LWP 15858]
 
[New LWP 15857]
 
[New LWP 15856]
 
[New LWP 15851]
 
[Thread debugging using libthread_db enabled]
 
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
 
Core was generated by `/mnt/buildbot/build/mariadb-10.4.18/sql/mysqld --defaults-group-suffix=.1 --def'.
 
Program terminated with signal SIGABRT, Aborted.
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
[Current thread is 1 (Thread 0xb6e6a7c0 (LWP 15843))]
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76aace1 in pthread_kill () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111041a in my_write_core (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/stacktrace.c:386
 
#3  0x8088dd0a in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/signal_handler.cc:343
 
#4  <signal handler called>
 
#5  0xb774ac31 in __kernel_vsyscall ()
 
#6  0xb70d9f8f in poll () from /lib/i386-linux-gnu/libc.so.6
 
#7  0x8040879a in handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:6481
 
#8  0x80407a7e in mysqld_main (argc=149, argv=0x825b20b0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:5917
 
#9  0x803fb48b in main (argc=25, argv=0xbfb8f824) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/main.cc:25
 
 
 
Thread 36 (Thread 0xb2bcfb40 (LWP 15851)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x824b35a0 <COND_timer>, mp=0x824b3520 <LOCK_timer>, abstime=0xb2bcf250, file=0x815e0854 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x8111630c in inline_mysql_cond_timedwait (that=0x824b35a0 <COND_timer>, mutex=0x824b3520 <LOCK_timer>, abstime=0xb2bcf250, src_file=0x815e08a4 "/home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_timer.c", src_line=292) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x81116eeb in timer_handler (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_timer.c:292
 
#5  0x80aef6ad in pfs_spawn_thread (arg=0x826d34f8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 35 (Thread 0xae4fbb40 (LWP 15856)):
 
#0  0xb7716570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77165e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80cebb67 in LinuxAIOHandler::collect (this=0xae4fb0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80cec273 in LinuxAIOHandler::poll (this=0xae4fb0f0, m1=0xae4fb154, m2=0xae4fb158, request=0xae4fb178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80cec3d7 in os_aio_linux_handler (global_segment=3, m1=0xae4fb154, m2=0xae4fb158, request=0xae4fb178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80cf19c9 in os_aio_handler (segment=3, m1=0xae4fb154, m2=0xae4fb158, request=0xae4fb178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x80fad431 in fil_aio_wait (segment=3) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e1788e in io_handler_thread (arg=0x8249ebac <n+12>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 34 (Thread 0xadcfab40 (LWP 15857)):
 
#0  0xb7716570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77165e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80cebb67 in LinuxAIOHandler::collect (this=0xadcfa0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80cec273 in LinuxAIOHandler::poll (this=0xadcfa0f0, m1=0xadcfa154, m2=0xadcfa158, request=0xadcfa178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80cec3d7 in os_aio_linux_handler (global_segment=4, m1=0xadcfa154, m2=0xadcfa158, request=0xadcfa178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80cf19c9 in os_aio_handler (segment=4, m1=0xadcfa154, m2=0xadcfa158, request=0xadcfa178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x80fad431 in fil_aio_wait (segment=4) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e1788e in io_handler_thread (arg=0x8249ebb0 <n+16>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 33 (Thread 0xad4f9b40 (LWP 15858)):
 
#0  0xb7716570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77165e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80cebb67 in LinuxAIOHandler::collect (this=0xad4f90f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80cec273 in LinuxAIOHandler::poll (this=0xad4f90f0, m1=0xad4f9154, m2=0xad4f9158, request=0xad4f9178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80cec3d7 in os_aio_linux_handler (global_segment=5, m1=0xad4f9154, m2=0xad4f9158, request=0xad4f9178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80cf19c9 in os_aio_handler (segment=5, m1=0xad4f9154, m2=0xad4f9158, request=0xad4f9178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x80fad431 in fil_aio_wait (segment=5) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e1788e in io_handler_thread (arg=0x8249ebb4 <n+20>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 32 (Thread 0xa98f5b40 (LWP 15865)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfba8c in os_event::timed_wait (this=0x828a6488, abstime=0xa98f50f8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80cfbd9d in os_event::wait_time_low (this=0x828a6488, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80cfbf73 in os_event_wait_time_low (event=0x828a6488, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80e82357 in ib_wqueue_timedwait (wq=0x828a6368, wait_in_usecs=5000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/ut/ut0wqueue.cc:156
 
#6  0x80ff8b62 in fts_optimize_thread (arg=0x828a6368) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fts/fts0opt.cc:2830
 
#7  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 31 (Thread 0xa8effb40 (LWP 15869)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc1fe in os_event::wait (this=0x8278e788) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80cfbc5f in os_event::wait_low (this=0x8278e788, reset_sig_count=28) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80cfbfa5 in os_event_wait_low (event=0x8278e788, reset_sig_count=28) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e10306 in srv_resume_thread (slot=0x81a94124 <srv_sys+228>, sig_count=28, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e15003 in srv_purge_coordinator_suspend (slot=0x81a94124 <srv_sys+228>, rseg_history_len=53) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2657
 
#7  0x80e152f5 in srv_purge_coordinator_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2732
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 30 (Thread 0xa86feb40 (LWP 15870)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc1fe in os_event::wait (this=0x8278e8a8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80cfbc5f in os_event::wait_low (this=0x8278e8a8, reset_sig_count=117) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80cfbfa5 in os_event_wait_low (event=0x8278e8a8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e10306 in srv_resume_thread (slot=0x81a94178 <srv_sys+312>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e14b2e in srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2517
 
#7  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 29 (Thread 0xa7efdb40 (LWP 15871)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc1fe in os_event::wait (this=0x8278e848) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80cfbc5f in os_event::wait_low (this=0x8278e848, reset_sig_count=77) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80cfbfa5 in os_event_wait_low (event=0x8278e848, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e10306 in srv_resume_thread (slot=0x81a9415c <srv_sys+284>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e14b2e in srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2517
 
#7  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 28 (Thread 0xa74ffb40 (LWP 15872)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc1fe in os_event::wait (this=0x8278e7e8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80cfbc5f in os_event::wait_low (this=0x8278e7e8, reset_sig_count=92) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80cfbfa5 in os_event_wait_low (event=0x8278e7e8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80e10306 in srv_resume_thread (slot=0x81a94140 <srv_sys+256>, sig_count=0, wait=true, timeout_usec=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:910
 
#6  0x80e14b2e in srv_worker_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2517
 
#7  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 27 (Thread 0xa90f4b40 (LWP 16114)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xa90f416c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xa90f416c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xac31c910) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xac31c910, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82916138) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 26 (Thread 0xab8f9b40 (LWP 15861)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfba8c in os_event::timed_wait (this=0x828c0440, abstime=0xab8f9218) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80cfbd9d in os_event::wait_time_low (this=0x828c0440, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80cfbf73 in os_event_wait_time_low (event=0x828c0440, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80cb8fc2 in lock_wait_timeout_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/lock/lock0wait.cc:522
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 25 (Thread 0xa90aab40 (LWP 16127)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c60b44 <mysql_bin_log+1508>, mp=0x81c60a14 <mysql_bin_log+1204>, abstime=0xa90a8c00, file=0x813e6ccc "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x809b0a2b in inline_mysql_cond_timedwait (that=0x81c60b44 <mysql_bin_log+1508>, mutex=0x81c60a14 <mysql_bin_log+1204>, abstime=0xa90a8c00, src_file=0x813e79dc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc", src_line=8517) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x809c9c59 in MYSQL_BIN_LOG::wait_for_update_binlog_end_pos (this=0x81c60560 <mysql_bin_log>, thd=0xb0d1ebc8, timeout=0xa90a8c00) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc:8517
 
#5  0x805773ba in wait_new_events (info=0xa90a9264, linfo=0xa90a9048, binlog_end_pos_filename=0xa90a8c7c "./master-bin.000002", end_pos_ptr=0xa90a8c68) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_repl.cc:2511
 
#6  0x80577793 in get_binlog_end_pos (info=0xa90a9264, log=0xa90a8f74, linfo=0xa90a9048) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_repl.cc:2631
 
#7  0x80578112 in send_one_binlog_file (info=0xa90a9264, log=0xa90a8f74, linfo=0xa90a9048, start_pos=4) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_repl.cc:2791
 
#8  0x8057879d in mysql_binlog_send (thd=0xb0d1ebc8, log_ident=0xb0d2ad28 "master-bin.000002", pos=4, flags=2) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_repl.cc:2936
 
#9  0x8053808c in dispatch_command (command=COM_BINLOG_DUMP, thd=0xb0d1ebc8, packet=0xb0d06571 "", packet_length=27, is_com_multi=false, is_next_command=false) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_parse.cc:2103
 
#10 0x805357ad in do_command (thd=0xb0d1ebc8) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_parse.cc:1358
 
#11 0x806b132c in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1412
 
#12 0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#13 0x80aef6ad in pfs_spawn_thread (arg=0x82916300) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#14 0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#15 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 24 (Thread 0xac0fab40 (LWP 15868)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76ad8aa in nanosleep () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc5f6 in os_thread_sleep (tm=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0thread.cc:231
 
#3  0x80e1418a in srv_master_sleep () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2341
 
#4  0x80e14307 in srv_master_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:2381
 
#5  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#6  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 23 (Thread 0xac4f7b40 (LWP 15966)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xac4f716c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xac4f716c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0x829398e8) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0x829398e8, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82767c40) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 22 (Thread 0xab0f8b40 (LWP 15862)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfba8c in os_event::timed_wait (this=0x8278e908, abstime=0xab0f8118) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80cfbd9d in os_event::wait_time_low (this=0x8278e908, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80cfbf73 in os_event_wait_time_low (event=0x8278e908, time_in_usec=1000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80e12900 in srv_error_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:1854
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 21 (Thread 0xac4adb40 (LWP 15988)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xac4ad16c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xac4ad16c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xa6b05d58) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xa6b05d58, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82915da8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 20 (Thread 0xaccf8b40 (LWP 15859)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfba8c in os_event::timed_wait (this=0x8278ea28, abstime=0xaccf8078) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80cfbd9d in os_event::wait_time_low (this=0x8278ea28, time_in_usec=1000000, reset_sig_count=901) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80cfbf73 in os_event_wait_time_low (event=0x8278ea28, time_in_usec=1000000, reset_sig_count=901) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80f28803 in pc_sleep_if_needed (next_loop_time=2523260, sig_count=901, cur_time=2522260) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0flu.cc:2628
 
#6  0x80f29a1b in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0flu.cc:3087
 
#7  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 19 (Thread 0xb0efeb40 (LWP 15892)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811150af in safe_cond_wait (cond=0x81c38ae0 <COND_slave_background>, mp=0x81c3b140 <LOCK_slave_background>, file=0x811de248 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x8042e29e in inline_mysql_cond_wait (that=0x81c38ae0 <COND_slave_background>, mutex=0x81c3b140 <LOCK_slave_background>, src_file=0x811deecc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc", src_line=522) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x8042fc86 in handle_slave_background (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/slave.cc:522
 
#5  0x80aef6ad in pfs_spawn_thread (arg=0x82984d70) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 18 (Thread 0xaf4fdb40 (LWP 15854)):
 
#0  0xb7716570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77165e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80cebb67 in LinuxAIOHandler::collect (this=0xaf4fd0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80cec273 in LinuxAIOHandler::poll (this=0xaf4fd0f0, m1=0xaf4fd154, m2=0xaf4fd158, request=0xaf4fd178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80cec3d7 in os_aio_linux_handler (global_segment=1, m1=0xaf4fd154, m2=0xaf4fd158, request=0xaf4fd178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80cf19c9 in os_aio_handler (segment=1, m1=0xaf4fd154, m2=0xaf4fd158, request=0xaf4fd178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x80fad431 in fil_aio_wait (segment=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e1788e in io_handler_thread (arg=0x8249eba4 <n+4>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 17 (Thread 0xb0eb4b40 (LWP 15917)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xb0eb416c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xb0eb416c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xa8f059d0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xa8f059d0, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82a4dfb0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 16 (Thread 0xb0e6ab40 (LWP 15934)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xb0e6a16c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xb0e6a16c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xafd059b8) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xafd059b8, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82a4e178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 15 (Thread 0xafcfeb40 (LWP 15853)):
 
#0  0xb7716570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77165e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80cebb67 in LinuxAIOHandler::collect (this=0xafcfe0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80cec273 in LinuxAIOHandler::poll (this=0xafcfe0f0, m1=0xafcfe154, m2=0xafcfe158, request=0xafcfe178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80cec3d7 in os_aio_linux_handler (global_segment=0, m1=0xafcfe154, m2=0xafcfe158, request=0xafcfe178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80cf19c9 in os_aio_handler (segment=0, m1=0xafcfe154, m2=0xafcfe158, request=0xafcfe178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x80fad431 in fil_aio_wait (segment=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e1788e in io_handler_thread (arg=0x8249eba0 <n>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 14 (Thread 0xb185bb40 (LWP 15885)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb7029e8c in sigwaitinfo () from /lib/i386-linux-gnu/libc.so.6
 
#2  0x803fb512 in my_sigwait (set=0xb185b1d0, sig=0xb185b1a0, code=0xb185b1a4) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/my_pthread.h:196
 
#3  0x804018bd in signal_hand (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:3251
 
#4  0x80aef6ad in pfs_spawn_thread (arg=0x8297da70) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#5  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#6  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 13 (Thread 0xafefeb40 (LWP 15938)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xafefe16c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xafefe16c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xac3067d0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xac3067d0, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82a4e340) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 12 (Thread 0xb18a5b40 (LWP 15881)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811150af in safe_cond_wait (cond=0x81c6125c <mysql_bin_log+3324>, mp=0x81c611f8 <mysql_bin_log+3224>, file=0x813e6ccc "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1174) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x809b0955 in inline_mysql_cond_wait (that=0x81c6125c <mysql_bin_log+3324>, mutex=0x81c611f8 <mysql_bin_log+3224>, src_file=0x813e79dc "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc", src_line=10061) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1174
 
#4  0x809cdf1d in binlog_background_thread (arg=0x0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/log.cc:10061
 
#5  0x80aef6ad in pfs_spawn_thread (arg=0x827095a0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 11 (Thread 0xa5afdb40 (LWP 15875)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc1fe in os_event::wait (this=0x8278ea88) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80cfbc5f in os_event::wait_low (this=0x8278ea88, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80cfbfa5 in os_event_wait_low (event=0x8278ea88, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80f0093d in buf_resize_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0buf.cc:3142
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 10 (Thread 0xa62feb40 (LWP 15874)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76ad8aa in nanosleep () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc5f6 in os_thread_sleep (tm=1000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0thread.cc:231
 
#3  0x80eeef1f in btr_defragment_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/btr/btr0defragment.cc:700
 
#4  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#5  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 9 (Thread 0xafe6ab40 (LWP 15958)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xafe6a16c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xafe6a16c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xb17022c0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xb17022c0, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82767a78) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 8 (Thread 0xa52fcb40 (LWP 15876)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x811150af in safe_cond_wait (cond=0xa52fc1c4, mp=0xa52fc1f8, file=0x8145a660 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1185) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:492
 
#3  0x80be74ad in inline_mysql_cond_wait (that=0xa52fc1c4, mutex=0xa52fc1f8, src_file=0x8145c4ec "/home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/handler/ha_innodb.cc", src_line=311) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1185
 
#4  0x80becdb8 in thd_destructor_proxy () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/handler/ha_innodb.cc:311
 
#5  0x80aef6ad in pfs_spawn_thread (arg=0x829140a8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 7 (Thread 0xac463b40 (LWP 16110)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x81c38aa0 <COND_thread_cache>, mp=0x81c3ab40 <LOCK_thread_cache>, abstime=0xac46316c, file=0x811ce024 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x803fbd89 in inline_mysql_cond_timedwait (that=0x81c38aa0 <COND_thread_cache>, mutex=0x81c3ab40 <LOCK_thread_cache>, abstime=0xac46316c, src_file=0x811d011c "/home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc", src_line=2695) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80400ad6 in cache_thread (thd=0xa4979050) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2694
 
#5  0x80400d64 in one_thread_per_connection_end (thd=0xa4979050, put_in_cache=true) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:2780
 
#6  0x806b139f in do_handle_one_connection (connect=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1423
 
#7  0x806b104f in handle_one_connection (arg=0x82a3c520) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/sql_connect.cc:1316
 
#8  0x80aef6ad in pfs_spawn_thread (arg=0x82915f70) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#9  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#10 0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 6 (Thread 0xaa8f7b40 (LWP 15863)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfba8c in os_event::timed_wait (this=0x8278e968, abstime=0xaa8f7208) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80cfbd9d in os_event::wait_time_low (this=0x8278e968, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80cfbf73 in os_event_wait_time_low (event=0x8278e968, time_in_usec=5000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80e12340 in srv_monitor_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0srv.cc:1690
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 5 (Thread 0xaecfcb40 (LWP 15855)):
 
#0  0xb7716570 in ?? () from /lib/i386-linux-gnu/libaio.so.1
 
#1  0xb77165e6 in io_getevents () from /lib/i386-linux-gnu/libaio.so.1
 
#2  0x80cebb67 in LinuxAIOHandler::collect (this=0xaecfc0f0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1819
 
#3  0x80cec273 in LinuxAIOHandler::poll (this=0xaecfc0f0, m1=0xaecfc154, m2=0xaecfc158, request=0xaecfc178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:1980
 
#4  0x80cec3d7 in os_aio_linux_handler (global_segment=2, m1=0xaecfc154, m2=0xaecfc158, request=0xaecfc178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:2034
 
#5  0x80cf19c9 in os_aio_handler (segment=2, m1=0xaecfc154, m2=0xaecfc158, request=0xaecfc178) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0file.cc:5607
 
#6  0x80fad431 in fil_aio_wait (segment=2) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/fil/fil0fil.cc:4316
 
#7  0x80e1788e in io_handler_thread (arg=0x8249eba8 <n+8>) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/srv/srv0start.cc:324
 
#8  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#9  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 4 (Thread 0xaa0f6b40 (LWP 15864)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfba8c in os_event::timed_wait (this=0x828e92d0, abstime=0xaa0f6238) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:275
 
#3  0x80cfbd9d in os_event::wait_time_low (this=0x828e92d0, time_in_usec=10000000, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:385
 
#4  0x80cfbf73 in os_event_wait_time_low (event=0x828e92d0, time_in_usec=10000000, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:485
 
#5  0x80f948eb in dict_stats_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/dict/dict0stats_bg.cc:476
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 3 (Thread 0xa6affb40 (LWP 15873)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9a8c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x80cfc1fe in os_event::wait (this=0x8278e9c8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:158
 
#3  0x80cfbc5f in os_event::wait_low (this=0x8278e9c8, reset_sig_count=1) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:325
 
#4  0x80cfbfa5 in os_event_wait_low (event=0x8278e9c8, reset_sig_count=0) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/os/os0event.cc:502
 
#5  0x80f1b156 in buf_dump_thread () at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/innobase/buf/buf0dump.cc:832
 
#6  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#7  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 2 (Thread 0xb16ffb40 (LWP 15852)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76a9e56 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111535a in safe_cond_timedwait (cond=0x8246aba0 <COND_checkpoint>, mp=0x8246ab20 <LOCK_checkpoint>, abstime=0xb16ff19c, file=0x8144c498 "/home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h", line=1211) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/thr_mutex.c:546
 
#3  0x80b58811 in inline_mysql_cond_timedwait (that=0x8246aba0 <COND_checkpoint>, mutex=0x8246ab20 <LOCK_checkpoint>, abstime=0xb16ff19c, src_file=0x8144c4e8 "/home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_servicethread.c", src_line=116) at /home/buildbot/buildbot/build/mariadb-10.4.18/include/mysql/psi/mysql_thread.h:1211
 
#4  0x80b58d76 in my_service_thread_sleep (control=0x81a8b4d8 <checkpoint_control>, sleep_time=30000000000) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_servicethread.c:115
 
#5  0x80b4c504 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/maria/ma_checkpoint.c:707
 
#6  0x80aef6ad in pfs_spawn_thread (arg=0x8275c9e8) at /home/buildbot/buildbot/build/mariadb-10.4.18/storage/perfschema/pfs.cc:1869
 
#7  0xb76a42b5 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
 
#8  0xb70e416e in clone () from /lib/i386-linux-gnu/libc.so.6
 
 
 
Thread 1 (Thread 0xb6e6a7c0 (LWP 15843)):
 
#0  0xb774ac31 in __kernel_vsyscall ()
 
#1  0xb76aace1 in pthread_kill () from /lib/i386-linux-gnu/libpthread.so.0
 
#2  0x8111041a in my_write_core (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/mysys/stacktrace.c:386
 
#3  0x8088dd0a in handle_fatal_signal (sig=6) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/signal_handler.cc:343
 
#4  <signal handler called>
 
#5  0xb774ac31 in __kernel_vsyscall ()
 
#6  0xb70d9f8f in poll () from /lib/i386-linux-gnu/libc.so.6
 
#7  0x8040879a in handle_connections_sockets () at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:6481
 
#8  0x80407a7e in mysqld_main (argc=149, argv=0x825b20b0) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/mysqld.cc:5917
 
#9  0x803fb48b in main (argc=25, argv=0xbfb8f824) at /home/buildbot/buildbot/build/mariadb-10.4.18/sql/main.cc:25
 
 
 
 - deleting it, already saved 0
 
 - saving '/mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/4/log/rpl.rpl_parallel-innodb,mix/' to '/mnt/buildbot/build/mariadb-10.4.18/mysql-test/var/log/rpl.rpl_parallel-innodb,mix/'

Comment by Alice Sherepa [ 2021-08-26 ]

http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos74-amd64-debug/builds/7015/steps/mtr/logs/stdio

binlog_encryption.rpl_parallel 'innodb,row' w3 [ fail ]
 
        Test ended at 2021-08-25 06:03:14
 
 
 
CURRENT_TEST: binlog_encryption.rpl_parallel
 
mysqltest: In included file "./include/wait_for_slave_param.inc": 
included from ./include/wait_for_slave_sql_to_start.inc at line 32:
 
included from ./include/wait_for_slave_to_start.inc at line 27:
 
included from ./include/start_slave.inc at line 35:
 
included from ./suite/rpl/include/rpl_parallel.inc at line 360:
 
included from /usr/share/mysql-test/suite/binlog_encryption/rpl_parallel.test at line 2:
 
At line 119: Timeout in include/wait_for_slave_param.inc
 
 
 
The result from queries just before the failure was:
 
< snip >
 
''))
 
master-bin.000002	1529	Xid	1	1560	COMMIT /* xid=8520 */
 
master-bin.000002	1560	Gtid	1	1602	BEGIN GTID 0-1-15
 
master-bin.000002	1602	Query	1	1695	use `test`; INSERT INTO t2 VALUES (20)
 
master-bin.000002	1695	Query	1	1788	use `test`; INSERT INTO t1 VALUES (20)
 
master-bin.000002	1788	Query	1	1881	use `test`; INSERT INTO t2 VALUES (21)
 
master-bin.000002	1881	Query	1	1978	use `test`; INSERT INTO t3 VALUES (20, 20)
 
master-bin.000002	1978	Xid	1	2009	COMMIT /* xid=8583 */
 
master-bin.000002	2009	Gtid	1	2051	BEGIN GTID 0-1-16
 
master-bin.000002	2051	Query	1	2147	use `test`; INSERT INTO t3 VALUES(21, 21)
 
master-bin.000002	2147	Xid	1	2178	COMMIT /* xid=8588 */
 
master-bin.000002	2178	Gtid	1	2220	BEGIN GTID 0-1-17
 
master-bin.000002	2220	Query	1	2316	use `test`; INSERT INTO t3 VALUES(22, 22)
 
master-bin.000002	2316	Xid	1	2347	COMMIT /* xid=8589 */
 
 
 
**** SHOW RELAYLOG EVENTS on server_1 ****
 
relaylog_name = 'No such row'
 
SHOW RELAYLOG EVENTS IN 'No such row';
 
Log_name	Pos	Event_type	Server_id	End_log_pos	Info
 
connection server_2;

Comment by Elena Stepanova [ 2023-06-10 ]

It was also seen to fail with a mutex error

2023-06-09 19:34:15 445 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000005' at position 3217, relay log './slave-relay-bin.000001' position: 4; GTID position '0-1-108,1-1-3,2-1-2,3-1-1'
 
safe_mutex: Trying to lock uninitialized mutex at /src/sql/rpl_parallel.cc, line 2381
 
230609 19:34:15 [ERROR] mysqld got signal 6 ;

Comment by Kristian Nielsen [ 2023-11-28 ]

I just got this trying to push to 10.4:

binlog_encryption.rpl_parallel 'innodb,row' w22 [ fail ]  timeout after 900 seconds
 
        Test ended at 2023-11-28 18:38:51
 
 
 
Test case timeout after 900 seconds
 
 
 
== /home/buildbot/amd64-ubuntu-2004-debug/build/mysql-test/var/22/log/rpl_parallel.log == 
65	65
 
66	66
 
67	67
 
68	68
 
69	69
 
70	70
 
SET debug_sync='RESET';
 
connection server_2;
 
SET debug_sync='now SIGNAL d0_cont';
 
SET debug_sync='now WAIT_FOR t1_waiting';
 
SET debug_sync='now SIGNAL d3_cont';
 
SET debug_sync='now WAIT_FOR t2_waiting';
 
SET debug_sync='now SIGNAL d1_cont';
 
SET debug_sync='now WAIT_FOR t3_waiting';
 
SET debug_sync='now SIGNAL d2_cont';
 
SET debug_sync='now WAIT_FOR t4_waiting';
 
KILL THD_ID;
 
SET debug_sync='now WAIT_FOR t3_killed';
 
SET debug_sync='now SIGNAL t1_cont';
 
include/wait_for_slave_sql_error.inc [errno=1317,1927,1964]

https://buildbot.mariadb.org/#/builders/369/builds/15486

(There are some different failures mentioned in this issue, but let's focus on this one.)
The problem is immediately clear from the stack traces, the server is deadlocked due to mutexes being taken in opposite order:

#3  0x000055e59c4f8370 in inline_mysql_mutex_lock (that=0x55e59ec252a0, src_file=0x55e59cede1f8 "/home/buildbot/amd64-ubuntu-2004-debug/build/sql/rpl_reporting.cc", src_line=42) at /home/buildbot/amd64-ubuntu-2004-debug/build/include/mysql/psi/mysql_thread.h:702
 
#4  0x000055e59c4f8587 in Slave_reporting_capability::report (this=0x55e59ec25298, level=ERROR_LEVEL, err_code=1927, extra_info=0x7f33bc0124c9 "Gtid 0-1-33", msg=0x55e59ccef935 "%s") at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/rpl_reporting.cc:42
 
#5  0x000055e59bf63c6b in slave_output_error_info (rgi=0x7f33bc011d60, thd=0x7f33d000c950) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/slave.cc:5141
 
#6  0x000055e59c25d835 in do_gco_wait (rgi=0x7f33bc011d60, gco=0x7f33bc02a690, did_enter_cond=0x7f343059fbf7, old_stage=0x7f343059fc90) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/rpl_parallel.cc:411
 
 
 
#3  0x000055e59c25ba76 in inline_mysql_mutex_lock (that=0x7f33bc014840, src_file=0x55e59cd70138 "/home/buildbot/amd64-ubuntu-2004-debug/build/sql/rpl_parallel.cc", src_line=2550) at /home/buildbot/amd64-ubuntu-2004-debug/build/include/mysql/psi/mysql_thread.h:702
 
#4  0x000055e59c263428 in rpl_parallel::workers_idle (this=0x55e59ec284b8) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/rpl_parallel.cc:2550
 
#5  0x000055e59bf5f2a3 in send_show_master_info_data (thd=0x7f33d8000d90, mi=0x55e59ec235d0, full=false, gtid_pos=0x7f3441369a50) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/slave.cc:3200
 
#6  0x000055e59bf5cee4 in show_master_info (thd=0x7f33d8000d90, mi=0x55e59ec235d0, full=false) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/slave.cc:2839
 
#7  0x000055e59c05ed0b in mysql_execute_command (thd=0x7f33d8000d90) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/sql_parse.cc:4199
 
#8  0x000055e59c06b608 in mysql_parse (thd=0x7f33d8000d90, rawbuf=0x7f33d80148e8 "SHOW SLAVE STATUS", length=17, parser_state=0x7f344136a380, is_com_multi=false, is_next_command=false) at /home/buildbot/amd64-ubuntu-2004-debug/build/sql/sql_parse.cc:8062

The worker thread is holding LOCK_parallel_entry in do_gco_wait() and trying to do slave_output_error_info() which waits for the mutex err_lock.

Meanwhile SHOW SLAVE STATUS is holding err_lock and trying to do rpl_parallel::workers_idle() which waits for LOG_parallel_entry.

Thus this is a real critical bug that can deadlock the server, and needs to be fixed.

A patch to make the failure reproduce reliably:

diff --git a/sql/rpl_parallel.cc b/sql/rpl_parallel.cc
 
index fc4434b75de..d6cf9d78b3a 100644
 
--- a/sql/rpl_parallel.cc
 
+++ b/sql/rpl_parallel.cc
 
@@ -408,6 +408,7 @@ do_gco_wait(rpl_group_info *rgi, group_commit_orderer *gco,
 
         thd->clear_error();
 
         thd->get_stmt_da()->reset_diagnostics_area();
 
         thd->send_kill_message();
 
+        my_sleep(300000);
 
         slave_output_error_info(rgi, thd);
 
         signal_error_to_sql_driver_thread(thd, rgi, 1);
 
         /*

Comment by Brandon Nesterenko [ 2023-11-28 ]

knielsen nice find! Looks to be fixable by MDEV-31894, which I had planned on doing in a future release, but may warrant as a bug-fix instead..

Comment by Kristian Nielsen [ 2023-11-28 ]

bnestere, that sounds good, will you take this bug then?

Comment by Brandon Nesterenko [ 2023-11-28 ]

Sure!

Comment by Andrei Elkin [ 2023-11-29 ]

Thanks Kristian for taking on it!

Just in case let me remind the test was split in 10.5 into a number smaller ones by

commit 714762ddb7ba0233ed584ed896e8f688fb085aeb
 
Author: Sujatha <sujatha.sivakumar@mariadb.com>
 
Date:   Mon Dec 23 17:48:01 2019 +0530
 
 
 
    MDEV-18648: slave_parallel_mode= optimistic default in 10.5

Comment by Brandon Nesterenko [ 2023-11-29 ]

Hi Andrei!

This is ready for review: PR-2882

Comment by Brandon Nesterenko [ 2023-12-11 ]

Pushed into 10.4 as 8dad51481b1

Generated at Thu Feb 08 07:43:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.