[MDEV-12979] rpl.rpl_parallel_mdev6589 fails in buildbot (Assertion `count > 0' failed) Created: 2017-06-02  Updated: 2020-09-04  Resolved: 2020-09-04

Status: Closed
Project: MariaDB Server
Component/s: Replication, Tests
Affects Version/s: 10.0, 10.1, 10.2
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Andrei Elkin
Resolution: Cannot Reproduce Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-14277 binlog_encryption.rpl_parallel faile... Closed

 Description   

http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/8172/steps/test_2/logs/stdio

rpl.rpl_parallel_mdev6589 'innodb_plugin,stmt' w4 [ fail ]
        Test ended at 2017-05-17 09:46:32
 
CURRENT_TEST: rpl.rpl_parallel_mdev6589
mysqltest: In included file "./include/sync_with_master_gtid.inc": 
included from /mnt/buildbot/build/mariadb-10.0.31/mysql-test/suite/rpl/t/rpl_parallel_mdev6589.test at line 116:
At line 43: query 'let $_result= `SELECT master_gtid_wait('$master_pos', $_slave_timeout)`' failed: 2013: Lost connection to MySQL server during query

mysqld: /home/buildbot/buildbot/build/mariadb-10.0.31/sql/rpl_parallel.cc:1869: rpl_parallel_thread* rpl_parallel_thread_pool::get_thread(rpl_parallel_thread**, rpl_parallel_entry*): Assertion `count > 0' failed.
170517  9:46:31 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
Server version: 10.0.31-MariaDB-debug
key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=5
max_threads=153
thread_count=5
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61724 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
 
Thread pointer: 0xa1085b0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xa8a24e20 thread_stack 0x48000
mysys/stacktrace.c:267(my_print_stacktrace)[0x8a89035]
sql/signal_handler.cc:161(handle_fatal_signal)[0x8473d87]
[0xb7724400]
[0xb7724424]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb72001ef]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7203835]
/lib/i386-linux-gnu/libc.so.6(+0x27095)[0xb71f9095]
/lib/i386-linux-gnu/libc.so.6(+0x27147)[0xb71f9147]
sql/rpl_parallel.cc:1870(rpl_parallel_thread_pool::get_thread(rpl_parallel_thread**, rpl_parallel_entry*))[0x8402a37]
sql/rpl_parallel.cc:2012(rpl_parallel_entry::choose_thread(rpl_group_info*, bool*, PSI_stage_info_v1*, bool))[0x8402f58]
sql/rpl_parallel.cc:2543(rpl_parallel::do_event(rpl_group_info*, Log_event*, unsigned long long))[0x840414b]
sql/slave.cc:3581(exec_relay_log_event)[0x81e9613]
sql/slave.cc:4752(handle_slave_sql)[0x81ece54]
perfschema/pfs.cc:1862(pfs_spawn_thread)[0x863e698]
/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb76b3d4c]
/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb72bcace]
 
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): 
Connection ID (thread ID): 981
Status: NOT_KILLED



 Comments   
Comment by Alice Sherepa [ 2017-11-06 ]

on 10.0, sporadic, http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/10283/steps/test_3/logs/stdio

rpl.rpl_parallel_mdev6589 'innodb_plugin,stmt' w4 [ fail ]
        Test ended at 2017-11-06 00:08:19
 
CURRENT_TEST: rpl.rpl_parallel_mdev6589
mysqltest: In included file "./include/sync_with_master_gtid.inc": 
included from /mnt/buildbot/build/mariadb-10.0.34/mysql-test/suite/rpl/t/rpl_parallel_mdev6589.test at line 116:
At line 48: Failed to sync with master
 
The result from queries just before the failure was:
< snip >
11
17
18
19
SELECT * FROM t2 ORDER BY a;
a
1
3
5
8
10
14
15
16
SET sql_log_bin=0;
DELETE FROM t2 WHERE a=5;
SET sql_log_bin=1;
include/start_slave.inc
include/sync_with_master_gtid.inc
Timeout in master_gtid_wait('1-1-3,0-1-12,3-1-4,2-1-4', 120), current slave GTID position is: 0-1-12,3-1-4,2-1-4.
 
More results from queries before failure can be found in /mnt/buildbot/build/mariadb-10.0.34/mysql-test/var/4/log/rpl_parallel_mdev6589.log
 
 - saving '/mnt/buildbot/build/mariadb-10.0.34/mysql-test/var/4/log/rpl.rpl_parallel_mdev6589-innodb_plugin,stmt/' to '/mnt/buildbot/build/mariadb-10.0.34/mysql-test/var/log/rpl.rpl_parallel_mdev6589-innodb_plugin,stmt/'
 
Retrying test rpl.rpl_parallel_mdev6589, attempt(2/3)...
 
worker[4] > Restart  - not started
worker[4] > Restart  - not started
***Warnings generated in error logs during shutdown after running tests: rpl.rpl_gtid_startpos rpl.rpl_commit_after_flush rpl.rpl_gtid_errorhandling rpl.rpl_foreign_key_innodb rpl.rpl_old_master rpl.rpl_bug26395 rpl.rpl_drop_db_fail rpl.rpl_gtid_mdev4484 rpl.rpl_failed_optimize rpl.rpl_dual_pos_advance rpl.rpl_gtid_ignored rpl.rpl_insert_id_pk rpl.rpl_gtid_reconnect rpl.rpl_insert_ignore rpl.rpl_insert_id rpl.rpl_parallel rpl.rpl_mdev10863 rpl.rpl_multi_engine rpl.rpl_parallel_mdev6589 rpl.rpl_gtid_until rpl.rpl_mysql_upgrade rpl.rpl_checksum_cache
 
171106  0:06:18 [Warning] Slave: Duplicate entry '5' for key 'PRIMARY' Error_code: 1062

Comment by Andrei Elkin [ 2018-06-15 ]

According to BB 'Assertion `count > 0' – still happening about once in a month on 10.1.

Comment by Alice Sherepa [ 2020-09-04 ]

the latest failure was 2018-08-28, I am closing the bug

Generated at Thu Feb 08 08:01:58 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.