[MDEV-18534] galera_sr.GCF-1018B: Test failure: mysqltest: mysqltest: Can't connect to local MySQL server; signal 6 Created: 2019-02-11  Updated: 2022-01-20  Resolved: 2022-01-20

Status: Closed
Project: MariaDB Server
Component/s: Galera, Tests
Affects Version/s: 10.4.3, 10.5
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Stepan Patryshev (Inactive) Assignee: Jan Lindström (Inactive)
Resolution: Fixed Votes: 0
Labels: galera_4, tests
Environment:

OS: CentOS Linux release 7.6.1810 (Core).


Attachments: Zip Archive 190208_galera_sr.GCF-1018B-innodb.zip     Text File 190208_galera_sr_stdout_desktop.log    
Issue Links:
Relates
relates to MDEV-17046 Galera 4: Streaming replication Closed
relates to MDEV-21613 galera_sr.GCF-1018B MTR failed: Faile... Closed
relates to MDEV-23328 Server hang due to Galera lock confli... Closed
relates to MDEV-24915 Galera conflict resolution is unneces... Closed

 Description   

MariaDB Server: 10.4.3-MariaDB-debug built from sources, commit: eb1d7aeeea2978f5be10cf95e6f638f184cfe4ad.
Galera4 lib: debug built from sources.

Run:

./mtr --suite galera_sr --force --big-test --max-test-fail=0 --parallel=auto

stdout.log: (the whole file is attached: 190208_galera_sr_stdout_desktop.log)

galera_sr.GCF-1018B 'innodb'             w3 [ fail ]  timeout after 900 seconds
 
mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/home/stepan/galera/git/10.4/server/mysql-test/var/tmp/3/mysqld.' (111)
 
CURRENT_TEST: galera_sr.GCF-1018B
mysqltest got signal 6

See also attached 190208_galera_sr.GCF-1018B-innodb.zip.



 Comments   
Comment by Stepan Patryshev (Inactive) [ 2019-03-29 ]

Closing as not reproduced. Last time it failed on my machine on 11.02.2019.

Comment by Marko Mäkelä [ 2021-03-19 ]

I am seeing a crash of this test on 10.5:

CURRENT_TEST: galera_sr.GCF-1018B
mysqltest: At line 37: query 'reap' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
2021-03-19 12:29:12 2 [Note] WSREP: Synchronized with group, ready for connections
2021-03-19 12:29:12 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
mariadbd: /mariadb/10.5m/wsrep-lib/src/transaction.cpp:193: void wsrep::transaction::adopt(const wsrep::transaction &): Assertion `transaction.is_streaming()' failed.
#7  0x00005644579d22d7 in wsrep::transaction::adopt (this=0x7f7dbc007470, transaction=@0x7f7e70011b70: {server_service_ = @0x564458f6cea0, client_service_ = @0x7f7e70011af0, client_state_ = @0x7f7e70011b08, server_id_ = {data_ = {buf = "\360F,&\210\235\021\353\235b\017\345\205\321\065\006"}}, id_ = {id_ = 18446744073709551615}, state_ = wsrep::transaction::s_aborted, state_hist_ = std::vector of length 5, capacity 16 = {wsrep::transaction::s_executing, wsrep::transaction::s_certifying, wsrep::transaction::s_executing, wsrep::transaction::s_must_abort, wsrep::transaction::s_aborting}, bf_abort_state_ = wsrep::transaction::s_executing, bf_abort_provider_status_ = wsrep::provider::success, bf_abort_client_state_ = 0, bf_aborted_in_total_order_ = false, ws_handle_ = {transaction_id_ = {id_ = 18446744073709551615}, opaque_ = 0x0}, ws_meta_ = {gtid_ = {id_ = {data_ = {buf = '\000' <repeats 15 times>}}, seqno_ = {seqno_ = -1}}, stid_ = {server_id_ = {data_ = {buf = '\000' <repeats 15 times>}}, transaction_id_ = {id_ = 18446744073709551615}, client_id_ = {id_ = 18446744073709551615}}, depends_on_ = {seqno_ = -1}, flags_ = 0}, flags_ = 0, pa_unsafe_ = false, implicit_deps_ = false, certified_ = false, fragments_certified_for_statement_ = 0, streaming_context_ = {fragments_certified_ = 0, fragments_ = std::vector of length 0, capacity 1, rollback_replicated_for_ = {id_ = 18446744073709551615}, fragment_unit_ = wsrep::streaming_context::bytes, fragment_size_ = 64, unit_counter_ = 0, log_position_ = 0}, sr_keys_ = {root_ = std::map with 0 elements}, apply_error_buf_ = {buffer_ = std::vector of length 0, capacity 0}, xid_ = {format_id_ = -1, gtrid_len_ = 0, bqual_len_ = 0, data_ = {buffer_ = std::vector of length 0, capacity 0}}}) at /mariadb/10.5m/wsrep-lib/src/transaction.cpp:193
#8  0x00005644574debf7 in wsrep::client_state::adopt_transaction (this=<optimized out>, transaction=<optimized out>) at /mariadb/10.5m/wsrep-lib/include/wsrep/client_state.hpp:693
#9  Wsrep_high_priority_service::adopt_transaction (this=0x7f7dbc025170, transaction=@0x7f7e94588390: {server_service_ = @0x87007, client_service_ = @0x7f7eabd0366c, client_state_ = @0x7f7efbad8000, server_id_ = {data_ = {buf = "\260\211\000\274}\177\000\000\025\212\000\274}\177\000"}}, id_ = {id_ = 140178001791408}, state_ = 3154151856, state_hist_ = std::vector of length 33, capacity -41 = {38144, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 577536, 8355260, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 2408550144, 2408550287, 2408550287, 8591, wsrep::transaction::s_executing, 32768, 8355260, 32768, 8355260, 8192, wsrep::transaction::s_executing, 82944, wsrep::transaction::s_executing, 2957303808, 8355440, 577536, 8355456, wsrep::transaction::s_executing, 2408550144, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 2408550287, 82063, wsrep::transaction::s_executing, 75008, wsrep::transaction::s_executing, 47360, 8355072, 948322304, 8355499, 71680, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 512, wsrep::transaction::s_executing, 1269504, wsrep::transaction::s_executing, 4294967040, 4294967295, 12543, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 1381193472, 1767395397, 905572198, 295536663, 50703339, 1262949911, 539586, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 256, 172288, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 2408550144, 2408550287, 766601359, 5653593, 87808, 87808, 2400573440, 399, 87808, 2400621056, 143, wsrep::transaction::s_executing, 2179278848, 8355439, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, wsrep::transaction::s_executing, 75008, wsrep::transaction::s_executing, 47360, 8355072...}, bf_abort_state_ = 3154152156, bf_abort_provider_status_ = 32637, bf_abort_client_state_ = 0, bf_aborted_in_total_order_ = false, ws_handle_ = {transaction_id_ = {id_ = 0}, opaque_ = 0x0}, ws_meta_ = {gtid_ = {id_ = {data_ = {buf = '\000' <repeats 15 times>}}, seqno_ = {seqno_ = -6442450945}}, stid_ = {server_id_ = {data_ = {buf = '\377' <repeats 16 times>}}, transaction_id_ = {id_ = 18446744073709551615}, client_id_ = {id_ = 18446744073709551615}}, depends_on_ = {seqno_ = -1}, flags_ = -1}, flags_ = -1, pa_unsafe_ = 255, implicit_deps_ = 255, certified_ = 255, fragments_certified_for_statement_ = 18446744073709551615, streaming_context_ = {fragments_certified_ = 18446744073709551615, fragments_ = std::vector of length 0, capacity 0, rollback_replicated_for_ = {id_ = 18446744073709551615}, fragment_unit_ = (wsrep::streaming_context::row | wsrep::streaming_context::statement | unknown: 0xfffffffc), fragment_size_ = 18446744073709551615, unit_counter_ = 94851527880923, log_position_ = 9914722007860092160}, sr_keys_ = {root_ = std::map with 0 elements}, apply_error_buf_ = {buffer_ = std::vector of length 0, capacity 0}, xid_ = {format_id_ = 0, gtrid_len_ = 0, bqual_len_ = 0, data_ = {buffer_ = std::vector of length 0, capacity 0}}}) at /mariadb/10.5m/sql/wsrep_high_priority_service.cc:239
#10 0x00005644579cfde1 in wsrep::server_state::convert_streaming_client_to_applier (this=0x564458f6cbd0, client_state=0x7f7e70011b08) at /mariadb/10.5m/wsrep-lib/src/server_state.cpp:1191
#11 0x00005644579d4d61 in wsrep::transaction::streaming_rollback (this=this@entry=0x7f7e70011b70, lock=@0x7f7e94588c30: {_M_device = 0x7f7e70011ad0, _M_owns = false}) at /mariadb/10.5m/wsrep-lib/src/transaction.cpp:1859
#12 0x00005644579d63ae in wsrep::transaction::bf_abort (this=0x7f7e70011b70, lock=@0x7f7e94588c30: {_M_device = 0x7f7e70011ad0, _M_owns = false}, bf_seqno=<optimized out>) at /mariadb/10.5m/wsrep-lib/src/transaction.cpp:1003
#13 0x00005644574f8590 in wsrep::client_state::bf_abort (this=<optimized out>, bf_seqno={seqno_ = 2187}) at /mariadb/10.5m/wsrep-lib/include/wsrep/client_state.hpp:669
#14 0x00005644574f7d40 in wsrep_bf_abort (bf_thd=bf_thd@entry=0x7f7e8005d0c8, victim_thd=<optimized out>, victim_thd@entry=0x7f7e7000b688) at /mariadb/10.5m/sql/wsrep_thd.cc:362
#15 0x00005644574fea1c in wsrep_thd_bf_abort (bf_thd=0x7f7e8005d0c8, victim_thd=0x7f7e7000b688, signal=1 '\001') at /mariadb/10.5m/sql/service_wsrep.cc:214
#16 0x0000564457529c7c in bg_wsrep_kill_trx (void_arg=0x7f7e70b055a0) at /mariadb/10.5m/storage/innobase/handler/ha_innodb.cc:18847

Note: the bg_wsrep_kill_trx thread was removed in 10.6 by MDEV-24915. Therefore, 10.6 might be unaffected by this.

Comment by Marko Mäkelä [ 2021-03-19 ]

The deadlock resolution was recently changed in MDEV-23328.

Generated at Thu Feb 08 08:44:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.