[MDEV-30172] Galera test case cleanup - Jira

Details

Type: Bug
Status: Stalled (View Workflow)
Priority: Major
Resolution: Unresolved
Affects Version/s: 10.4(EOL), 10.5, 10.6
Fix Version/s: 10.11
Component/s: Galera, Tests
Labels:
None

Description

Galera test case cleanup

Make sure that every test failure has open MDEV or MENT.
Make sure that disabled.def files are up-to-date.
Monitor buildbot and Jenkings test periodically for failing test cases.
Pick one linked here and fix it.

Attachments

Issue Links

is blocked by

MDEV-18590 galera.versioning_trx_id: Test failure: mysqltest: Result content mismatch

Closed

MDEV-24481 galera_3nodes.galera_vote_rejoin_mysqldump MTR failed: mysql_shutdown failed

Closed

MDEV-25089 Assertion `error.len > 0' failed in wsrep_status_t galera::ReplicatorSMM::handle_apply_error(galera::TrxHandleSlave&, const wsrep_buf_t&, const string&)

Closed

MDEV-25199 galera.galera_pc_recovery fails to start up

Closed

MDEV-25718 galera_sr.GCF-1060: void wsrep::transaction::adopt(const wsrep::transaction&): Assertion `transaction.is_streaming()' failed

Closed

MDEV-26528 galera_sr.GCF-1060: wrong usage of mutex LOCK_thd_kill and LOCK_thd_kill

Closed

MDEV-28378 galera.galera_as_slave_ctas fails with a timeout

Closed

MDEV-29861 Galera "notify" test cases hang

Closed

MDEV-29876 Galera test failure on galera_sst_encrypted

Closed

MDEV-29878 Galera test failure on MDEV-26575

Closed

MDEV-29882 Galera test failure on galera_sr_cc_master

Closed

MDEV-29892 Galera test failure on galera_sr_kill_slave_after_apply_rollback2

Closed

MDEV-29918 Assertion failure on galera_bf_abort_shutdown

Closed

MDEV-30536 no expected deadlock in galera_insert_bulk test

Closed

MDEV-30855 Remove test galera.galera_bf_abort_group_commit

Closed

MDEV-32160 GCF-1060 test failure due to wsrep MDL conflict

Closed

MDEV-32561 WSREP FSM failure: no such a transition REPLICATING -> COMMITTED

Closed

MDEV-32631 galera_2_cluster: before_rollback(): Assertion `0' failed

Closed

MDEV-32633 galera_gtid_2_cluster: Assertion `thd->wsrep_next_trx_id() != (0x7fffffffffffffffLL * 2ULL + 1)'

Closed

MDEV-32635 galera_shutdown_nonprim: mysql_shutdown failed

Closed

MDEV-32778 galera_ssl_reload failed with warning message

Closed

MDEV-32779 galera_concurrent_ctas: assertion in the galera::ReplicatorSMM::finish_cert()

Closed

MDEV-32780 galera_as_slave_replay: assertion in the wsrep::transaction::before_rollback()

Closed

MDEV-32781 galera_bf_lock_wait test failed

Closed

MDEV-32782 galera_sst_mysqldump_with_key test failed

Closed

MDEV-32993 Improve stability of node joining process

Closed

MDEV-32995 Remove not supported tests from wsrep suite

Closed

MDEV-32996 galera.galera_var_ignore_apply_errors -> [ERROR] WSREP: Inconsistency detected

Closed

MDEV-33000 Fix test result file on some test cases

Closed

MDEV-33035 Galera test case MDEV-16509 unstable

Closed

MDEV-33036 Galera test case galera_3nodes.galera_ist_gcache_rollover has warning

Closed

MDEV-33037 Galera test failure because TLS connection was non-properly terminated

Closed

MDEV-33038 Galera test failure on galera_concurrent_ctas

Closed

MDEV-33039 Galera test failure on mysql-wsrep-features#165

Closed

MDEV-33138 Galera test case MW-336 unstable

Closed

MDEV-33172 Galera test case galera_mdl_race unstable

Closed

MDEV-33173 Galera test case galera_sr_kill_slave_before_apply unstable

Closed

MDEV-33778 galera.galera_wan_restart_ist test failure

Closed

MDEV-34640 galera_var_ignore_apply_errors test freezes

Closed

relates to

MDEV-34640 galera_var_ignore_apply_errors test freezes

Closed

(34 is blocked by, 1 relates to)

Activity

Ascending order - Click to sort in descending order

View 9 older comments

Marko Mäkelä added a comment - 2022-12-16 10:18

Note: Locally I get even more failures, apparently related to a newer SSL library, like these:

CURRENT_TEST: galera_sr.MDEV-27615

mysqltest: At line 47: query 'reap' failed with wrong errno <Unknown> (2026): 'TLS/SSL error: unexpected eof while reading', instead of <Unknown> (2013)...

galera_3nodes.galera_dynamic_protocol 'innodb' w4 [ fail ]  Found warnings/errors in server log file!

        Test ended at 2022-12-16 11:29:19

line

2022-12-16 11:28:58 0 [Warning] WSREP: read_completion_condition(): unexpected eof while reading (167772454: 'error:0A000126:SSL routines::unexpected eof while reading')

2022-12-16 11:28:58 0 [Warning] WSREP: read_handler(): unexpected eof while reading (167772454: 'error:0A000126:SSL routines::unexpected eof while reading')

^ Found warnings in /dev/shm/10.10/mysql-test/var/4/log/mysqld.3.err

While I was looking for the Debian Sid failures on buildbot, I noticed lots of crashes on kvm-deb-sid-x86 like this:

CURRENT_TEST: galera.galera_bf_abort_ps_bind

mysqltest: At line 41: query 'update t set j='node2' where i=1' failed: <Unknown> (2013): Lost connection to server during query

…

#10 0x00ed8958 in vsnprintf (__ap=0xb0d93a68 "\002", __fmt=0x13d53c4 "wsrep_commit_empty for %llu client_state %s client_mode %s trans_state %s sql %s", __n=128, __s=0xb0d939bc "wsrep_commit_empty for 82112017988583426 client_state high priority client_mode executing trans_state NULL sql ") at /usr/include/i386-linux-gnu/bits/stdio2.h:85

#11 WSREP_LOG (fun=0xcfa120 <sql_print_information(char const*, ...)>, fmt=0x13d53c4 "wsrep_commit_empty for %llu client_state %s client_mode %s trans_state %s sql %s") at ./sql/wsrep_mysqld.cc:299

#12 0x00edec83 in wsrep_commit_empty (thd=0xb08006c8, all=true) at ./sql/wsrep_mysqld.cc:3693

The cause of the SIGSEGV is a vsnprintf format string mismatch:

void wsrep_commit_empty(THD* thd, bool all)

  DBUG_ENTER("wsrep_commit_empty");

  WSREP_DEBUG("wsrep_commit_empty for %llu client_state %s client_mode"

              " %s trans_state %s sql %s",

              thd_get_thread_id(thd),

…

The function thd_get_thread_id() returns unsigned long, not unsigned long long, so the first parameter in the format string should be %lu instead of %llu. No warning is issued at compilation time, due to the lack of ~~MDEV-21978~~.

Marko Mäkelä added a comment - 2022-12-16 10:18 Note: Locally I get even more failures, apparently related to a newer SSL library, like these: CURRENT_TEST: galera_sr.MDEV-27615 mysqltest: At line 47: query 'reap' failed with wrong errno <Unknown> (2026): 'TLS/SSL error: unexpected eof while reading', instead of <Unknown> (2013)... galera_3nodes.galera_dynamic_protocol 'innodb' w4 [ fail ] Found warnings/errors in server log file! Test ended at 2022-12-16 11:29:19 line 2022-12-16 11:28:58 0 [Warning] WSREP: read_completion_condition(): unexpected eof while reading (167772454: 'error:0A000126:SSL routines::unexpected eof while reading') 2022-12-16 11:28:58 0 [Warning] WSREP: read_handler(): unexpected eof while reading (167772454: 'error:0A000126:SSL routines::unexpected eof while reading') ^ Found warnings in /dev/shm/10.10/mysql-test/var/4/log/mysqld.3.err While I was looking for the Debian Sid failures on buildbot, I noticed lots of crashes on kvm-deb-sid-x86 like this: CURRENT_TEST: galera.galera_bf_abort_ps_bind mysqltest: At line 41: query 'update t set j='node2' where i=1' failed: <Unknown> (2013): Lost connection to server during query … #10 0x00ed8958 in vsnprintf (__ap=0xb0d93a68 "\002", __fmt=0x13d53c4 "wsrep_commit_empty for %llu client_state %s client_mode %s trans_state %s sql %s", __n=128, __s=0xb0d939bc "wsrep_commit_empty for 82112017988583426 client_state high priority client_mode executing trans_state NULL sql ") at /usr/include/i386-linux-gnu/bits/stdio2.h:85 #11 WSREP_LOG (fun=0xcfa120 <sql_print_information(char const*, ...)>, fmt=0x13d53c4 "wsrep_commit_empty for %llu client_state %s client_mode %s trans_state %s sql %s") at ./sql/wsrep_mysqld.cc:299 #12 0x00edec83 in wsrep_commit_empty (thd=0xb08006c8, all=true) at ./sql/wsrep_mysqld.cc:3693 The cause of the SIGSEGV is a vsnprintf format string mismatch: void wsrep_commit_empty(THD* thd, bool all) { DBUG_ENTER( "wsrep_commit_empty" ); WSREP_DEBUG( "wsrep_commit_empty for %llu client_state %s client_mode" " %s trans_state %s sql %s" , thd_get_thread_id(thd), … The function thd_get_thread_id() returns unsigned long , not unsigned long long , so the first parameter in the format string should be %lu instead of %llu . No warning is issued at compilation time, due to the lack of MDEV-21978 .

Sergei Golubchik added a comment - 2022-12-19 10:49

jplindst, there were more commits disabling more tests. Please, check them all before closing the bug

Sergei Golubchik added a comment - 2022-12-19 10:49 jplindst , there were more commits disabling more tests. Please, check them all before closing the bug

Marko Mäkelä added a comment - 2022-12-19 11:03

Moreover, many of the disabled tests refer only to this ticket, and hence this ticket should not be closed before all such tests have either been re-enabled or associated with more specific tickets.

Marko Mäkelä added a comment - 2022-12-19 11:03 Moreover, many of the disabled tests refer only to this ticket, and hence this ticket should not be closed before all such tests have either been re-enabled or associated with more specific tickets.

Marko Mäkelä added a comment - 2023-11-20 08:07

The test galera_sr.MDEV-25718 is still disabled in 10.11.

Marko Mäkelä added a comment - 2023-11-20 08:07 The test galera_sr. MDEV-25718 is still disabled in 10.11.

Marko Mäkelä added a comment - 3 days ago

The test galera_sr.MDEV-25718 is still disabled in 10.11. It currently is the only disabled test in that suite.

Marko Mäkelä added a comment - 3 days ago The test galera_sr. MDEV-25718 is still disabled in 10.11. It currently is the only disabled test in that suite.

People

Assignee:: Jan Lindström

Reporter:: Jan Lindström (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 2022-12-07 05:32

Updated:: 4 days ago 05:49

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server