[MDEV-24097] galera_3nodes suite tests in MTR sporadically fails: Failed to start mysqld or mysql_shutdown failed Created: 2020-11-02  Updated: 2022-02-18  Resolved: 2022-01-28

Status: Closed
Project: MariaDB Server
Component/s: Galera, Tests
Affects Version/s: 10.2.41, 10.3.32, 10.4.22, 10.5.13, 10.6.5, 10.7.1
Fix Version/s: 10.2.42, 10.3.33, 10.4.23, 10.5.14, 10.6.6, 10.7.2, 10.8.1

Type: Bug Priority: Critical
Reporter: Ramesh Sivaraman Assignee: Julius Goryavsky
Resolution: Fixed Votes: 0
Labels: None
Environment:

kvm-deb-stretch-amd64 kvm-deb-jessie-amd64


Attachments: Zip Archive 201023_10_4_ES_galera_safe_to_bootstrap.zip     Text File 201023_10_4_ES_stdout.log     Zip Archive MDEV-24097_104_logs_201215.zip    
Issue Links:
Issue split
split to MDEV-27524 Incorrect binlogs after Galera SST us... Closed
Relates
relates to MDEV-23747 socket.ssl_ca argument being ignored. Closed
relates to MDEV-25880 rsync may be mistakenly killed when o... Closed
relates to MDEV-24481 galera_3nodes.galera_vote_rejoin_mysq... Closed

 Description   

galera_3nodes suite is unstable in following test cases:

  • galera_pc_bootstrap
  • galera_ipv6_mariabackup
  • galera_ipv6_mariabackup_section
  • galera_ipv6_rsync
  • galera_ipv6_rsync_section
  • galera_ssl_reload
  • galera_toi_vote
  • galera_wsrep_schema_init

because MTR sporadaically fails: Failed to start mysqld or mysql_shutdown failed

Latest occurrence in 10.6 as a example: http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/8868/steps/mtr-galera/logs/stdio



 Comments   
Comment by Stepan Patryshev (Inactive) [ 2020-12-22 ]

Reopened since it was sporadaically reproduced on BB 10.4 CS, BB 10.5 CS and BB 10.6 CS: "mysql_shutdown failed".
stdio.log:

10.4.18, d4c35fb21b11f05b317d2e98de42f9a2cab46c27, kvm-rpm-rhel8-amd64

galera_3nodes.galera_safe_to_bootstrap 'innodb' w2 [ fail ]
        Test ended at 2020-12-15 06:43:24
 
CURRENT_TEST: galera_3nodes.galera_safe_to_bootstrap
mysqltest: In included file "./include/shutdown_mysqld.inc": 
included from /usr/share/mysql-test/suite/galera_3nodes/t/galera_safe_to_bootstrap.test at line 50:
At line 40: mysql_shutdown failed
 
The result from queries just before the failure was:
connection node_2;
connection node_1;
connection node_1;
connection node_2;
connection node_3;
connection node_1;
CREATE TABLE t1 (f1 INTEGER NOT NULL PRIMARY KEY) ENGINE=InnoDB;
INSERT INTO t1 VALUES (1);
include/assert_grep.inc [grastate.dat does not have 'safe_to_bootstrap: 0']
include/assert_grep.inc [grastate.dat does not have 'safe_to_bootstrap: 0']
include/assert_grep.inc [grastate.dat does not have 'safe_to_bootstrap: 0']
connection node_2;
 
 - saving '/dev/shm/var/2/log/galera_3nodes.galera_safe_to_bootstrap-innodb/' to '/dev/shm/var/log/galera_3nodes.galera_safe_to_bootstrap-innodb/'
 
Retrying test galera_3nodes.galera_safe_to_bootstrap, attempt(2/3)...
 
worker[2] > Restart  - not started
worker[2] > Restart  - not started
worker[2] > Restart  - not started

Server logs.

Comment by Jan Lindström (Inactive) [ 2021-06-04 ]

Test does fail at least on buildbot.

Comment by Jan Lindström (Inactive) [ 2021-12-21 ]

I'm ok with these changes. Note that change on mysql-test/suite/galera/t/galera_autoinc_sst_xtrabackup.cnf has no effect (please check is this intentional).

Comment by Julius Goryavsky [ 2022-01-28 ]

Fixed and merged, remaining binlogs issue moved to a separate task (MDEV-27524)

Generated at Thu Feb 08 09:27:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.