Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
10.11.10
-
Ubuntu 20.04
Description
When upgrading one node in my Galera cluster from 10.11.9 to 10.11.10, i noticed that the upgraded node was unable to start correctly. The following lines were appearing in syslog"
Nov 29 13:19:34 node02 -wsrep-sst-joiner: Cleanup after exit with status: 1
|
Nov 29 13:19:34 node02 mariadbd[2661482]: 2024-11-29 13:19:34 0 [ERROR] WSREP: Failed to read 'ready <addr>' (got '(null)') from: wsrep_sst_mariabackup --role 'joiner' --address '10.97.149.12' --datadir '/var/lib/mysql/' --parent 2661482 --progress 0 --binlog '/var/log/mysql/mysql-bin' --binlog-index '/var/log/mysql/mariadb-bin.index' --mysqld-args --wsrep_start_position=e3a052f3-0858-11ef-b83e-67fa45d180b1:191
|
Nov 29 13:19:34 node02 mariadbd[2661482]: 2024-11-29 13:19:34 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '10.97.149.12' --datadir '/var/lib/mysql/' --parent 2661482 --progress 0 --binlog '/var/log/mysql/mysql-bin' --binlog-index '/var/log/mysql/mariadb-bin.index' --mysqld-args --wsrep_start_position=e3a052f3-0858-11ef-b83e-67fa45d180b1:191: 1 (Operation not permitted)
|
Nov 29 13:19:34 node02 mariadbd[2661482]: 2024-11-29 13:19:34 2 [ERROR] WSREP: Failed to prepare for 'mariabackup' SST. Unrecoverable.
|
Nov 29 13:19:34 node02 mariadbd[2661482]: 2024-11-29 13:19:34 2 [ERROR] WSREP: SST request callback failed. This is unrecoverable, restart required.
|
I've attached some more logging in an attachment
I was able to fix replication by replacing the new script with the old script, used in 10.11.9, in /usr/bin/wsrep_sst_mariabackup
Attachments
Activity
@Julius Goryavsky, I'm a colleague of @Lann Veltmaat, I've updated the scripts on one of our clusters running 10.11.10. Both a full transfer (removed all data from /var/lib/mysql) and an incremental transfer (reboot of a node) worked.
I don't believe we use rsync in any way (please correct me if I'm wrong), so I'm not sure if the update of wsrep_sst_rsync.sh did anything / was included in the test, common and mariabackup are confirmed to be working in my opinion.
Looking forward to the release, thank you for you quick action.
Fixed by the latest changes to the scripts that will be included in the 10.11.11 release.
rtuk Thanks for testing, the rsync script is not needed in your case, but I attached it just in case (if you suddenly use this sst mode in some scenario), there are also some fixes compared to the previous release.
Can you please indicate what in the script was broken.
Will I experience the same issue when upgrading from 10.11.9 to 10.11.10?
lveltmaat Hello, thanks for reporting this bug. Can you check the new scripts attached? I hope they fix this error. wsrep_sst_common.sh
wsrep_sst_mariabackup.sh
wsrep_sst_rsync.sh