Details
-
Task
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
For a HA solution like galera to require downtime to perform an upgrade inconvenience for the users. A traditional async replication based upgrade can be done without downtime, so our HA product Galera should be able to as well/
Problem: (MDEV-27347 summary)
The mariadb-backup sst doesn't work between major version as the donor streams the result to the joiner. This stream and the saved version afterwards has not had its redo log applied and is effectively crashed. The joiner mariadb-backup being a newer version won't apply the changes in the prepare stage.
Fix:
The mariadb-backup prepare part of the SST could be performed by the donor using a sshfs mount SSHFS exported by the JOINER. This enables the mariadb-backup of the right version to apply the redo log without error.
tested (out of a sst context) with:
socat TCP-LISTEN:4444,reuseaddr EXEC:"/usr/libexec/openssh/sftp-server -e -d $DATA"
|
The xtrabackup_info info file contains the donor server version so this can be made conditional.
Donor:
localdir=$(mktemp -d)
|
socat TCP:$REMOTEIP:4444 EXEC:"sshfs \:$DATA $localdir -o passive" &
|
# or sshfs $REMOTEIP:$DATA $localdir -o directport=4444 &
|
fusepid=$!
|
mariabackup --target-dir=$localdir --prepare
|
kill $fusepid
|
Note, not socat -u - bidirection is needed
Alternate untested modification for the sst script on the joiner side:
diff --git a/scripts/wsrep_sst_mariabackup.sh b/scripts/wsrep_sst_mariabackup.sh
|
index d6334052f24..c00fc57b390 100644
|
--- a/scripts/wsrep_sst_mariabackup.sh
|
+++ b/scripts/wsrep_sst_mariabackup.sh
|
@@ -1512,7 +1512,10 @@ else # joiner
|
|
wsrep_log_info "Preparing the backup at $DATA"
|
setup_commands
|
- timeit 'mariadb-backup prepare stage' "$INNOAPPLY"
|
+ DONOR_SERVER_VERSION=$(grep server_version $DATA/xtrabackup_info)
|
+ DONOR_SERVER_VERSION=${DONOR_SERVER_VERSION#* = }
|
+ DONOR_SERVER_VERSION=${DONOR_SERVER_VERSION%%-*}
|
+ timeit 'mariadb-backup prepare stage' docker run --rm -v $DATA:/var/lib/mysql:z mariadb:$DONOR_SERVER_VERSION mariadb-backup --prepare > "$INNOAPPLYLOG" 2>&1
|
if [ $? -ne 0 ]; then
|
wsrep_log_error "mariadb-backup apply finished with errors." \
|
"Check syslog or '$INNOAPPLYLOG' for details." |
--defaults-file=/var/lib/mysql/backup-my.cnf might be needed as the first arg on mariadb-backup if non-standard innodb options related to storage are used.
Attachments
Issue Links
- relates to
-
MDEV-27437 Galera snapshot transfer fails to upgrade between some major versions
- Closed
-
MDEV-28483 Include the details about 10.4 to 10.5 and newer major versions upgrades for Galera clusters into the Knowledge Base
- Closed
-
MDEV-29162 Major upgrade instructions missing for Galera
- Closed
-
MDEV-31506 Updating cluster to a new major version Reports Upgrade after a crash is not supported with rsync sst
- Confirmed