[MDEV-19874] Unable to join galera node. SST ERROR with mariabackup Created: 2019-06-26  Updated: 2022-06-24  Resolved: 2022-02-22

Status: Closed
Project: MariaDB Server
Component/s: Backup, Galera SST
Affects Version/s: 10.3.16
Fix Version/s: 10.9.0, 10.2.44, 10.3.35, 10.4.25, 10.5.16, 10.6.8, 10.7.4, 10.8.3

Type: Bug Priority: Minor
Reporter: Jon Barca Assignee: Julius Goryavsky
Resolution: Fixed Votes: 0
Labels: None
Environment:

CentOS 7.6


Issue Links:
Relates
relates to MDEV-27524 Incorrect binlogs after Galera SST us... Closed

 Description   

JOINER node shows next error at startup:

_2019-06-26 12:38:30 2 [Note] WSREP: GCache history reset: 00000000-0000-0000-0000-000000000000:0 -> 7b7e89ca-973f-11e9-a0d6-2e5b7bc7be09:66
WSREP_SST: [INFO] Proceeding with SST (20190626 12:38:31.100)
WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20190626 12:38:31.103)
WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20190626 12:38:31.114)
find: ‘NULL’: No such file or directory
removed ‘/data/mariadb/data/aria_log_control’
removed ‘/data/mariadb/data/aria_log.00000001’
removed ‘/data/mariadb/data/ibdata1’
removed ‘/data/mariadb/data/ib_logfile1’
WSREP_SST: [ERROR] Cleanup after exit with status:1 (20190626 12:38:31.125)
2019-06-26 12:38:43 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '115.10.18.102' --datadir '/data/mariadb/data/' --parent '2379' --binlog '/data/mariadb/binlog/webregqbd-bin' '': 1 (Operation not permitted)
2019-06-26 12:38:43 0 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script.
2019-06-26 12:38:43 0 [ERROR] WSREP: SST failed: 1 (Operation not permitted)
2019-06-26 12:38:43 0 [ERROR] Aborting
Error in my_thread_global_end(): 1 threads didn't exit_

Backup works fine in DONOR node and DATADIR was recovered properly in JOINER node but mariabackup did not execute the steps of PREPARE and MOVE.

With wsrep_sst_method=rsync or running a mariabackup manually works fine.

WORKAROUND

Problem is because I do not have the innodb-undo-directory parameter defined in my.cnf.
I have modified the wsrep_sst_mariabackup script and now works well.

[root@vmxas1]$ diff /bin/wsrep_sst_mariabackup.orig /bin/wsrep_sst_mariabackup
1027c1027
< find $ib_home_dir $ib_log_dir $ib_undo_dir $DATA -mindepth 1 -prune -regex $cpat -o -exec rm -rfv {} 1>&2 +

> find $ib_home_dir $ib_log_dir $DATA -mindepth 1 -prune -regex $cpat -o -exec rm -rfv {} 1>&2 +

I do not know if it's a possible bug or just a mismatch...

Thanks in advance,
Jon K.



 Comments   
Comment by Jon Barca [ 2019-06-27 ]

Hi, obviously you can also comment "innodb_undo_directory=NULL" (default value) in my.cnf.

Regards,
JonK.

Comment by Julius Goryavsky [ 2022-02-18 ]

Fixed by MDEV-27524

Comment by Julius Goryavsky [ 2022-02-22 ]

Fixed by MDEV-27524

Generated at Thu Feb 08 08:55:01 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.