[MDEV-23988] SST failed: No route to host after set global wsrep_node_name on donor Created: 2020-10-20  Updated: 2021-12-23

Status: Open
Project: MariaDB Server
Component/s: Galera, wsrep
Affects Version/s: 10.4.15, 10.5.6
Fix Version/s: 10.4, 10.5

Type: Bug Priority: Minor
Reporter: Nickolay Ihalainen Assignee: Alexey
Resolution: Unresolved Votes: 1
Labels: None
Environment:

CentOS Linux release 7.8.2003 (Core)



 Description   

Steps to reproduce:
1. Setup MariaDB Galera Cluster node0, node1, node2
2. Change wsrep_node_name with set global, without restart
node1> set global wsrep_node_name='my-node-name';
3. Update my.cnf on node2
wsrep_sst_donor="my-node-name"
4. Stop mariadb service on node2 and force SST
systemctl stop mariadb ; rm -rf /var/lib/mysql/grastate.dat ; systemctl start mariadb

Startup failed:

journalctl -u mariadb
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: State transfer required:
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: Group state: 3f285531-1284-11eb-9209-e2d05afd7a49:203
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: Local state: 00000000-0000-0000-0000-000000000000:-1
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: Server status change connected -> joiner
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 0 [Note] WSREP: Joiner monitor thread started to monitor
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 0 [Note] WSREP: Running: 'wsrep_sst_rsync --role 'joiner' --address '10.0.3.213' --datadir '/var/lib/mysql/' --parent '19364' --binlog 'mysqld-bin' --mysqld-args --wsrep_s
Oct 20 03:58:20 ihanick-node2 rsyncd[19426]: rsyncd version 3.1.2 starting, listening on port 4444
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: Prepared SST request: rsync|10.0.3.213:4444/rsync_sst
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: ####### IST uuid:00000000-0000-0000-0000-000000000000 f: 0, l: 203, STRv: 3
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: IST receiver addr using tcp://10.0.3.213:4568
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: Prepared IST receiver for 0-203, listening at: tcp://10.0.3.213:4568
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 0 [Warning] WSREP: Member 1.0 (cluster1-node-167773141) requested state transfer from 'my-node-name', but it is impossible to select State Transfer donor: No route to host
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [ERROR] WSREP: Requesting state transfer failed: -113(No route to host)
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [ERROR] WSREP: State transfer request failed unrecoverably: 113 (No route to host). Most likely it is due to inability to communicate with the cluster primary component.
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: ReplicatorSMM::abort()
Oct 20 03:58:20 ihanick-node2 mariadbd[19364]: 2020-10-20  3:58:20 2 [Note] WSREP: Closing send monitor...
Oct 20 03:58:20 ihanick-node2 systemd[1]: mariadb.service: main process exited, code=killed, status=6/ABRT

Exactly the same setup works fine if wsrep_node_name is modified by my.cnf change + restart.



 Comments   
Comment by Daniel Black [ 2020-10-20 ]

Due to belligerence Codership still don't want meaningful error messages.

"no route to host" it seems still equates to the wsrep_sst_donor wasn't recognised.

Sorry I unsuccessfully tried to change this: https://github.com/codership/galera/pull/109

Good test case. Thanks for the bug report.

Generated at Thu Feb 08 09:26:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.