[MDEV-30696] [ERROR] WSREP gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():780: Will never receive state. Need to abort. Created: 2023-02-21  Updated: 2023-02-21

Status: Open
Project: MariaDB Server
Component/s: Galera SST
Affects Version/s: 10.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Jon Barca Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Comments   
Comment by Jon Barca [ 2023-02-21 ]

Hello,
I have a galera cluster with MariaDB 10.2.38 (Galera 25.3.31) with two nodes started and when it tried to start the third node, it gives me the following error in joiner node:

*_*[Warning] WSREP: 1.0 (node-220): State transfer to 2.0 (node-221) failed: -1 (Operation not permitted)
[ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():780: Will never receive state. Need to abort.*_*

The error occurs with any method (mysqldump, mariabackup, rsync) and both when executing an IST and a full SST.

The following messages appear in the donor node:

_
[ERROR] WSREP: Failed to read from: wsrep_sst_mariabackup --role 'donor' --address '172.57.12.14:4444/xtrabackup_sst//1' --socket '/dbdata/tmp/mysqld.sock' --datadir '/dbdata/data/' --gtid '81c4043d-31d0-23e5-9980-0b30d4de12fs:44178191' --gtid-domain-id '69' --binlog '/dbdata/binlog/node-bin' --bypass --mysqld-args --basedir=/dbdata/exe/mariadb_bin --datadir=/dbdata/data --plugin-dir=/dbdata/exe/mariadb_bin/lib/plugin --user=mysql --wsrep_on=on --wsrep_provider=/dbdata/exe/mariadb_bin/lib/libgalera_smm.so --log-error=/dbdata/log/mysqld_err.log --open-files-limit=512 --pid-file=/dbdata/data/node-220.vm.clhermes.local.pid --socket=/dbdata/tmp/mysqld.sock --port=3499 --wsrep_start_position=81c4043d-31d0-23e5-9980-0b30d4de12fs:34911380
[ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'donor' --address '172.57.12.14:4444/xtrabackup_sst//1' --socket '/dbdata/tmp/mysqld.sock' --datadir '/dbdata/data/' --gtid '81c4043d-31d0-23e5-9980-0b30d4de12fs:44178191' --gtid-domain-id '69' --binlog '/dbdata/binlog/node-bin' --bypass --mysqld-args --basedir=/dbdata/exe/mariadb_bin --datadir=/dbdata/data --plugin-dir=/dbdata/exe/mariadb_bin/lib/plugin --user=mysql --wsrep_on=on --wsrep_provider=/dbdata/exe/mariadb_bin/lib/libgalera_smm.so --log-error=/dbdata/log/mysqld_err.log --open-files-limit=512 --pid-file=/dbdata/data/node-220.vm.clhermes.local.pid --socket=/dbdata/tmp/mysqld.sock --port=3499 --wsrep_start_position=81c4043d-31d0-23e5-9980-0b30d4de12fs:34911380: 1 (Operation not permitted)
[ERROR] WSREP: Command did not run: wsrep_sst_mariabackup --role 'donor' --address '172.57.12.14:4444/xtrabackup_sst//1' --socket '/dbdata/tmp/mysqld.sock' --datadir '/dbdata/data/' --gtid '81c4043d-31d0-23e5-9980-0b30d4de12fs:44178191' --gtid-domain-id '69' --binlog '/dbdata/binlog/node-bin' --bypass --mysqld-args --basedir=/dbdata/exe/mariadb_bin --datadir=/dbdata/data --plugin-dir=/dbdata/exe/mariadb_bin/lib/plugin --user=mysql --wsrep_on=on --wsrep_provider=/dbdata/exe/mariadb_bin/lib/libgalera_smm.so --log-error=/dbdata/log/mysqld_err.log --open-files-limit=512 --pid-file=/dbdata/data/node-220.vm.clhermes.local.pid --socket=/dbdata/tmp/mysqld.sock --port=3499 --wsrep_start_position=81c4043d-31d0-23e5-9980-0b30d4de12fs:34911380
[Warning] WSREP: 1.0 (node-220): State transfer to 2.0 (node-221) failed: -1 (Operation not permitted)_

Since the last time it worked, nothing has been changed (variables, versions).
I suspect it may be a connectivity problem, but I don't know how to test it.
Is it possible to obtain a trace with a more descriptive error?

Thanks in advance,
Jon.

Generated at Thu Feb 08 10:18:12 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.