[MDEV-23594] galera_3nodes.galera_toi_vote MTR failed: WSREP_SST: [ERROR] rsync daemon port '16008' has been taken Created: 2020-08-26  Updated: 2020-11-11  Resolved: 2020-10-21

Status: Closed
Project: MariaDB Server
Component/s: Galera, Tests
Affects Version/s: 10.5.6
Fix Version/s: 10.2.37, 10.3.28, 10.4.18, 10.5.9, 10.6.0

Type: Bug Priority: Major
Reporter: Stepan Patryshev (Inactive) Assignee: Julius Goryavsky
Resolution: Fixed Votes: 0
Labels: None
Environment:

kvm-zyp-sles123-amd64

Issue Links:
Duplicate
duplicates MDEV-21770 `galera_3nodes.galera_ipv6_mariabacku... Closed

 Description   

galera_3nodes.galera_toi_vote failed on BB 10.5: "WSREP_SST: [ERROR] rsync daemon port '16008' has been taken".

stdio.log:

10.5.6 8f8f2aea93835899345454f87768fd649749e29c

 
galera_3nodes.galera_toi_vote 'innodb'   w2 [ fail ]
 
        Test ended at 2020-08-26 07:19:23
 
 
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup_section
 
CURRENT_TEST: galera_3nodes.galera_ipv6_rsync
 
CURRENT_TEST: galera_3nodes.galera_ipv6_rsync_section
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup_section
 
CURRENT_TEST: galera_3nodes.galera_ist_gcache_rollover
 
CURRENT_TEST: galera_3nodes.galera_ipv6_rsync
 
CURRENT_TEST: galera_3nodes.galera_toi_vote
 
 
 
 
 
Failed to start mysqld.2
 
 
 
 
 
worker[2] > Restart  - using different config file
 
worker[2] > Restart  - using different config file
 
worker[2] > Restart  - using different config file
 
 - skipping '/dev/shm/var/2/log/galera_3nodes.galera_toi_vote-innodb/'
 
 
 
Retrying test galera_3nodes.galera_toi_vote, attempt(2/3)...
 
 
 
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_toi_vote
 
 
 
WSREP_SST: [ERROR] rsync daemon port '16008' has been taken (20200826 07:19:22.430)
 
2020-08-26  7:19:22 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '127.0.0.1:16008' --datadir '/dev/shm/var/2/mysqld.2/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.2' --parent '2366' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.2 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
 
2020-08-26  7:19:22 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '127.0.0.1:16008' --datadir '/dev/shm/var/2/mysqld.2/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.2' --parent '2366' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.2 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
 
2020-08-26  7:19:22 1 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
 
2020-08-26  7:19:22 1 [ERROR] WSREP: SST request callback failed. This is unrecoverable, restart required.
 
WSREP_SST: [ERROR] rsync daemon port '16011' has been taken (20200826 07:19:22.414)
 
2020-08-26  7:19:22 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '127.0.0.1:16011' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '2369' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
 
2020-08-26  7:19:22 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '127.0.0.1:16011' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '2369' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
 
2020-08-26  7:19:22 1 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
 
2020-08-26  7:19:22 1 [ERROR] WSREP: SST request callback failed. This is unrecoverable, restart required.
 
 
 
worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 2589, winpid: 2589, exit: 256] died after mysql-test-run waited 1.3 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.



 Comments   
Comment by Julius Goryavsky [ 2020-10-21 ]

This is a duplicate of MDEV-21770, corrected here: https://github.com/MariaDB/server/commit/ac3c19de6967abbaaa661f72526b233d11bd6868, tests re-enabled here: https://github.com/MariaDB/server/commit/967f11a68af9c8855621b00d839c91d8895ccbf4

Generated at Thu Feb 08 09:23:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.