[MDEV-23581] galera_3nodes.galera_ipv6_rsync MTR failed: WSREP_SST: [ERROR] rsync daemon port '16008' has been taken Created: 2020-08-25  Updated: 2020-12-07  Resolved: 2020-10-21

Status: Closed
Project: MariaDB Server
Component/s: Galera, Tests
Affects Version/s: 10.2.34, 10.3.25, 10.4.15, 10.5.6
Fix Version/s: 10.2.35, 10.3.26, 10.4.16, 10.5.7, 10.6.0

Type: Bug Priority: Major
Reporter: Stepan Patryshev (Inactive) Assignee: Julius Goryavsky
Resolution: Fixed Votes: 0
Labels: None
Environment:

kvm-zyp-sles123-amd64


Issue Links:
Duplicate
duplicates MDEV-21770 `galera_3nodes.galera_ipv6_mariabacku... Closed

 Description   

galera_3nodes.galera_ipv6_rsync failed on BB 10.2, BB 10.3, BB 10.4, and BB 10.5: "WSREP_SST: [ERROR] rsync daemon port '16008' has been taken".

stdio.log:

10.2.34 0be70a1b773ce66ef1803fcce19522fd9c60c07d

 
galera_3nodes.galera_ipv6_rsync 'innodb' w1 [ fail ]
 
        Test ended at 2020-08-25 13:32:41
 
 
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup_section
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mysqldump
 
CURRENT_TEST: galera_3nodes.galera_ipv6_mariabackup_section
 
CURRENT_TEST: galera_3nodes.galera_ipv6_rsync
 
 
 
 
 
Failed to start mysqld.2
 
 
 
 
 
 - skipping '/dev/shm/var/1/log/galera_3nodes.galera_ipv6_rsync-innodb/'
 
 
 
Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...
 
 
 
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync
 
 
 
WSREP_SST: [ERROR] rsync daemon port '16008' has been taken (20200825 13:32:32.184)
 
2020-08-25 13:32:32 139949739325184 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16008' --datadir '/dev/shm/var/1/mysqld.2/data/' --defaults-file '/dev/shm/var/1/my.cnf' --defaults-group-suffix '.2' --parent '29037' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.2 --defaults-file=/dev/shm/var/1/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --bind-address=:: --core-file --loose-debug-sync-timeout=300
 
2020-08-25 13:32:32 139949739325184 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16008' --datadir '/dev/shm/var/1/mysqld.2/data/' --defaults-file '/dev/shm/var/1/my.cnf' --defaults-group-suffix '.2' --parent '29037' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.2 --defaults-file=/dev/shm/var/1/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --bind-address=:: --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
 
2020-08-25 13:32:32 139949973714688 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
 
2020-08-25 13:32:32 139949973714688 [ERROR] Aborting
 
Warning: Memory not freed: 1427624
 
WSREP_SST: [ERROR] rsync daemon port '16011' has been taken (20200825 13:32:32.183)
 
2020-08-25 13:32:32 140613823465216 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16011' --datadir '/dev/shm/var/1/mysqld.3/data/' --defaults-file '/dev/shm/var/1/my.cnf' --defaults-group-suffix '.3' --parent '29039' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/1/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --bind-address=:: --core-file --loose-debug-sync-timeout=300
 
2020-08-25 13:32:32 140613823465216 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16011' --datadir '/dev/shm/var/1/mysqld.3/data/' --defaults-file '/dev/shm/var/1/my.cnf' --defaults-group-suffix '.3' --parent '29039' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/1/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --bind-address=:: --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
 
2020-08-25 13:32:32 140614080018176 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
 
2020-08-25 13:32:32 140614080018176 [ERROR] Aborting
 
Warning: Memory not freed: 1427624
 
 
 
wsrep.wsrep-recover-v25 'binlogoff,innodb' w2 [ skipped ]  debug_sync is not available
 
worker[2] > Restart [mysqld.1 - pid: 29555, winpid: 29555] - using different config file
 
wsrep.wsrep_rpl 'innodb,row'             w2 [ pass ]   1046
 
worker[2] > Restart [mysqld.1 - pid: 29602, winpid: 29602] - running with different options '--binlog-format=statement --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --log-bin=master-bin' != '--binlog-format=row --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --log-bin=master-bin'
 
worker[2] > Restart [mysqld.2 - pid: 29604, winpid: 29604] - running with different options '--binlog-format=statement --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --log-bin=slave-bin' != '--binlog-format=row --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --log-bin=slave-bin'



 Comments   
Comment by Julius Goryavsky [ 2020-10-21 ]

This is a duplicate of MDEV-21770, corrected here: https://github.com/MariaDB/server/commit/ac3c19de6967abbaaa661f72526b233d11bd6868, tests re-enabled here: https://github.com/MariaDB/server/commit/967f11a68af9c8855621b00d839c91d8895ccbf4

Generated at Thu Feb 08 09:23:29 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.