[MDEV-24890] galera_3nodes.galera_ipv6_rsync MTR failed: rsync daemon port '16031' has been taken - Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: 10.3.28, 10.6.0
Fix Version/s: 10.5.10, 10.6.1, 10.2.39, 10.3.30, 10.4.20
Component/s: Galera, Tests
Labels:
None
Environment:
kvm-deb-bionic-amd64

Description

galera_3nodes.galera_ipv6_rsync failed on BB 10.3 CS and BB 10.6 CS: "rsync daemon port '16031' has been taken".
It seems to be a sporadic issue.

stdio.log:

10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ]
Test ended at 2021-01-12 03:59:00

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync


Failed to start mysqld.3
mysqltest failed but provided no output


- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...

worker[2] > Restart - not started
worker[2] > Restart - not started
worker[2] > Restart - not started
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128)
2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:58:46 2 [ERROR] Aborting
Warning: Memory not freed: 1381120

galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ]
Test ended at 2021-01-12 03:59:32

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync


Failed to start mysqld.2
mysqltest failed but provided no output


- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries!

worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

2021-01-12 3:59:31 0 [ERROR] Aborting
WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289)
2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:59:02 2 [ERROR] Aborting
Warning: Memory not freed: 1381120

Server logs.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

210112_galera_ipv6_rsync_103_CS_logs.zip
327 kB
2021-02-16 16:41
210112_stdio_103_CS.html
84 kB
2021-02-16 16:41

Issue Links

duplicates

MDEV-23580 galera_3nodes.galera_ipv6_rsync_section MTR failed: WSREP_SST: [ERROR] rsync daemon port '16008' has been taken

Closed

Activity

Ascending order - Click to sort in descending order

Stepan Patryshev (Inactive) created issue - 2021-02-16 16:09

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:10

Field	Original Value	New Value
Description	galera_3nodes.galera_ipv6_rsync failed on [BB 10.2 CS\|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/] and [BB 10.6 CS\|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/]: "rsync daemon port '16031' has been taken". It seems to be a sporadic issue. [stdio.log\|^210112_stdio_102_CS.html]: {code:title=10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64} galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ] Test ended at 2021-01-12 03:59:00 CURRENT_TEST: galera_3nodes.galera_ipv6_rsync Failed to start mysqld.3 mysqltest failed but provided no output - skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/' Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)... worker[2] > Restart - not started worker[2] > Restart - not started worker[2] > Restart - not started *Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128) 2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300 2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy) 2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2021-01-12 3:58:46 2 [ERROR] Aborting Warning: Memory not freed: 1381120 galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117 worker[1] > Restart - using different config file worker[1] > Restart - using different config file worker[1] > Restart - using different config file worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created. galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ] Test ended at 2021-01-12 03:59:32 CURRENT_TEST: galera_3nodes.galera_ipv6_rsync Failed to start mysqld.2 mysqltest failed but provided no output - skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/' Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries! worker[2] > Restart - using different config file worker[2] > Restart - using different config file worker[2] > Restart - using different config file Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync 2021-01-12 3:59:31 0 [ERROR] Aborting WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289) 2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300 2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy) 2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2021-01-12 3:59:02 2 [ERROR] Aborting Warning: Memory not freed: 1381120 {code} [Server logs\|^210112_galera_ipv6_rsync_102_CS_logs.zip]*.	galera_3nodes.galera_ipv6_rsync failed on [BB 10.3 CS\|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/] and [BB 10.6 CS\|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/]: "rsync daemon port '16031' has been taken". It seems to be a sporadic issue. [stdio.log\|^210112_stdio_102_CS.html]: {code:title=10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64} galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ] Test ended at 2021-01-12 03:59:00 CURRENT_TEST: galera_3nodes.galera_ipv6_rsync Failed to start mysqld.3 mysqltest failed but provided no output - skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/' Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)... worker[2] > Restart - not started worker[2] > Restart - not started worker[2] > Restart - not started *Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128) 2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300 2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy) 2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2021-01-12 3:58:46 2 [ERROR] Aborting Warning: Memory not freed: 1381120 galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117 worker[1] > Restart - using different config file worker[1] > Restart - using different config file worker[1] > Restart - using different config file worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created. galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ] Test ended at 2021-01-12 03:59:32 CURRENT_TEST: galera_3nodes.galera_ipv6_rsync Failed to start mysqld.2 mysqltest failed but provided no output - skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/' Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries! worker[2] > Restart - using different config file worker[2] > Restart - using different config file worker[2] > Restart - using different config file Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync 2021-01-12 3:59:31 0 [ERROR] Aborting WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289) 2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300 2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy) 2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable. 2021-01-12 3:59:02 2 [ERROR] Aborting Warning: Memory not freed: 1381120 {code} [Server logs\|^210112_galera_ipv6_rsync_102_CS_logs.zip]*.

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:10

Description

galera_3nodes.galera_ipv6_rsync failed on [BB 10.3 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/] and [BB 10.6 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/]: "rsync daemon port '16031' has been taken".
It seems to be a sporadic issue.

[stdio.log|^210112_stdio_102_CS.html]:
{code:title=10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64}
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ]
Test ended at 2021-01-12 03:59:00

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.3
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...

worker[2] > Restart - not started
worker[2] > Restart - not started
worker[2] > Restart - not started
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128)
2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:58:46 2 [ERROR] Aborting
Warning: Memory not freed: 1381120

galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ]
Test ended at 2021-01-12 03:59:32

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.2
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries!

worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

2021-01-12 3:59:31 0 [ERROR] Aborting
WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289)
2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:59:02 2 [ERROR] Aborting
Warning: Memory not freed: 1381120
{code}

*[Server logs|^210112_galera_ipv6_rsync_102_CS_logs.zip]*.

galera_3nodes.galera_ipv6_rsync failed on [BB 10.3 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/] and [BB 10.6 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos8-amd64/builds/2123]: "rsync daemon port '16031' has been taken".
It seems to be a sporadic issue.

[stdio.log|^210112_stdio_102_CS.html]:
{code:title=10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64}
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ]
Test ended at 2021-01-12 03:59:00

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.3
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...

worker[2] > Restart - not started
worker[2] > Restart - not started
worker[2] > Restart - not started
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128)
2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:58:46 2 [ERROR] Aborting
Warning: Memory not freed: 1381120

galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ]
Test ended at 2021-01-12 03:59:32

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.2
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries!

worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

2021-01-12 3:59:31 0 [ERROR] Aborting
WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289)
2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:59:02 2 [ERROR] Aborting
Warning: Memory not freed: 1381120
{code}

*[Server logs|^210112_galera_ipv6_rsync_102_CS_logs.zip]*.

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:11

Assignee

Stepan Patryshev [ stepan.patryshev ]

Jan Lindström [ jplindst ]

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:25

Link

This issue blocks ~~MDEV-19937~~ [ ~~MDEV-19937~~ ]

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:40

Attachment

210112_galera_ipv6_rsync_102_CS_logs.zip [ 56224 ]

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:40

Attachment

210112_stdio_102_CS.html [ 56223 ]

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:41

Attachment		210112_galera_ipv6_rsync_103_CS_logs.zip [ 56227 ]
Attachment		210112_stdio_103_CS.html [ 56228 ]

Stepan Patryshev (Inactive) made changes - 2021-02-16 16:41

Description

galera_3nodes.galera_ipv6_rsync failed on [BB 10.3 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/] and [BB 10.6 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos8-amd64/builds/2123]: "rsync daemon port '16031' has been taken".
It seems to be a sporadic issue.

[stdio.log|^210112_stdio_102_CS.html]:
{code:title=10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64}
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ]
Test ended at 2021-01-12 03:59:00

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.3
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...

worker[2] > Restart - not started
worker[2] > Restart - not started
worker[2] > Restart - not started
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128)
2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:58:46 2 [ERROR] Aborting
Warning: Memory not freed: 1381120

galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ]
Test ended at 2021-01-12 03:59:32

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.2
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries!

worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

2021-01-12 3:59:31 0 [ERROR] Aborting
WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289)
2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:59:02 2 [ERROR] Aborting
Warning: Memory not freed: 1381120
{code}

*[Server logs|^210112_galera_ipv6_rsync_102_CS_logs.zip]*.

galera_3nodes.galera_ipv6_rsync failed on [BB 10.3 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-deb-bionic-amd64/builds/6586/] and [BB 10.6 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos8-amd64/builds/2123]: "rsync daemon port '16031' has been taken".
It seems to be a sporadic issue.

[stdio.log|^210112_stdio_103_CS.html]:
{code:title=10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64}
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ]
Test ended at 2021-01-12 03:59:00

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.3
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...

worker[2] > Restart - not started
worker[2] > Restart - not started
worker[2] > Restart - not started
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128)
2021-01-12 3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:58:46 2 [ERROR] Aborting
Warning: Memory not freed: 1381120

galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ] 14117
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[1] > Restart - using different config file
worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.
galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ]
Test ended at 2021-01-12 03:59:32

CURRENT_TEST: galera_3nodes.galera_ipv6_rsync

Failed to start mysqld.2
mysqltest failed but provided no output

- skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'

Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries!

worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
worker[2] > Restart - using different config file
***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync

2021-01-12 3:59:31 0 [ERROR] Aborting
WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289)
2021-01-12 3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
2021-01-12 3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
2021-01-12 3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
2021-01-12 3:59:02 2 [ERROR] Aborting
Warning: Memory not freed: 1381120
{code}

*[Server logs|^210112_galera_ipv6_rsync_103_CS_logs.zip]*.

Sergei Golubchik made changes - 2021-02-22 08:02

Fix Version/s		10.3 [ 22126 ]
Fix Version/s		10.4 [ 22408 ]
Fix Version/s		10.5 [ 23123 ]
Fix Version/s	10.6 [ 24028 ]
Fix Version/s	10.3.27 [ 25025 ]

Julius Goryavsky made changes - 2021-05-22 21:52

Assignee

Jan Lindström [ jplindst ]

Julius Goryavsky [ sysprg ]

Julius Goryavsky made changes - 2021-05-22 21:53

Link

This issue duplicates ~~MDEV-23580~~ [ ~~MDEV-23580~~ ]

Julius Goryavsky made changes - 2021-05-22 21:53

Status

Open [ 1 ]

Confirmed [ 10101 ]

Julius Goryavsky made changes - 2021-05-22 21:53

Status

Confirmed [ 10101 ]

In Progress [ 3 ]

Julius Goryavsky added a comment - 2021-05-22 21:56

Fixed as part of https://jira.mariadb.org/browse/MDEV-23580

Julius Goryavsky added a comment - 2021-05-22 21:56 Fixed as part of https://jira.mariadb.org/browse/MDEV-23580

Julius Goryavsky made changes - 2021-05-22 21:56

issue.field.resolutiondate

2021-05-22 21:56:00.0

2021-05-22 21:56:00.057

Julius Goryavsky made changes - 2021-05-22 21:56

Fix Version/s		10.2.39 [ 25731 ]
Fix Version/s		10.3.30 [ 25732 ]
Fix Version/s		10.4.20 [ 25733 ]
Fix Version/s		10.6.1 [ 24437 ]
Fix Version/s		10.5.10 [ 25204 ]
Fix Version/s	10.3 [ 22126 ]
Fix Version/s	10.4 [ 22408 ]
Fix Version/s	10.5 [ 23123 ]
Resolution		Fixed [ 1 ]
Status	In Progress [ 3 ]	Closed [ 6 ]

Julien Fritsch made changes - 2021-09-29 13:10

Link

This issue blocks ~~MDEV-19937~~ [ ~~MDEV-19937~~ ]

Sergei Golubchik made changes - 2021-12-06 21:53

Workflow

MariaDB v3 [ 119210 ]

MariaDB v4 [ 158913 ]

People

Assignee:: Julius Goryavsky

Reporter:: Stepan Patryshev (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 2021-02-16 16:09

Updated:: 2021-09-29 13:10

Resolved:: 2021-05-22 21:56

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.

MariaDB Server

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates

Git Integration