Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24890

galera_3nodes.galera_ipv6_rsync MTR failed: rsync daemon port '16031' has been taken

    XMLWordPrintable

    Details

      Description

      galera_3nodes.galera_ipv6_rsync failed on BB 10.3 CS and BB 10.6 CS: "rsync daemon port '16031' has been taken".
      It seems to be a sporadic issue.

      stdio.log:

      10.3.28, 1be707286ef13041a67e6e6ca30125470245a898, kvm-deb-bionic-amd64

      galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ fail ]
              Test ended at 2021-01-12 03:59:00
       
      CURRENT_TEST: galera_3nodes.galera_ipv6_rsync
       
       
      Failed to start mysqld.3
      mysqltest failed but provided no output
       
       
       - skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'
       
      Retrying test galera_3nodes.galera_ipv6_rsync, attempt(2/3)...
       
      worker[2] > Restart  - not started
      worker[2] > Restart  - not started
      worker[2] > Restart  - not started
      ***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync
       
      WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:58:46.128)
      2021-01-12  3:58:46 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
      2021-01-12  3:58:46 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '30628' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
      2021-01-12  3:58:46 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
      2021-01-12  3:58:46 2 [ERROR] Aborting
      Warning: Memory not freed: 1381120
       
      galera_3nodes.galera_ipv6_rsync_section 'innodb' w1 [ pass ]  14117
      worker[1] > Restart  - using different config file
      worker[1] > Restart  - using different config file
      worker[1] > Restart  - using different config file
      worker[2] mysql-test-run: WARNING: Process [mysqld.2 - pid: 31409, winpid: 31409, exit: 256] died after mysql-test-run waited 31.4 seconds for /dev/shm/var/2/run/mysqld.2.pid to be created.
      galera_3nodes.galera_ipv6_rsync 'innodb' w2 [ retry-fail ]
              Test ended at 2021-01-12 03:59:32
       
      CURRENT_TEST: galera_3nodes.galera_ipv6_rsync
       
       
      Failed to start mysqld.2
      mysqltest failed but provided no output
       
       
       - skipping '/dev/shm/var/2/log/galera_3nodes.galera_ipv6_rsync-innodb/'
       
      Test galera_3nodes.galera_ipv6_rsync has failed 2 times, no more retries!
       
      worker[2] > Restart  - using different config file
      worker[2] > Restart  - using different config file
      worker[2] > Restart  - using different config file
      ***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_ipv6_rsync
       
      2021-01-12  3:59:31 0 [ERROR] Aborting
      WSREP_SST: [ERROR] rsync daemon port '16031' has been taken (20210112 03:59:02.289)
      2021-01-12  3:59:02 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300
      2021-01-12  3:59:02 0 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'joiner' --address '[::1]:16031' --datadir '/dev/shm/var/2/mysqld.3/data/' --defaults-file '/dev/shm/var/2/my.cnf' --defaults-group-suffix '.3' --parent '31412' --binlog-index 'mysqld-bin.index' --mysqld-args --defaults-group-suffix=.3 --defaults-file=/dev/shm/var/2/my.cnf --log-output=file --innodb --innodb-cmpmem --innodb-cmp-per-index --innodb-trx --innodb-locks --innodb-lock-waits --innodb-metrics --innodb-buffer-pool-stats --innodb-buffer-page --innodb-buffer-page-lru --innodb-sys-columns --innodb-sys-fields --innodb-sys-foreign --innodb-sys-foreign-cols --innodb-sys-indexes --innodb-sys-tables --innodb-sys-virtual --core-file --loose-debug-sync-timeout=300: 16 (Device or resource busy)
      2021-01-12  3:59:02 2 [ERROR] WSREP: Failed to prepare for 'rsync' SST. Unrecoverable.
      2021-01-12  3:59:02 2 [ERROR] Aborting
      Warning: Memory not freed: 1381120
      

      Server logs.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              sysprg Julius Goryavsky
              Reporter:
              stepan.patryshev Stepan Patryshev (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration