Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-14287

(draft) make sure no leftover rsync processes after mtr

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 10.0-galera
    • Fix Version/s: N/A
    • Component/s: Galera, Tests
    • Labels:
      None

      Description

      Sometimes rsync process may leftover (after ungraceful test exit?) :

      ps auxw | grep mysqld
      a         7853  0.3  0.0   9940  2988 pts/1    S    22:44   0:00 /bin/bash -ue /home/a/env1/m2-10.0-galera/bld/scripts/wsrep_sst_rsync --role joiner --address 127.0.0.1:16008 --datadir /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/mysqld.3/data/ --defaults-file /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/my.cnf --defaults-group-suffix .3 --parent 7791 --binlog master-bin
      a         7943  0.0  0.0  11152   952 pts/1    S    22:44   0:00 rsync --daemon --no-detach --port 16008 --config /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/mysqld.3/data//rsync_sst.conf
      a        16484  0.0  0.0  14240   932 pts/6    S+   22:48   0:00 grep --color=auto mysqld
      

      Further tests attempts may fail like:

       ./mtr --mem --enable-disabled --force  galera_as_slave_replication_bundle
      Logging: /home/a/env1/m2-10.0-galera/src/mysql-test/mysql-test-run.pl  --mem --enable-disabled --force galera_as_slave_replication_bundle
      vardir: /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var
      Checking leftover processes...
      Removing old var directory...
      Creating var directory '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var'...
       - symlinking 'var' to '/dev/shm/var_auto_VjD1'
      Checking supported features...
      MariaDB Version 10.0.33-MariaDB-wsrep
       - SSL connections supported
      Sphinx 'indexer' binary not found, sphinx suite will be skipped
      Collecting tests...
       - galera.galera_as_slave_replication_bundle wil be run although it's been disabled
         due to 'MDEV-13549 Galera test failures'
      Installing system database...
       
      ==============================================================================
       
      TEST                                      RESULT   TIME (ms) or COMMENT
      --------------------------------------------------------------------------
       
      worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
      worker[1] mysql-test-run: WARNING: Process [mysqld.3 - pid: 7511, winpid: 7511, exit: 256] died after mysql-test-run waited 6.4 seconds for /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/run/mysqld.3.pid to be created.
      galera.galera_as_slave_replication_bundle 'innodb_plugin' [ fail ]
              Test ended at 2017-11-04 22:44:43
       
      CURRENT_TEST: galera.galera_as_slave_replication_bundle
       
       
      Failed to start mysqld.3
      mysqltest failed but provided no output
       
       
       - saving '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/log/galera.galera_as_slave_replication_bundle-innodb_plugin/' to '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/log/galera.galera_as_slave_replication_bundle-innodb_plugin/'
      ***Warnings generated in error logs during shutdown after running tests: galera.galera_as_slave_replication_bundle
       
      171104 22:44:38 [ERROR] WSREP: Failed to read from: wsrep_sst_rsync --role 'donor' --address '127.0.0.1:16008/rsync_sst' --socket '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/tmp/mysqld.2.sock' --datadir '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/mysqld.2/data/' --defaults-file '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/my.cnf' --defaults-group-suffix '.2'  --binlog 'master-bin' --gtid '5a6903d9-c1a9-11e7-8d6a-6f0a27b55e8c:0'
      171104 22:44:38 [ERROR] WSREP: Process completed with error: wsrep_sst_rsync --role 'donor' --address '127.0.0.1:16008/rsync_sst' --socket '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/tmp/mysqld.2.sock' --datadir '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/mysqld.2/data/' --defaults-file '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/my.cnf' --defaults-group-suffix '.2'  --binlog 'master-bin' --gtid '5a6903d9-c1a9-11e7-8d6a-6f0a27b55e8c:0': 1 (Operation not permitted)
      171104 22:44:38 [ERROR] WSREP: Command did not run: wsrep_sst_rsync --role 'donor' --address '127.0.0.1:16008/rsync_sst' --socket '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/tmp/mysqld.2.sock' --datadir '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/mysqld.2/data/' --defaults-file '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/my.cnf' --defaults-group-suffix '.2'  --binlog 'master-bin' --gtid '5a6903d9-c1a9-11e7-8d6a-6f0a27b55e8c:0'
      171104 22:44:38 [Warning] WSREP: 0.0 (UBINTI): State transfer to 1.0 (UBINTI) failed: -1 (Operation not permitted)
      171104 22:44:38 [Warning] WSREP: 0.0 (UBINTI): State transfer to 1.0 (UBINTI) failed: -1 (Operation not permitted)
      171104 22:44:38 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():736: Will never receive state. Need to abort.
       
      worker[1] mysql-test-run: WARNING: Waited 60 seconds for /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/run/mysqld.3.pid to be created, still waiting for 120 seconds...
      worker[1] mysql-test-run: WARNING: Waited 120 seconds for /home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/run/mysqld.3.pid to be created, still waiting for 60 seconds...
      worker[1] mysql-test-run: WARNING: Timeout after mysql-test-run waited 180 seconds for the process [mysqld.3 - pid: 7790, winpid: 7790] to create a pid file.
      galera.galera_as_slave_replication_bundle 'xtradb' [ fail ]
              Test ended at 2017-11-04 22:47:44
       
      CURRENT_TEST: galera.galera_as_slave_replication_bundle
      CURRENT_TEST: galera.galera_as_slave_replication_bundle
       
       
      Failed to start mysqld.3
       
       
       - saving '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/log/galera.galera_as_slave_replication_bundle-xtradb/' to '/home/a/env1/_depot/m-branch/m2-10.0-galera-bld/mysql-test/var/log/galera.galera_as_slave_replication_bundle-xtradb/'
      ***Warnings generated in error logs during shutdown after running tests: galera.galera_as_slave_replication_bundle
       
      WSREP_SST: [ERROR] Parent mysqld process (PID:7512) terminated unexpectedly. (20171104 22:44:44.074)
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jplindst Jan Lindström
              Reporter:
              anikitin Andrii Nikitin (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: