Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24481

galera_3nodes.galera_vote_rejoin_mysqldump MTR failed: mysql_shutdown failed

Details

    Description

      galera_3nodes.galera_vote_rejoin_mysqldump failed on BB 10.5 CS and BB 10.6 CS: "mysql_shutdown failed".
      It seems to be a sporadic issue.

      [stdio.log|url2log]:

      10.5.9, 8677c14e65f436db00be5aedb1f644fcffc70f7e, kvm-zyp-opensuse150-amd64

      galera_3nodes.galera_vote_rejoin_mysqldump 'innodb' w1 [ fail ]
              Test ended at 2020-12-11 17:06:52
       
      CURRENT_TEST: galera_3nodes.galera_vote_rejoin_mysqldump
      mysqltest: In included file "./include/shutdown_mysqld.inc": 
      included from ./include/restart_mysqld.inc at line 10:
      included from /usr/share/mysql-test/suite/galera_3nodes/t/galera_vote_rejoin_mysqldump.test at line 78:
      At line 50: mysql_shutdown failed
       
      The result from queries just before the failure was:
      < snip >
      expect_Primary
      Primary
      connection node_2;
      SET SESSION wsrep_on=OFF;
      SELECT VARIABLE_VALUE AS expect_Disconnected FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_status';
      expect_Disconnected
      Disconnected
      SET SESSION wsrep_on=ON;
      SELECT VARIABLE_VALUE AS expect_Primary FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_status';
      expect_Primary
      Primary
      connection node_1;
      SHOW CREATE TABLE t1;
      Table	Create Table
      t1	CREATE TABLE `t1` (
        `f1` int(11) DEFAULT NULL
      ) ENGINE=InnoDB DEFAULT CHARSET=latin1
      connection node_2;
      SET SESSION wsrep_on=OFF;
      SET SESSION wsrep_on=ON;
       
      More results from queries before failure can be found in /dev/shm/var/1/log/galera_vote_rejoin_mysqldump.log
       
       - saving '/dev/shm/var/1/log/galera_3nodes.galera_vote_rejoin_mysqldump-innodb/' to '/dev/shm/var/log/galera_3nodes.galera_vote_rejoin_mysqldump-innodb/'
       
      Retrying test galera_3nodes.galera_vote_rejoin_mysqldump, attempt(2/3)...
       
      worker[1] > Restart  - not started
      worker[1] > Restart  - not started
      worker[1] > Restart  - not started
      ***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_vote_rejoin_mysqldump
       
      2020-12-11 17:06:48 2 [ERROR] WSREP: Vote 0 (success) on 3a0023f3-3bd3-11eb-827e-fb59270527e2:7 is inconsistent with group. Leaving cluster.
      
      

      Server logs.

      Attachments

        Issue Links

          Activity

            stepan.patryshev Stepan Patryshev (Inactive) created issue -
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            Field Original Value New Value
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            Attachment MDEV-24481_105_logs_201211.zip [ 55452 ]
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            Description galera_3nodes.galera_vote_rejoin_mysqldump failed on [BB 10.5 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-zyp-opensuse150-amd64/builds/4710] and [BB 10.6 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos8-amd64/builds/2023]: "mysql_shutdown failed".
            It seems to be a sporadic issue.

            [stdio.log|url2log]:
            {code:title=10.5.9, 8677c14e65f436db00be5aedb1f644fcffc70f7e, kvm-zyp-opensuse150-amd64}
            galera_3nodes.galera_vote_rejoin_mysqldump 'innodb' w1 [ fail ]
                    Test ended at 2020-12-11 17:06:52

            CURRENT_TEST: galera_3nodes.galera_vote_rejoin_mysqldump
            mysqltest: In included file "./include/shutdown_mysqld.inc":
            included from ./include/restart_mysqld.inc at line 10:
            included from /usr/share/mysql-test/suite/galera_3nodes/t/galera_vote_rejoin_mysqldump.test at line 78:
            At line 50: mysql_shutdown failed

            The result from queries just before the failure was:
            < snip >
            expect_Primary
            Primary
            connection node_2;
            SET SESSION wsrep_on=OFF;
            SELECT VARIABLE_VALUE AS expect_Disconnected FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_status';
            expect_Disconnected
            Disconnected
            SET SESSION wsrep_on=ON;
            SELECT VARIABLE_VALUE AS expect_Primary FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_status';
            expect_Primary
            Primary
            connection node_1;
            SHOW CREATE TABLE t1;
            Table Create Table
            t1 CREATE TABLE `t1` (
              `f1` int(11) DEFAULT NULL
            ) ENGINE=InnoDB DEFAULT CHARSET=latin1
            connection node_2;
            SET SESSION wsrep_on=OFF;
            SET SESSION wsrep_on=ON;

            More results from queries before failure can be found in /dev/shm/var/1/log/galera_vote_rejoin_mysqldump.log

             - saving '/dev/shm/var/1/log/galera_3nodes.galera_vote_rejoin_mysqldump-innodb/' to '/dev/shm/var/log/galera_3nodes.galera_vote_rejoin_mysqldump-innodb/'

            Retrying test galera_3nodes.galera_vote_rejoin_mysqldump, attempt(2/3)...

            worker[1] > Restart - not started
            worker[1] > Restart - not started
            worker[1] > Restart - not started
            ***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_vote_rejoin_mysqldump

            2020-12-11 17:06:48 2 [ERROR] WSREP: Vote 0 (success) on 3a0023f3-3bd3-11eb-827e-fb59270527e2:7 is inconsistent with group. Leaving cluster.

            {code}
            galera_3nodes.galera_vote_rejoin_mysqldump failed on [BB 10.5 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-zyp-opensuse150-amd64/builds/4710] and [BB 10.6 CS|http://buildbot.askmonty.org/buildbot/builders/kvm-rpm-centos8-amd64/builds/2023]: "mysql_shutdown failed".
            It seems to be a sporadic issue.

            [stdio.log|url2log]:
            {code:title=10.5.9, 8677c14e65f436db00be5aedb1f644fcffc70f7e, kvm-zyp-opensuse150-amd64}
            galera_3nodes.galera_vote_rejoin_mysqldump 'innodb' w1 [ fail ]
                    Test ended at 2020-12-11 17:06:52

            CURRENT_TEST: galera_3nodes.galera_vote_rejoin_mysqldump
            mysqltest: In included file "./include/shutdown_mysqld.inc":
            included from ./include/restart_mysqld.inc at line 10:
            included from /usr/share/mysql-test/suite/galera_3nodes/t/galera_vote_rejoin_mysqldump.test at line 78:
            At line 50: mysql_shutdown failed

            The result from queries just before the failure was:
            < snip >
            expect_Primary
            Primary
            connection node_2;
            SET SESSION wsrep_on=OFF;
            SELECT VARIABLE_VALUE AS expect_Disconnected FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_status';
            expect_Disconnected
            Disconnected
            SET SESSION wsrep_on=ON;
            SELECT VARIABLE_VALUE AS expect_Primary FROM INFORMATION_SCHEMA.GLOBAL_STATUS WHERE VARIABLE_NAME = 'wsrep_cluster_status';
            expect_Primary
            Primary
            connection node_1;
            SHOW CREATE TABLE t1;
            Table Create Table
            t1 CREATE TABLE `t1` (
              `f1` int(11) DEFAULT NULL
            ) ENGINE=InnoDB DEFAULT CHARSET=latin1
            connection node_2;
            SET SESSION wsrep_on=OFF;
            SET SESSION wsrep_on=ON;

            More results from queries before failure can be found in /dev/shm/var/1/log/galera_vote_rejoin_mysqldump.log

             - saving '/dev/shm/var/1/log/galera_3nodes.galera_vote_rejoin_mysqldump-innodb/' to '/dev/shm/var/log/galera_3nodes.galera_vote_rejoin_mysqldump-innodb/'

            Retrying test galera_3nodes.galera_vote_rejoin_mysqldump, attempt(2/3)...

            worker[1] > Restart - not started
            worker[1] > Restart - not started
            worker[1] > Restart - not started
            ***Warnings generated in error logs during shutdown after running tests: galera_3nodes.galera_vote_rejoin_mysqldump

            2020-12-11 17:06:48 2 [ERROR] WSREP: Vote 0 (success) on 3a0023f3-3bd3-11eb-827e-fb59270527e2:7 is inconsistent with group. Leaving cluster.

            {code}

            *[Server logs|^MDEV-24481_105_logs_201211.zip]*.
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            Assignee Stepan Patryshev [ stepan.patryshev ] Jan Lindström [ jplindst ]
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            stepan.patryshev Stepan Patryshev (Inactive) made changes -
            serg Sergei Golubchik made changes -
            Fix Version/s 10.6 [ 24028 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 117346 ] MariaDB v4 [ 142453 ]
            janlindstrom Jan Lindström made changes -
            Assignee Jan Lindström [ jplindst ] Julius Goryavsky [ sysprg ]
            sysprg Julius Goryavsky made changes -
            sysprg Julius Goryavsky made changes -
            janlindstrom Jan Lindström made changes -
            JIraAutomate JiraAutomate made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            janlindstrom Jan Lindström made changes -
            Assignee Julius Goryavsky [ sysprg ] Jan Lindström [ JIRAUSER53125 ]
            janlindstrom Jan Lindström made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            janlindstrom Jan Lindström added a comment - https://github.com/MariaDB/server/pull/2945
            janlindstrom Jan Lindström made changes -
            Assignee Jan Lindström [ JIRAUSER53125 ] Julius Goryavsky [ sysprg ]
            Status In Progress [ 3 ] In Review [ 10002 ]
            sysprg Julius Goryavsky added a comment - Thanks, change merged with head revision: https://github.com/MariaDB/server/commit/cfaab614ea9bf0df4b5398c5d51f18cda8a9a00f
            sysprg Julius Goryavsky made changes -
            Status In Review [ 10002 ] Stalled [ 10000 ]
            sysprg Julius Goryavsky added a comment - Fix merged with head revision: https://github.com/MariaDB/server/commit/cfaab614ea9bf0df4b5398c5d51f18cda8a9a00f
            sysprg Julius Goryavsky made changes -
            issue.field.resolutiondate 2023-12-21 09:41:37.0 2023-12-21 09:41:36.799
            sysprg Julius Goryavsky made changes -
            Fix Version/s 10.5.24 [ 29517 ]
            Fix Version/s 10.5 [ 23123 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            JIraAutomate JiraAutomate made changes -
            Fix Version/s 10.6.17 [ 29518 ]
            Fix Version/s 10.11.7 [ 29519 ]
            Fix Version/s 11.0.5 [ 29520 ]
            Fix Version/s 11.1.4 [ 29024 ]
            Fix Version/s 11.2.3 [ 29521 ]
            danblack Daniel Black made changes -

            People

              sysprg Julius Goryavsky
              stepan.patryshev Stepan Patryshev (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.