Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Done
    • 10.1(EOL), 10.2(EOL)
    • N/A
    • Galera SST, Tests
    • None
    • 10.2.11, 10.2.14

    Description

      Current mtr sst tests are complicated with heavy dependencies and often skipped.
      So we need simple quick smoke sst tests with default configuration.

      Attachments

        Issue Links

          Activity

            anikitin Andrii Nikitin (Inactive) added a comment - - edited

            suggested test for 10.2

            sst_smoke.test

            # let cluster initialize
            --sleep 5
             
            --source include/galera_cluster.inc
            --connection node_1
            show variables like 'wsrep_sst_method';
            CREATE TABLE t1 ENGINE=InnoDB select 5;
             
            --connection node_2
            show variables like 'wsrep_sst_method';
            show status like 'wsrep_cluster_size';
            SELECT * FROM t1;
            DROP TABLE t1;
            

            sst_smoke.combinations

            [mariabackup]
            wsrep-sst-method=mariabackup
             
            [xtrabackup]
            wsrep-sst-method=xtrabackup
             
            [xtrabackup-v2]
            wsrep-sst-method=xtrabackup-v2
             
            [rsync]
            wsrep-sst-method=rsync
             
            [mysqldump]
            wsrep-sst-method=mysqldump
            bind-address=0.0.0.0
            # good for smoke test to not deal with ACL
            skip-grant-tables
            wsrep_sst_receive_address=AUTO
            # for some reasons it is actually 15, so 'show variables' hangs indefinitely
            wsrep_sync_wait=0
            
            

            With following patch:

            --- a/mysql-test/suite/galera/disabled.def
            +++ b/mysql-test/suite/galera/disabled.def
            @@ -56,3 +56,4 @@ MW-284 : MDEV-13549 Galera test failures 10.1
             galera_as_slave : MDEV-13549 Galera test failures 10.1
             galera_var_innodb_disallow_writes : MDEV-10949
             galera_kill_applier : race condition at the start of the test
            +sst_smoke : run only with --nocheck-testcase , MDEV-14305
            diff --git a/mysql-test/suite/galera/suite.pm b/mysql-test/suite/galera/suite.pm
            index 361743f1243..3a0ba42f261 100644
            --- a/mysql-test/suite/galera/suite.pm
            +++ b/mysql-test/suite/galera/suite.pm
            @@ -25,6 +25,9 @@ return "No scritps" unless $cpath;
             my ($epath) = grep { -f "$_/my_print_defaults"; } "$::bindir/extra", $::path_client_bindir;
             return "No my_print_defaults" unless $epath;
             
            +# let sst find mariabackup and mbstream
            +my ($bpath) = grep { -f "$_/mariabackup"; } "$::bindir/extra/mariabackup", $::path_client_bindir;
            +
             push @::global_suppressions,
               (
                  qr(WSREP: wsrep_sst_receive_address is set to '127.0.0.1),
            @@ -81,6 +84,8 @@ push @::global_suppressions,
             $ENV{PATH}="$epath:$ENV{PATH}";
             $ENV{PATH}="$spath:$ENV{PATH}" unless $epath eq $spath;
             $ENV{PATH}="$cpath:$ENV{PATH}" unless $cpath eq $spath;
            +# let mariabackup test just fail if no mariabackup is installed instead of creating dependency on backup package
            +$ENV{PATH}="$bpath:$ENV{PATH}" unless $cpath eq $bpath or $epath eq $spath or (!$bpath);
             
             bless { };
            

            While it is disabled (because of mtr limitations) it is still very useful to run it manually with --enable-disabled --nocheck-testcase

            It may need fixes for MDEV-14030 to run mariabackup sst properly

            anikitin Andrii Nikitin (Inactive) added a comment - - edited suggested test for 10.2 sst_smoke.test # let cluster initialize --sleep 5   --source include/galera_cluster.inc --connection node_1 show variables like 'wsrep_sst_method'; CREATE TABLE t1 ENGINE=InnoDB select 5;   --connection node_2 show variables like 'wsrep_sst_method'; show status like 'wsrep_cluster_size'; SELECT * FROM t1; DROP TABLE t1; sst_smoke.combinations [mariabackup] wsrep-sst-method=mariabackup   [xtrabackup] wsrep-sst-method=xtrabackup   [xtrabackup-v2] wsrep-sst-method=xtrabackup-v2   [rsync] wsrep-sst-method=rsync   [mysqldump] wsrep-sst-method=mysqldump bind-address=0.0.0.0 # good for smoke test to not deal with ACL skip-grant-tables wsrep_sst_receive_address=AUTO # for some reasons it is actually 15, so 'show variables' hangs indefinitely wsrep_sync_wait=0 With following patch: --- a/mysql-test/suite/galera/disabled.def +++ b/mysql-test/suite/galera/disabled.def @@ -56,3 +56,4 @@ MW-284 : MDEV-13549 Galera test failures 10.1 galera_as_slave : MDEV-13549 Galera test failures 10.1 galera_var_innodb_disallow_writes : MDEV-10949 galera_kill_applier : race condition at the start of the test +sst_smoke : run only with --nocheck-testcase , MDEV-14305 diff --git a/mysql-test/suite/galera/suite.pm b/mysql-test/suite/galera/suite.pm index 361743f1243..3a0ba42f261 100644 --- a/mysql-test/suite/galera/suite.pm +++ b/mysql-test/suite/galera/suite.pm @@ -25,6 +25,9 @@ return "No scritps" unless $cpath ; my ( $epath ) = grep { -f "$_/my_print_defaults" ; } "$::bindir/extra" , $::path_client_bindir; return "No my_print_defaults" unless $epath ; + # let sst find mariabackup and mbstream + my ( $bpath ) = grep { -f "$_/mariabackup" ; } "$::bindir/extra/mariabackup" , $::path_client_bindir; + push @::global_suppressions, ( qr(WSREP: wsrep_sst_receive_address is set to '127.0.0.1), @@ -81,6 +84,8 @@ push @::global_suppressions, $ENV {PATH}= "$epath:$ENV{PATH}" ; $ENV {PATH}= "$spath:$ENV{PATH}" unless $epath eq $spath ; $ENV {PATH}= "$cpath:$ENV{PATH}" unless $cpath eq $spath ; + # let mariabackup test just fail if no mariabackup is installed instead of creating dependency on backup package + $ENV {PATH}= "$bpath:$ENV{PATH}" unless $cpath eq $bpath or $epath eq $spath or (! $bpath ); bless { }; While it is disabled (because of mtr limitations) it is still very useful to run it manually with --enable-disabled --nocheck-testcase It may need fixes for MDEV-14030 to run mariabackup sst properly
            1. normally tests with failing prerequisites should be skipped, not failed. you can skip mariabackup combination if backup package is not installed.
            2. why is it it disabled, because of what mtr limitations?
            serg Sergei Golubchik added a comment - normally tests with failing prerequisites should be skipped, not failed. you can skip mariabackup combination if backup package is not installed. why is it it disabled, because of what mtr limitations?
            anikitin Andrii Nikitin (Inactive) added a comment - - edited

            1. The Idea for this mtr test was to be able to "just start Cluster with corresponding sst method". This is very useful at least for manual testing, because it is really time-consuming work to set up Cluster with other means.
            Since sst is performed before first line of test code is executed - there is no way to skip test using mysqltest command. (Btw current galera*xtrabackup* tests have the same problem - they will fail if all other prerequisites are met but no xtrabackup is installed)

            I guess current mtr approach for such cases is to startup cluster with unrelated configuration, then check prerequisites and change configuration to what needs to be tested. It is probably OK, but adds a lot of unrelated logs and may fail somewhere, which is not related to what actually needs to be tested. (And the main purpose of unit tests is to show where exactly failure is = the more unrelated code exists - the bigger chance that failure will happen in unrelated functionality and the more time is needed to troubleshoot failures).

            2. mtr by default does db object verification, which interfere with cluster startup when sst´mysqldump is used (maybe this could be fixed as part of MDEV-13329 or MDEV-14069, but mysqldump does recreate all objects, so mtr check may actually show difference anyway). So, (currently) mysqldump sst method works only when test is started with --nocheck-testcase .

            So my idea is to have this test just for friendly manual testing for now and later enable it if/when mtr is able to have configured --nocheck-testcase for individual tests and check prerequisites before starting servers.

            anikitin Andrii Nikitin (Inactive) added a comment - - edited 1. The Idea for this mtr test was to be able to "just start Cluster with corresponding sst method". This is very useful at least for manual testing, because it is really time-consuming work to set up Cluster with other means. Since sst is performed before first line of test code is executed - there is no way to skip test using mysqltest command. (Btw current galera*xtrabackup* tests have the same problem - they will fail if all other prerequisites are met but no xtrabackup is installed) I guess current mtr approach for such cases is to startup cluster with unrelated configuration, then check prerequisites and change configuration to what needs to be tested. It is probably OK, but adds a lot of unrelated logs and may fail somewhere, which is not related to what actually needs to be tested. (And the main purpose of unit tests is to show where exactly failure is = the more unrelated code exists - the bigger chance that failure will happen in unrelated functionality and the more time is needed to troubleshoot failures). 2. mtr by default does db object verification, which interfere with cluster startup when sst´mysqldump is used (maybe this could be fixed as part of MDEV-13329 or MDEV-14069 , but mysqldump does recreate all objects, so mtr check may actually show difference anyway). So, (currently) mysqldump sst method works only when test is started with --nocheck-testcase . So my idea is to have this test just for friendly manual testing for now and later enable it if/when mtr is able to have configured --nocheck-testcase for individual tests and check prerequisites before starting servers.

            1. you can skip from suite.pm, if needed. we do it for, say, ipv6 tests where mysqld won't even start if ipv6 is not available.

            serg Sergei Golubchik added a comment - 1. you can skip from suite.pm, if needed. we do it for, say, ipv6 tests where mysqld won't even start if ipv6 is not available.
            anikitin Andrii Nikitin (Inactive) added a comment - - edited

            Regarding 2. - I see following error before test starts , so galera will not allow any query from mtr check until sst is completed.

            query 'call mtr.check_testcase()' failed: 1047: WSREP has not yet prepared node for application use
            

            Maybe we can educate mtr later to re-try check_testcase for some time if error 1047 is received. Or maybe setting `session.WSREP_ON=0` from ./mtr will be sufficient?

            anikitin Andrii Nikitin (Inactive) added a comment - - edited Regarding 2. - I see following error before test starts , so galera will not allow any query from mtr check until sst is completed. query 'call mtr.check_testcase()' failed: 1047 : WSREP has not yet prepared node for application use Maybe we can educate mtr later to re-try check_testcase for some time if error 1047 is received. Or maybe setting `session.WSREP_ON=0` from ./mtr will be sufficient?

            already done in MDEV-15409

            serg Sergei Golubchik added a comment - already done in MDEV-15409

            People

              serg Sergei Golubchik
              anikitin Andrii Nikitin (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.