Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-7079

rpl.rpl_parallel_temptable fails sporadically in standard buildbot tests and persistently in valgrind builder

Details

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/2885/steps/test_7/logs/stdio

      CURRENT_TEST: rpl.rpl_parallel_temptable
      mysqltest: In included file "./include/sync_with_master_gtid.inc": 
      included from /mnt/buildbot/build/mariadb-10.0.15/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 205:
      At line 44: Failed to sync with master
       
      The result from queries just before the failure was:
      < snip >
       
      ALTER TABLE t1 ENGINE=InnoDB;
      CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY);
      INSERT INTO t2 VALUES (1);
      INSERT INTO t2 VALUES (2);
      SET gtid_domain_id= 2;
      CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY);
      INSERT INTO t3 VALUES (10);
      INSERT INTO t3 VALUES (20);
      INSERT INTO t1 SELECT a, 'server_1' FROM t2;
      INSERT INTO t1 SELECT a, 'default' FROM t3;
      INSERT INTO t1 SELECT a+2, '+server_1' FROM t2;
      FLUSH TABLES;
      SET SESSION debug_dbug="+d,crash_before_writing_xid";
      INSERT INTO t1 SELECT a+4, '++server_1' FROM t2;
      Got one of the listed errors
      INSERT INTO t1 VALUES (0, 1);
      include/save_master_gtid.inc
      include/start_slave.inc
      include/sync_with_master_gtid.inc
      Timeout in master_gtid_wait('1-1-31,0-1-4,2-1-17', 120), current slave GTID position is: 1-1-31,2-1-17,0-1-3.

      Retry passed.

      http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6310/steps/test/logs/stdio

      rpl.rpl_parallel_temptable 'innodb_plugin,stmt' w3 [ retry-fail ]  timeout after 9000 seconds
              Test ended at 2014-11-11 06:46:44
       
      Test case timeout after 9000 seconds
       
      == /dev/shm/var_auto_xTWj/3/log/rpl_parallel_temptable.log == 
      Slave_open_temp_tables	0
      *** Test that if master logged partial event group before crash, we finish that group correctly before executing format description event ***
      include/stop_slave.inc
      CALL mtr.add_suppression("Statement accesses nontransactional table as well as transactional or temporary table, and writes to any of them");
      SET gtid_domain_id= 1;
      DELETE FROM t1;
      ALTER TABLE t1 ENGINE=InnoDB;
      CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY);
      INSERT INTO t2 VALUES (1);
      INSERT INTO t2 VALUES (2);
      SET gtid_domain_id= 2;
      CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY);
      INSERT INTO t3 VALUES (10);
      INSERT INTO t3 VALUES (20);
      INSERT INTO t1 SELECT a, 'server_1' FROM t2;
      INSERT INTO t1 SELECT a, 'default' FROM t3;
      INSERT INTO t1 SELECT a+2, '+server_1' FROM t2;
      FLUSH TABLES;
      SET SESSION debug_dbug="+d,crash_before_writing_xid";
      INSERT INTO t1 SELECT a+4, '++server_1' FROM t2;
       
       == /dev/shm/var_auto_xTWj/3/tmp/analyze-timeout-mysqld.2.err ==
      mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/var/lib/buildbot/maria-slave/work-opensuse-amd64/build/mysql-test/var/tmp/3/mysqld.2.sock' (111 "Connection refused")
       
       == /dev/shm/var_auto_xTWj/3/tmp/analyze-timeout-mysqld.1.err ==
      mysqltest: Could not open connection 'default': 2013 Lost connection to MySQL server at 'waiting for initial communication packet', system error: 110 "Connection timed out"
       
       
       - skipping '/dev/shm/var_auto_xTWj/3/log/rpl.rpl_parallel_temptable-innodb_plugin,stmt/'
       
      Test rpl.rpl_parallel_temptable has failed 2 times, no more retries!

      I'm getting the last one on my local machine, too.
      On the second time when the server is supposed to die due to the debug injection, the error log says SIGKILL myself and ends with it.

      Attachments

        Issue Links

          Activity

            elenst Elena Stepanova created issue -
            elenst Elena Stepanova made changes -
            Field Original Value New Value
            elenst Elena Stepanova made changes -
            Summary rpl.rpl_parallel_temptable fails sporadically in buildbot rpl.rpl_parallel_temptable fails sporadically in standard buildbot tests and persistently in valgrind builder
            elenst Elena Stepanova made changes -
            Description http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/2885/steps/test_7/logs/stdio

            {noformat}
            CURRENT_TEST: rpl.rpl_parallel_temptable
            mysqltest: In included file "./include/sync_with_master_gtid.inc":
            included from /mnt/buildbot/build/mariadb-10.0.15/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 205:
            At line 44: Failed to sync with master

            The result from queries just before the failure was:
            < snip >

            ALTER TABLE t1 ENGINE=InnoDB;
            CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY);
            INSERT INTO t2 VALUES (1);
            INSERT INTO t2 VALUES (2);
            SET gtid_domain_id= 2;
            CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY);
            INSERT INTO t3 VALUES (10);
            INSERT INTO t3 VALUES (20);
            INSERT INTO t1 SELECT a, 'server_1' FROM t2;
            INSERT INTO t1 SELECT a, 'default' FROM t3;
            INSERT INTO t1 SELECT a+2, '+server_1' FROM t2;
            FLUSH TABLES;
            SET SESSION debug_dbug="+d,crash_before_writing_xid";
            INSERT INTO t1 SELECT a+4, '++server_1' FROM t2;
            Got one of the listed errors
            INSERT INTO t1 VALUES (0, 1);
            include/save_master_gtid.inc
            include/start_slave.inc
            include/sync_with_master_gtid.inc
            Timeout in master_gtid_wait('1-1-31,0-1-4,2-1-17', 120), current slave GTID position is: 1-1-31,2-1-17,0-1-3.
            {noformat}

            Retry passed.
            http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/2885/steps/test_7/logs/stdio

            {noformat}
            CURRENT_TEST: rpl.rpl_parallel_temptable
            mysqltest: In included file "./include/sync_with_master_gtid.inc":
            included from /mnt/buildbot/build/mariadb-10.0.15/mysql-test/suite/rpl/t/rpl_parallel_temptable.test at line 205:
            At line 44: Failed to sync with master

            The result from queries just before the failure was:
            < snip >

            ALTER TABLE t1 ENGINE=InnoDB;
            CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY);
            INSERT INTO t2 VALUES (1);
            INSERT INTO t2 VALUES (2);
            SET gtid_domain_id= 2;
            CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY);
            INSERT INTO t3 VALUES (10);
            INSERT INTO t3 VALUES (20);
            INSERT INTO t1 SELECT a, 'server_1' FROM t2;
            INSERT INTO t1 SELECT a, 'default' FROM t3;
            INSERT INTO t1 SELECT a+2, '+server_1' FROM t2;
            FLUSH TABLES;
            SET SESSION debug_dbug="+d,crash_before_writing_xid";
            INSERT INTO t1 SELECT a+4, '++server_1' FROM t2;
            Got one of the listed errors
            INSERT INTO t1 VALUES (0, 1);
            include/save_master_gtid.inc
            include/start_slave.inc
            include/sync_with_master_gtid.inc
            Timeout in master_gtid_wait('1-1-31,0-1-4,2-1-17', 120), current slave GTID position is: 1-1-31,2-1-17,0-1-3.
            {noformat}

            Retry passed.


            http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/6310/steps/test/logs/stdio
            {noformat}
            rpl.rpl_parallel_temptable 'innodb_plugin,stmt' w3 [ retry-fail ] timeout after 9000 seconds
                    Test ended at 2014-11-11 06:46:44

            Test case timeout after 9000 seconds

            == /dev/shm/var_auto_xTWj/3/log/rpl_parallel_temptable.log ==
            Slave_open_temp_tables 0
            *** Test that if master logged partial event group before crash, we finish that group correctly before executing format description event ***
            include/stop_slave.inc
            CALL mtr.add_suppression("Statement accesses nontransactional table as well as transactional or temporary table, and writes to any of them");
            SET gtid_domain_id= 1;
            DELETE FROM t1;
            ALTER TABLE t1 ENGINE=InnoDB;
            CREATE TEMPORARY TABLE t2 (a INT PRIMARY KEY);
            INSERT INTO t2 VALUES (1);
            INSERT INTO t2 VALUES (2);
            SET gtid_domain_id= 2;
            CREATE TEMPORARY TABLE t3 (a INT PRIMARY KEY);
            INSERT INTO t3 VALUES (10);
            INSERT INTO t3 VALUES (20);
            INSERT INTO t1 SELECT a, 'server_1' FROM t2;
            INSERT INTO t1 SELECT a, 'default' FROM t3;
            INSERT INTO t1 SELECT a+2, '+server_1' FROM t2;
            FLUSH TABLES;
            SET SESSION debug_dbug="+d,crash_before_writing_xid";
            INSERT INTO t1 SELECT a+4, '++server_1' FROM t2;

             == /dev/shm/var_auto_xTWj/3/tmp/analyze-timeout-mysqld.2.err ==
            mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/var/lib/buildbot/maria-slave/work-opensuse-amd64/build/mysql-test/var/tmp/3/mysqld.2.sock' (111 "Connection refused")

             == /dev/shm/var_auto_xTWj/3/tmp/analyze-timeout-mysqld.1.err ==
            mysqltest: Could not open connection 'default': 2013 Lost connection to MySQL server at 'waiting for initial communication packet', system error: 110 "Connection timed out"


             - skipping '/dev/shm/var_auto_xTWj/3/log/rpl.rpl_parallel_temptable-innodb_plugin,stmt/'

            Test rpl.rpl_parallel_temptable has failed 2 times, no more retries!
            {noformat}

            I'm getting the last one on my local machine, too.
            On the second time when the server is supposed to die due to the debug injection, the error log says SIGKILL myself and ends with it.
            knielsen Kristian Nielsen made changes -
            knielsen Kristian Nielsen made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            knielsen Kristian Nielsen made changes -
            Component/s Tests [ 10800 ]
            Component/s OTHER [ 10125 ]
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Closed [ 6 ]
            elenst Elena Stepanova made changes -
            elenst Elena Stepanova made changes -
            ratzpo Rasmus Johansson (Inactive) made changes -
            Workflow MariaDB v2 [ 58420 ] MariaDB v3 [ 65456 ]
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 65456 ] MariaDB v4 [ 148451 ]

            People

              knielsen Kristian Nielsen
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.