Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-10254

innodb.innodb_simulate_comp_failures fails with timeout when it's run on bb04

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 5.5(EOL), 10.0(EOL)
    • N/A
    • Tests
    • None

    Description

      Recently we started running fulltest builder on bb04 (in addition to knielsen-kvm-x86 and centos56-quality2). On some reason, tests sporadically fails with a timeout there:
      http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/5453/steps/test_3/logs/stdio

      innodb.innodb_simulate_comp_failures 'innodb_plugin' w1 [ fail ]  timeout after 7200 seconds
              Test ended at 2016-06-17 21:25:23
       
      Test case timeout after 7200 seconds
       
      == /mnt/buildbot/build/mariadb-5.5.50/mysql-test/var/1/log/innodb_simulate_comp_failures.log == 
      #
      # Testing robustness against random compression failures
      #
      CREATE TABLE t1(id INT AUTO_INCREMENT PRIMARY KEY, msg VARCHAR(255), KEY msg_i(msg)) ENGINE=INNODB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8;
      SET GLOBAL innodb_simulate_comp_failures = 25;
      SELECT COUNT(*) FROM t1;
      COUNT(*)
      100000
       
       == /mnt/buildbot/build/mariadb-5.5.50/mysql-test/var/1/tmp/analyze-timeout-mysqld.1.err ==
      mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/mnt/buildbot/build/mariadb-5.5.50/mysql-test/var/tmp/1/mysqld.1.sock' (111)
       
       
       - saving '/mnt/buildbot/build/mariadb-5.5.50/mysql-test/var/1/log/innodb.innodb_simulate_comp_failures-innodb_plugin/' to '/mnt/buildbot/build/mariadb-5.5.50/mysql-test/var/log/innodb.innodb_simulate_comp_failures-innodb_plugin/'
      

      When they pass, they take longer than on other machines, but not that much longer, so something else seems to be wrong, it needs to be investigated.

      Similar thing happens with parts.partition_basic_innodb on 10.0.
      http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/6125/steps/test_3/logs/stdio
      It does not show up as a test failure, but the whole suite times out due to it:

      worker[1] Test still running: parts.partition_basic_innodb
      Test suite timeout! Terminating...
      

      Attachments

        Issue Links

          Activity

            elenst Elena Stepanova added a comment - - edited

            Comparing http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/5453/steps/test_3/logs/stdio with http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/5454/steps/test_3/logs/stdio

                                                                 :      old     bb04   ratio
            -------------------------------------------------------------------------------
                        archive.archive-big                      :   574488   414089   0.72
                        federated.federated_server 'X'           :    25102    11662   0.46
                        funcs_1.myisam_views-big                 :    90606    45783   0.51
                   innodb.innodb_simulate_comp_failures 'xtradb' :  1158284  7144846   6.17
                        innodb.sp_temp_table 'innodb_plugin'     :    30246    30461   1.01
                        innodb.sp_temp_table 'xtradb'            :    30223    30544   1.01
                        main.alter_table-big                     :      153       46   0.30
                        main.count_distinct3                     :    48758    29837   0.61
                        main.create-big                          :      362      122   0.34
                        main.create_delayed                      :     6306     3096   0.49
                        main.events_stress                       :     6092     5575   0.92
                        main.information_schema-big 'xtradb'     :     6275     2691   0.43
                        main.merge-big                           :    30069    30056   1.00
                        main.myisam-big                          :    26938     9199   0.34
                        main.myisam-metadata                     :    96422    36288   0.38
                        main.mysqlbinlog_row_big                 :    14942     4903   0.33
                        main.ssl-big                             :   326184   156805   0.48
                        main.sum_distinct-big 'innodb_plugin'    :   711773   309295   0.43
                        main.sum_distinct-big 'xtradb'           :   707946   296763   0.42
                        main.type_newdecimal-big                 :     9845     4703   0.48
                        maria.maria-big2                         :     5139     1401   0.27
                        maria.maria-big                          :    87788    52240   0.60
                        maria.maria-purge                        :    58973    24562   0.42
                        maria.maria-recovery-big                 :    37318    19806   0.53
                        maria.max_length                         :    54232    34330   0.63
               parts.partition_alter1_1_2_innodb 'innodb_plugin' :    65855   235758   3.58
                      parts.partition_alter1_1_2_innodb 'xtradb' :    65731   233383   3.55
                 parts.partition_alter1_2_innodb 'innodb_plugin' :    78650   292255   3.72
                        parts.partition_alter1_2_innodb 'xtradb' :    76681   292108   3.81
               parts.partition_alter2_1_1_innodb 'innodb_plugin' :    42463   164177   3.87
                      parts.partition_alter2_1_1_innodb 'xtradb' :    42821   164448   3.84
               parts.partition_alter2_1_2_innodb 'innodb_plugin' :    43707   174318   3.99
                      parts.partition_alter2_1_2_innodb 'xtradb' :    42499   175112   4.12
                        parts.partition_alter2_1_maria           :    40384    17810   0.44
               parts.partition_alter2_2_1_innodb 'innodb_plugin' :    42138   222652   5.28
                      parts.partition_alter2_2_1_innodb 'xtradb' :    42518   226342   5.32
               parts.partition_alter2_2_2_innodb 'innodb_plugin' :    42877   226207   5.28
                      parts.partition_alter2_2_2_innodb 'xtradb' :    43668   221382   5.07
                        parts.partition_alter2_2_maria           :    39575    17981   0.45
                   parts.partition_alter4_innodb 'innodb_plugin' :   178323   656458   3.68
                        parts.partition_alter4_innodb 'xtradb'   :   174092   649903   3.73
                        parts.partition_alter4_myisam            :   120843    49739   0.41
                    parts.partition_basic_innodb 'innodb_plugin' :    61234   219048   3.58
                        parts.partition_basic_innodb 'xtradb'    :    60369   216364   3.58
                  parts.partition_decimal_innodb 'innodb_plugin' :     2838     2751   0.97
                        parts.partition_decimal_innodb 'xtradb'  :     2749     3103   1.13
                        parts.partition_decimal_myisam           :   219034    91726   0.42
            parts.part_supported_sql_func_innodb 'innodb_plugin' :    28560   148257   5.19
                   parts.part_supported_sql_func_innodb 'xtradb' :    28317   148345   5.24
                        parts.rpl_partition 'innodb_plugin,mix'  :     5367     6661   1.24
                        parts.rpl_partition 'innodb_plugin,row'  :     5109     6189   1.21
                        parts.rpl_partition 'innodb_plugin,stmt' :     5319     5868   1.10
                        parts.rpl_partition 'mix,xtradb'         :     5199     8571   1.65
                        parts.rpl_partition 'row,xtradb'         :     5237     5681   1.08
                        parts.rpl_partition 'stmt,xtradb'        :     5220     5124   0.98
                        percona.innodb_sys_index 'xtradb'        :      140      198   1.41
             percona.percona_flush_contiguous_neighbors 'xtradb' :    50570    35161   0.70
                     rpl.rpl_innodb_bug28430 'innodb_plugin,mix' :     5295     4349   0.82
                     rpl.rpl_innodb_bug28430 'innodb_plugin,row' :     5569     4116   0.74
                        rpl.rpl_innodb_bug28430 'mix,xtradb'     :     5381     3700   0.69
                        rpl.rpl_innodb_bug28430 'row,xtradb'     :     6050     6203   1.03
            

            elenst Elena Stepanova added a comment - - edited Comparing http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/5453/steps/test_3/logs/stdio with http://buildbot.askmonty.org/buildbot/builders/kvm-fulltest2/builds/5454/steps/test_3/logs/stdio : old bb04 ratio ------------------------------------------------------------------------------- archive.archive-big : 574488 414089 0.72 federated.federated_server 'X' : 25102 11662 0.46 funcs_1.myisam_views-big : 90606 45783 0.51 innodb.innodb_simulate_comp_failures 'xtradb' : 1158284 7144846 6.17 innodb.sp_temp_table 'innodb_plugin' : 30246 30461 1.01 innodb.sp_temp_table 'xtradb' : 30223 30544 1.01 main.alter_table-big : 153 46 0.30 main.count_distinct3 : 48758 29837 0.61 main.create-big : 362 122 0.34 main.create_delayed : 6306 3096 0.49 main.events_stress : 6092 5575 0.92 main.information_schema-big 'xtradb' : 6275 2691 0.43 main.merge-big : 30069 30056 1.00 main.myisam-big : 26938 9199 0.34 main.myisam-metadata : 96422 36288 0.38 main.mysqlbinlog_row_big : 14942 4903 0.33 main.ssl-big : 326184 156805 0.48 main.sum_distinct-big 'innodb_plugin' : 711773 309295 0.43 main.sum_distinct-big 'xtradb' : 707946 296763 0.42 main.type_newdecimal-big : 9845 4703 0.48 maria.maria-big2 : 5139 1401 0.27 maria.maria-big : 87788 52240 0.60 maria.maria-purge : 58973 24562 0.42 maria.maria-recovery-big : 37318 19806 0.53 maria.max_length : 54232 34330 0.63 parts.partition_alter1_1_2_innodb 'innodb_plugin' : 65855 235758 3.58 parts.partition_alter1_1_2_innodb 'xtradb' : 65731 233383 3.55 parts.partition_alter1_2_innodb 'innodb_plugin' : 78650 292255 3.72 parts.partition_alter1_2_innodb 'xtradb' : 76681 292108 3.81 parts.partition_alter2_1_1_innodb 'innodb_plugin' : 42463 164177 3.87 parts.partition_alter2_1_1_innodb 'xtradb' : 42821 164448 3.84 parts.partition_alter2_1_2_innodb 'innodb_plugin' : 43707 174318 3.99 parts.partition_alter2_1_2_innodb 'xtradb' : 42499 175112 4.12 parts.partition_alter2_1_maria : 40384 17810 0.44 parts.partition_alter2_2_1_innodb 'innodb_plugin' : 42138 222652 5.28 parts.partition_alter2_2_1_innodb 'xtradb' : 42518 226342 5.32 parts.partition_alter2_2_2_innodb 'innodb_plugin' : 42877 226207 5.28 parts.partition_alter2_2_2_innodb 'xtradb' : 43668 221382 5.07 parts.partition_alter2_2_maria : 39575 17981 0.45 parts.partition_alter4_innodb 'innodb_plugin' : 178323 656458 3.68 parts.partition_alter4_innodb 'xtradb' : 174092 649903 3.73 parts.partition_alter4_myisam : 120843 49739 0.41 parts.partition_basic_innodb 'innodb_plugin' : 61234 219048 3.58 parts.partition_basic_innodb 'xtradb' : 60369 216364 3.58 parts.partition_decimal_innodb 'innodb_plugin' : 2838 2751 0.97 parts.partition_decimal_innodb 'xtradb' : 2749 3103 1.13 parts.partition_decimal_myisam : 219034 91726 0.42 parts.part_supported_sql_func_innodb 'innodb_plugin' : 28560 148257 5.19 parts.part_supported_sql_func_innodb 'xtradb' : 28317 148345 5.24 parts.rpl_partition 'innodb_plugin,mix' : 5367 6661 1.24 parts.rpl_partition 'innodb_plugin,row' : 5109 6189 1.21 parts.rpl_partition 'innodb_plugin,stmt' : 5319 5868 1.10 parts.rpl_partition 'mix,xtradb' : 5199 8571 1.65 parts.rpl_partition 'row,xtradb' : 5237 5681 1.08 parts.rpl_partition 'stmt,xtradb' : 5220 5124 0.98 percona.innodb_sys_index 'xtradb' : 140 198 1.41 percona.percona_flush_contiguous_neighbors 'xtradb' : 50570 35161 0.70 rpl.rpl_innodb_bug28430 'innodb_plugin,mix' : 5295 4349 0.82 rpl.rpl_innodb_bug28430 'innodb_plugin,row' : 5569 4116 0.74 rpl.rpl_innodb_bug28430 'mix,xtradb' : 5381 3700 0.69 rpl.rpl_innodb_bug28430 'row,xtradb' : 6050 6203 1.03

            I've removed bb04 from the list of slaves for fulltests for now.

            elenst Elena Stepanova added a comment - I've removed bb04 from the list of slaves for fulltests for now.

            People

              elenst Elena Stepanova
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.