Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-9730

Re-enable Spider tests in Debian git tree

    XMLWordPrintable

Details

    • 2017-02, 10.1.22

    Description

      Spider tests require more resources than usual tests. Sergei Vojtovich made this estimation:

      8 mysqld instances are started by spider test suites (other suites need max 4)
      48 mysqld instances will be started with --parallel=6

      1 mysqld instance started by mtr needs ~100M RAM (=> 48 need ~4.8Gb)

      main.alias needs 50Mb disk space to complete (=> 6 parallel runs need 300Mb)
      spider.* need up to 300Mb disk space to complete (=> 6 parallel runs need 1.8G)

      In other words: spider needs 6x more disk space and 8x more RAM than an average
      test case.

      Usually Spider tests pass in MariaDB build-bot, but often fail in Debian build-bot.

      Otto disabled the following tests (see MDEV-9329) to make Debian builds clean:

      spider.basic_sql : https://jira.mariadb.org/browse/MDEV-9329
      spider.basic_sql_part : MDEV-9329
      spider/bg.ha : MDEV-9329
      spider/bg.ha_part : MDEV-9329
      spider/bg.vp_fixes : MDEV-9329
      spider.spider_fixes : MDEV-9329
      spider.spider3_fixes : MDEV-9329
      spider.spider3_fixes_part : MDEV-9329
      spider.direct_update : MDEV-9329
      spider.function : MDEV-9329
      spider.direct_aggregate : MDEV-9329
      spider.direct_aggregate_part : MDEV-9329
      

      See here for details:
      https://github.com/ottok/mariadb-10.0/blob/master/debian/unstable-tests.s390x

      We should somehow make these tests pass and re-enable them.

      One of the ideas is to run resource hangry tests without using threads, in a separate run.
      Sergei Golubchik proposed the following solution:

      Depends on how Debian run tests. If it wants to run all tests, the best
      approach would be

      ./mtr --parallel N; ./mtr --big --big
      

      that is to run all non-big tests with parallel, and then only big tests
      in single-process mode.

      Technically, we can add some kind of --auto mode that does the above
      automatically... But then Debian scripts would need to be fixed anyway
      (to use --auto). So it's simpler to fix them to run --big --big tests.

      Kentoku proposed this solution:

      I just feel this issues is caused by creating threads. By default
      settings, Spider creates a thread per table/partition for getting
      statistical information from data nodes.
      You can switch creating threads off by setting "spider_crd_bg_mode =
      0" and "spider_sts_bg_mode = 0" in my.cnf.

      Attachments

        Issue Links

          Activity

            People

              ycp Yuchen Pei
              bar Alexander Barkov
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.