Details
-
Task
-
Status: Stalled (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
-
None
-
2017-02, 10.1.22
Description
Spider tests require more resources than usual tests. Sergei Vojtovich made this estimation:
8 mysqld instances are started by spider test suites (other suites need max 4)
48 mysqld instances will be started with --parallel=61 mysqld instance started by mtr needs ~100M RAM (=> 48 need ~4.8Gb)
main.alias needs 50Mb disk space to complete (=> 6 parallel runs need 300Mb)
spider.* need up to 300Mb disk space to complete (=> 6 parallel runs need 1.8G)In other words: spider needs 6x more disk space and 8x more RAM than an average
test case.
Usually Spider tests pass in MariaDB build-bot, but often fail in Debian build-bot.
Otto disabled the following tests (see MDEV-9329) to make Debian builds clean:
spider.basic_sql : https://jira.mariadb.org/browse/MDEV-9329
|
spider.basic_sql_part : MDEV-9329
|
spider/bg.ha : MDEV-9329
|
spider/bg.ha_part : MDEV-9329
|
spider/bg.vp_fixes : MDEV-9329
|
spider.spider_fixes : MDEV-9329
|
spider.spider3_fixes : MDEV-9329
|
spider.spider3_fixes_part : MDEV-9329
|
spider.direct_update : MDEV-9329
|
spider.function : MDEV-9329
|
spider.direct_aggregate : MDEV-9329
|
spider.direct_aggregate_part : MDEV-9329
|
See here for details:
https://github.com/ottok/mariadb-10.0/blob/master/debian/unstable-tests.s390x
We should somehow make these tests pass and re-enable them.
One of the ideas is to run resource hangry tests without using threads, in a separate run.
Sergei Golubchik proposed the following solution:
Depends on how Debian run tests. If it wants to run all tests, the best
approach would be
./mtr --parallel N; ./mtr --big --big
that is to run all non-big tests with parallel, and then only big tests
in single-process mode.Technically, we can add some kind of --auto mode that does the above
automatically... But then Debian scripts would need to be fixed anyway
(to use --auto). So it's simpler to fix them to run --big --big tests.
Kentoku proposed this solution:
I just feel this issues is caused by creating threads. By default
settings, Spider creates a thread per table/partition for getting
statistical information from data nodes.
You can switch creating threads off by setting "spider_crd_bg_mode =
0" and "spider_sts_bg_mode = 0" in my.cnf.
Attachments
Issue Links
- relates to
-
MDEV-9329 10.0 fails to build on official Ubuntu/s390x builders due to Spider test failures
-
- Closed
-