[MDEV-9730] Re-enable Spider tests in Debian git tree Created: 2016-03-15  Updated: 2023-04-14

Status: Stalled
Project: MariaDB Server
Component/s: Storage Engine - Spider
Fix Version/s: None

Type: Task Priority: Major
Reporter: Alexander Barkov Assignee: Yuchen Pei
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-9329 10.0 fails to build on official Ubunt... Closed
Sprint: 2017-02, 10.1.22

 Description   

Spider tests require more resources than usual tests. Sergei Vojtovich made this estimation:

8 mysqld instances are started by spider test suites (other suites need max 4)
48 mysqld instances will be started with --parallel=6

1 mysqld instance started by mtr needs ~100M RAM (=> 48 need ~4.8Gb)

main.alias needs 50Mb disk space to complete (=> 6 parallel runs need 300Mb)
spider.* need up to 300Mb disk space to complete (=> 6 parallel runs need 1.8G)

In other words: spider needs 6x more disk space and 8x more RAM than an average
test case.

Usually Spider tests pass in MariaDB build-bot, but often fail in Debian build-bot.

Otto disabled the following tests (see MDEV-9329) to make Debian builds clean:

spider.basic_sql : https://jira.mariadb.org/browse/MDEV-9329
 
spider.basic_sql_part : MDEV-9329
 
spider/bg.ha : MDEV-9329
 
spider/bg.ha_part : MDEV-9329
 
spider/bg.vp_fixes : MDEV-9329
 
spider.spider_fixes : MDEV-9329
 
spider.spider3_fixes : MDEV-9329
 
spider.spider3_fixes_part : MDEV-9329
 
spider.direct_update : MDEV-9329
 
spider.function : MDEV-9329
 
spider.direct_aggregate : MDEV-9329
 
spider.direct_aggregate_part : MDEV-9329

See here for details:
https://github.com/ottok/mariadb-10.0/blob/master/debian/unstable-tests.s390x

We should somehow make these tests pass and re-enable them.

One of the ideas is to run resource hangry tests without using threads, in a separate run.
Sergei Golubchik proposed the following solution:

Depends on how Debian run tests. If it wants to run all tests, the best
approach would be

./mtr --parallel N; ./mtr --big --big

that is to run all non-big tests with parallel, and then only big tests
in single-process mode.

Technically, we can add some kind of --auto mode that does the above
automatically... But then Debian scripts would need to be fixed anyway
(to use --auto). So it's simpler to fix them to run --big --big tests.

Kentoku proposed this solution:

I just feel this issues is caused by creating threads. By default
settings, Spider creates a thread per table/partition for getting
statistical information from data nodes.
You can switch creating threads off by setting "spider_crd_bg_mode =
0" and "spider_sts_bg_mode = 0" in my.cnf.



 Comments   
Comment by Jacob Mathew (Inactive) [ 2017-02-27 ]

I am implementing Sergei Golubchik's proposed solution, even though it requires more changes then Kentoku's solution. The tests are more meaningful if they are run with a configuration that is more like a real end user's configuration, not using a configuration that no real end user will actually use.

A test is marked as a big test by including the line 

 --source include/big_test.inc

in its .test file. A test that is marked as a big test is run only if the mysql-test option for big tests 

 --big-test

is used, or if the environment variable BIG_TEST is set to 1.

Comment by Jacob Mathew (Inactive) [ 2017-02-28 ]

To test the effect of inserting the additional include file at the beginning of a test, I added that change to 8 tests in just the spider/bg suite, which consists of 14 tests.

Without the --big-test option, the 8 tests marked as big tests are skipped. With the --big-test option specified once, all 14 tests are run. With the --big-test option specified twice, the 6 tests not marked as big tests are skipped. With the --big-test option specified thrice, the behavior is the same as when that option is specified twice. Evidently, this change has exactly the desired effect.

Comment by Jacob Mathew (Inactive) [ 2017-02-28 ]

At this point, I need to know precisely which of the Spider tests need to be marked as big tests. According to the unstable_tests list, all Spider tests (spider.*) consume too much memory. So, should all Spider tests be marked as big tests?

Also, do the 8 spider/bg tests listed in unstable_tests need to remain in that list? Or, should the 2 tests listed as failing due to bug MDEV-9329 be marked as big tests, with the others remaining in the unstable_tests list?

Comment by Jacob Mathew (Inactive) [ 2017-03-01 ]

We need more information about the test failures on Debian to know which Spider tests need to be marked as big tests. There are also other ways to run the Spider tests separately that we should consider as well. We also need to know exactly what Debian wants from us in addressing the Spider test failures. The best source for this information is Vicentiu's contact Otto at Debian. Vicentiu is contacting Otto to get this information. Once we have this information, we can make a more informed decision about how to proceed with this issue.

Generated at Thu Feb 08 07:36:53 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.