[MDEV-102] mysql-test-run runs too slow and/or consumes too much disk space and memory Created: 2012-01-19 Updated: 2012-01-27 Resolved: 2012-01-27 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | None |
| Affects Version/s: | 5.5.20 |
| Fix Version/s: | 5.5.20 |
| Type: | Bug | Priority: | Major |
| Reporter: | Kristian Nielsen | Assignee: | Oleksandr Byelkin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | tests | ||
| Description |
|
mysql-test-run consumes much more resources in 5.5 compared to 5.3. In 5.3 I was able to run the entire test suite (with --mem --parallel=10) in around 6 minutes on a machine with 8GB of memory. 3 min for a release build. In 5.5 it takes much longer. And worse, we see failures on hosts due to consuming too much resource. Eg. the DGCov builder in buildbot ran out of memory just using --mem --parallel=2, with 3GB of memory available. It still needs to be quantified more precisely what the problems are. Possibilities include
|
| Comments |
| Comment by Kristian Nielsen [ 2012-01-20 ] |
|
For example, after I reduced DGCov in buildbot to --parallel=2 and no --mem, the test works, but it takes around 10 hours to complete, which is hardly acceptable. |
| Comment by Vladislav Vaintroub [ 2012-01-20 ] |
|
Please ignore all the changes I done to that bug so far.Mixed something up in JIRA. I'm not working on this one. |
| Comment by Sergei Golubchik [ 2012-01-23 ] |
|
I'm not sure we're talking about the same buildbot and the same dgcov. http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/1097 http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/1094 5.5 build Other builders (that I've checked) also show that in 5.5 tests don't take significantly more time that in 5.3 |
| Comment by Sergei Golubchik [ 2012-01-23 ] |
|
ok. tests are only slow if --mem is used. They consume all available memory. ma_pagecache_single_8k-t => 160 Mb |
| Comment by Sergei Golubchik [ 2012-01-23 ] |
|
although it's easy to skip the complete test as "big", it would be better to adjust the test size to create smaller files, unless "big" test is requested (that is, skip_big_tests is 0). when fixing it, one also needs to fix suite/unit/suite.pm to pass "big" option down to test executables: $ENV {MYTAP_CONFIG}='big' if $::opt_big_test; |
| Comment by Oleksandr Byelkin [ 2012-01-24 ] |
|
It has no sens to run that test on smaller files (nothing to test). so I'll just skip it on BIG. But have we at least one buildbot slave which runs "big" test? |
| Comment by Oleksandr Byelkin [ 2012-01-27 ] |
|
=== modified file 'storage/maria/unittest/CMakeLists.txt' ADD_EXECUTABLE(ma_pagecache_single_8k-t ${ma_pagecache_single_src})
ADD_EXECUTABLE(ma_pagecache_single_64k-t ${ma_pagecache_single_src})
ADD_EXECUTABLE(ma_pagecache_consist_1k-t ${ma_pagecache_consist_src}) === modified file 'storage/maria/unittest/ma_test_loghandler_multigroup-t.c' #else @@ -48,6 +49,7 @@ static TRN *trn= &dummy_transaction_obje #endif /READONLY_TEST/ @@ -243,6 +245,15 @@ int main(int argc _attribute_((unused) + plan(0); // read configuration (MYTAP_CONFIG) +#endif load_defaults("my", load_default_groups, &argc, &argv); === modified file 'storage/maria/unittest/ma_test_loghandler_multithread-t.c' #ifdef MULTIFLUSH_TEST -#define LONG_BUFFER_SIZE (16384L) #else -#define LONG_BUFFER_SIZE (512L*1024L*1024L) #endif +#define LONG_BUFFER_SIZE (LONG_BUFFER_SZ >> (skip_big_tests ? 4 : 0)) @@ -270,6 +272,7 @@ int main(int argc _attribute_((unused) + // plan read MYTAP_CONFIG so skip_big_tests will be set before using |
| Comment by Oleksandr Byelkin [ 2012-01-27 ] |
|
fix pushed |