[MDEV-102] mysql-test-run runs too slow and/or consumes too much disk space and memory Created: 2012-01-19  Updated: 2012-01-27  Resolved: 2012-01-27

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 5.5.20
Fix Version/s: 5.5.20

Type: Bug Priority: Major
Reporter: Kristian Nielsen Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 0
Labels: tests


 Description   

mysql-test-run consumes much more resources in 5.5 compared to 5.3.

In 5.3 I was able to run the entire test suite (with --mem --parallel=10) in around 6 minutes on a machine with 8GB of memory. 3 min for a release build.

In 5.5 it takes much longer. And worse, we see failures on hosts due to consuming too much resource. Eg. the DGCov builder in buildbot ran out of memory just using --mem --parallel=2, with 3GB of memory available.

It still needs to be quantified more precisely what the problems are. Possibilities include

  • Writing too large files in mysql-test/var/, overflowing memory
  • safemalloc overhead
  • unit tests being included in mtr (f.ex. some of the aria unit tests consume huge amount of resources)
  • something else entirely


 Comments   
Comment by Kristian Nielsen [ 2012-01-20 ]

For example, after I reduced DGCov in buildbot to --parallel=2 and no --mem, the test works, but it takes around 10 hours to complete, which is hardly acceptable.

Comment by Vladislav Vaintroub [ 2012-01-20 ]

Please ignore all the changes I done to that bug so far.Mixed something up in JIRA. I'm not working on this one.

Comment by Sergei Golubchik [ 2012-01-23 ]

I'm not sure we're talking about the same buildbot and the same dgcov.

http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/1097
5.3 build.
test nm: 2 hrs, 43 mins, 33 secs
total elapsed (compiling, gcov, few test runs): 6 hrs, 53 mins, 31 secs

http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/1094

5.5 build
test nm: 2 hrs, 43 mins, 37 secs
total elapsed: 6 hrs, 18 mins, 51 secs

Other builders (that I've checked) also show that in 5.5 tests don't take significantly more time that in 5.3

Comment by Sergei Golubchik [ 2012-01-23 ]

ok. tests are only slow if --mem is used. They consume all available memory.
Because in 5.5 unit tests are part of mtr run, and few aria unit tests create huge files:

ma_pagecache_single_8k-t => 160 Mb
ma_test_loghandler_multithread-t => 692 Mb
ma_test_loghandler_multigroup-t => 1189 Mb
ma_pagecache_single_64k-t => 1280 Mb
ma_test_loghandler_multithread-t => 1536 Mb

Comment by Sergei Golubchik [ 2012-01-23 ]

although it's easy to skip the complete test as "big", it would be better to adjust the test size to create smaller files, unless "big" test is requested (that is, skip_big_tests is 0).

when fixing it, one also needs to fix suite/unit/suite.pm to pass "big" option down to test executables:

$ENV

{MYTAP_CONFIG}

='big' if $::opt_big_test;

Comment by Oleksandr Byelkin [ 2012-01-24 ]

It has no sens to run that test on smaller files (nothing to test). so I'll just skip it on BIG. But have we at least one buildbot slave which runs "big" test?

Comment by Oleksandr Byelkin [ 2012-01-27 ]

=== modified file 'storage/maria/unittest/CMakeLists.txt'
— storage/maria/unittest/CMakeLists.txt 2011-11-22 17:04:38 +0000
+++ storage/maria/unittest/CMakeLists.txt 2012-01-27 08:24:08 +0000
@@ -76,12 +76,12 @@ MY_ADD_TEST(ma_pagecache_single_1k)

ADD_EXECUTABLE(ma_pagecache_single_8k-t ${ma_pagecache_single_src})
SET_TARGET_PROPERTIES(ma_pagecache_single_8k-t

  • PROPERTIES COMPILE_FLAGS "${ma_pagecache_common_cppflags} -DTEST_PAGE_SIZE=8192")
    + PROPERTIES COMPILE_FLAGS "${ma_pagecache_common_cppflags} -DTEST_PAGE_SIZE=8192 -DBIG")
    MY_ADD_TEST(ma_pagecache_single_8k)

ADD_EXECUTABLE(ma_pagecache_single_64k-t ${ma_pagecache_single_src})
SET_TARGET_PROPERTIES(ma_pagecache_single_64k-t

  • PROPERTIES COMPILE_FLAGS "${ma_pagecache_common_cppflags} -DTEST_PAGE_SIZE=65536")
    + PROPERTIES COMPILE_FLAGS "${ma_pagecache_common_cppflags} -DTEST_PAGE_SIZE=65536 -DBIG")
    MY_ADD_TEST(ma_pagecache_single_64k)

ADD_EXECUTABLE(ma_pagecache_consist_1k-t ${ma_pagecache_consist_src})

=== modified file 'storage/maria/unittest/ma_test_loghandler_multigroup-t.c'
— storage/maria/unittest/ma_test_loghandler_multigroup-t.c 2011-10-19 19:45:18 +0000
+++ storage/maria/unittest/ma_test_loghandler_multigroup-t.c 2012-01-27 08:53:46 +0000
@@ -39,6 +39,7 @@ static TRN *trn= &dummy_transaction_obje
#define LOG_FILE_SIZE (1024L*1024L*1024L + 1024L*1024L*512)
#define ITERATIONS 2
#define READONLY 0
+#define BIG 1

#else

@@ -48,6 +49,7 @@ static TRN *trn= &dummy_transaction_obje
#define LOG_FILE_SIZE (1024L*1024L*1024L + 1024L*1024L*512)
#define ITERATIONS 2
#define READONLY 1
+#undef BIG

#endif /READONLY_TEST/

@@ -243,6 +245,15 @@ int main(int argc _attribute_((unused)
int rc;
MY_INIT(argv[0]);

+ plan(0); // read configuration (MYTAP_CONFIG)
+#ifdef BIG
+ if (skip_big_tests)
+

{ + plan(1); + ok(1, "skipped as big test"); + return 0; + }

+#endif

load_defaults("my", load_default_groups, &argc, &argv);
get_options(&argc, &argv);

=== modified file 'storage/maria/unittest/ma_test_loghandler_multithread-t.c'
— storage/maria/unittest/ma_test_loghandler_multithread-t.c 2011-10-19 19:45:18 +0000
+++ storage/maria/unittest/ma_test_loghandler_multithread-t.c 2012-01-27 08:55:11 +0000
@@ -35,7 +35,7 @@ static const char *default_dbug_option;

#ifdef MULTIFLUSH_TEST

-#define LONG_BUFFER_SIZE (16384L)
+#define LONG_BUFFER_SZ (16384L)
#define MIN_REC_LENGTH 10
#define SHOW_DIVIDER 20
#define ITERATIONS 10000
@@ -45,7 +45,7 @@ static const char *default_dbug_option;

#else

-#define LONG_BUFFER_SIZE (512L*1024L*1024L)
+#define LONG_BUFFER_SZ (512L*1024L*1024L)
#define MIN_REC_LENGTH 30
#define SHOW_DIVIDER 10
#define ITERATIONS 3
@@ -55,6 +55,8 @@ static const char *default_dbug_option;

#endif

+#define LONG_BUFFER_SIZE (LONG_BUFFER_SZ >> (skip_big_tests ? 4 : 0))
+
static uint number_of_writers= WRITERS;
static uint number_of_flushers= FLUSHERS;

@@ -270,6 +272,7 @@ int main(int argc _attribute_((unused)
int rc;
MY_INIT(argv[0]);

+ // plan read MYTAP_CONFIG so skip_big_tests will be set before using
plan(WRITERS + FLUSHERS +
ITERATIONS * WRITERS * 3 + FLUSH_ITERATIONS * FLUSHERS );
/* We don't need to do physical syncs in this test */

Comment by Oleksandr Byelkin [ 2012-01-27 ]

fix pushed

Generated at Thu Feb 08 06:26:17 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.