Details
-
Bug
-
Status: Needs Feedback (View Workflow)
-
Minor
-
Resolution: Unresolved
-
10.6.1
-
None
Description
I noticed that when running 'make tests' directly after a build (any build, on all platforms I tested) it does not pass.
The CUnit tests should pass. If there are tests that are permanently broken or require something special, then they should not be part of the standard CUnit test run, but separated. It does not make sense for a 'make test' to fail on a vanilla build.
Full output:
# make test
|
Running tests...
|
Test project /build/builddir
|
Start 1: strings
|
1/86 Test #1: strings .......................... Passed 0.01 sec
|
Start 2: json
|
2/86 Test #2: json ............................. Passed 0.00 sec
|
Start 3: simple
|
3/86 Test #3: simple ........................... Passed 0.21 sec
|
Start 4: skip
|
4/86 Test #4: skip ............................. Passed 0.00 sec
|
Start 5: todo
|
5/86 Test #5: todo ............................. Passed 0.00 sec
|
Start 6: skip_all
|
6/86 Test #6: skip_all ......................... Passed 0.00 sec
|
Start 7: no_plan
|
7/86 Test #7: no_plan .......................... Passed 0.00 sec
|
Start 8: bitmap
|
8/86 Test #8: bitmap ........................... Passed 0.16 sec
|
Start 9: base64
|
9/86 Test #9: base64 ........................... Passed 0.02 sec
|
Start 10: my_atomic
|
10/86 Test #10: my_atomic ........................ Passed 0.02 sec
|
Start 11: my_rdtsc
|
11/86 Test #11: my_rdtsc ......................... Passed 0.02 sec
|
Start 12: lf
|
12/86 Test #12: lf ............................... Passed 0.39 sec
|
Start 13: my_malloc
|
13/86 Test #13: my_malloc ........................ Passed 0.00 sec
|
Start 14: my_getopt
|
14/86 Test #14: my_getopt ........................ Passed 0.00 sec
|
Start 15: dynstring
|
15/86 Test #15: dynstring ........................ Passed 0.00 sec
|
Start 16: byte_order
|
16/86 Test #16: byte_order ....................... Passed 0.00 sec
|
Start 17: queues
|
17/86 Test #17: queues ........................... Passed 0.01 sec
|
Start 18: stacktrace
|
18/86 Test #18: stacktrace ....................... Passed 0.10 sec
|
Start 19: crc32
|
19/86 Test #19: crc32 ............................ Passed 0.00 sec
|
Start 20: my_vsnprintf
|
20/86 Test #20: my_vsnprintf ..................... Passed 0.00 sec
|
Start 21: aes
|
21/86 Test #21: aes .............................. Passed 0.00 sec
|
Start 22: ma_dyncol
|
22/86 Test #22: ma_dyncol ........................ Passed 0.01 sec
|
Start 23: my_decimal
|
23/86 Test #23: my_decimal ....................... Passed 0.00 sec
|
Start 24: json_lib
|
24/86 Test #24: json_lib ......................... Passed 0.00 sec
|
Start 25: my_apc
|
25/86 Test #25: my_apc ........................... Passed 7.54 sec
|
Start 26: explain_filename
|
26/86 Test #26: explain_filename ................. Passed 0.05 sec
|
Start 27: mf_iocache
|
27/86 Test #27: mf_iocache ....................... Passed 0.03 sec
|
Start 28: conc336
|
28/86 Test #28: conc336 ..........................***Failed 0.01 sec
|
Start 29: bulk1
|
29/86 Test #29: bulk1 ............................***Failed 0.01 sec
|
Start 30: performance
|
30/86 Test #30: performance ......................***Failed 0.01 sec
|
Start 31: basic-t
|
31/86 Test #31: basic-t ..........................***Failed 0.01 sec
|
Start 32: fetch
|
32/86 Test #32: fetch ............................***Failed 0.01 sec
|
Start 33: charset
|
33/86 Test #33: charset ..........................***Failed 0.01 sec
|
Start 34: logs
|
34/86 Test #34: logs .............................***Failed 0.01 sec
|
Start 35: cursor
|
35/86 Test #35: cursor ...........................***Failed 0.01 sec
|
Start 36: errors
|
36/86 Test #36: errors ...........................***Failed 0.01 sec
|
Start 37: view
|
37/86 Test #37: view .............................***Failed 0.01 sec
|
Start 38: ps
|
38/86 Test #38: ps ...............................***Failed 0.01 sec
|
Start 39: ps_bugs
|
39/86 Test #39: ps_bugs ..........................***Failed 0.01 sec
|
Start 40: sp
|
40/86 Test #40: sp ...............................***Failed 0.01 sec
|
Start 41: result
|
41/86 Test #41: result ...........................***Failed 0.01 sec
|
Start 42: connection
|
42/86 Test #42: connection .......................***Failed 0.01 sec
|
Start 43: misc
|
43/86 Test #43: misc .............................***Failed 0.01 sec
|
Start 44: ps_new
|
44/86 Test #44: ps_new ...........................***Failed 0.01 sec
|
Start 45: thread
|
45/86 Test #45: thread ...........................***Failed 0.01 sec
|
Start 46: features-10_2
|
46/86 Test #46: features-10_2 ....................***Failed 0.01 sec
|
Start 47: async
|
47/86 Test #47: async ............................***Failed 0.01 sec
|
Start 48: innodb_sync
|
48/86 Test #48: innodb_sync ...................... Passed 1.43 sec
|
Start 49: ma_control_file
|
49/86 Test #49: ma_control_file .................. Passed 0.02 sec
|
Start 50: trnman
|
50/86 Test #50: trnman ........................... Passed 0.23 sec
|
Start 51: ma_test_loghandler
|
51/86 Test #51: ma_test_loghandler ............... Passed 0.50 sec
|
Start 52: ma_test_loghandler_multigroup
|
52/86 Test #52: ma_test_loghandler_multigroup .... Passed 0.01 sec
|
Start 53: ma_test_loghandler_multithread
|
53/86 Test #53: ma_test_loghandler_multithread ... Passed 0.48 sec
|
Start 54: ma_test_loghandler_pagecache
|
54/86 Test #54: ma_test_loghandler_pagecache ..... Passed 0.02 sec
|
Start 55: ma_test_loghandler_long
|
55/86 Test #55: ma_test_loghandler_long .......... Passed 0.17 sec
|
Start 56: ma_test_loghandler_noflush
|
56/86 Test #56: ma_test_loghandler_noflush ....... Passed 0.02 sec
|
Start 57: ma_test_loghandler_first_lsn
|
57/86 Test #57: ma_test_loghandler_first_lsn ..... Passed 0.03 sec
|
Start 58: ma_test_loghandler_max_lsn
|
58/86 Test #58: ma_test_loghandler_max_lsn ....... Passed 0.24 sec
|
Start 59: ma_test_loghandler_purge
|
59/86 Test #59: ma_test_loghandler_purge ......... Passed 0.20 sec
|
Start 60: ma_test_loghandler_readonly
|
60/86 Test #60: ma_test_loghandler_readonly ...... Passed 0.02 sec
|
Start 61: ma_test_loghandler_nologs
|
61/86 Test #61: ma_test_loghandler_nologs ........ Passed 0.20 sec
|
Start 62: ma_pagecache_single_1k
|
62/86 Test #62: ma_pagecache_single_1k ........... Passed 0.16 sec
|
Start 63: ma_pagecache_single_8k
|
63/86 Test #63: ma_pagecache_single_8k ........... Passed 0.01 sec
|
Start 64: ma_pagecache_single_64k
|
64/86 Test #64: ma_pagecache_single_64k .......... Passed 0.01 sec
|
Start 65: ma_pagecache_consist_1k
|
65/86 Test #65: ma_pagecache_consist_1k .......... Passed 0.01 sec
|
Start 66: ma_pagecache_consist_64k
|
66/86 Test #66: ma_pagecache_consist_64k ......... Passed 0.01 sec
|
Start 67: ma_pagecache_consist_1kHC
|
67/86 Test #67: ma_pagecache_consist_1kHC ........ Passed 0.01 sec
|
Start 68: ma_pagecache_consist_64kHC
|
68/86 Test #68: ma_pagecache_consist_64kHC ....... Passed 0.01 sec
|
Start 69: ma_pagecache_consist_1kRD
|
69/86 Test #69: ma_pagecache_consist_1kRD ........ Passed 0.28 sec
|
Start 70: ma_pagecache_consist_64kRD
|
70/86 Test #70: ma_pagecache_consist_64kRD ....... Passed 5.33 sec
|
Start 71: ma_pagecache_consist_1kWR
|
71/86 Test #71: ma_pagecache_consist_1kWR ........ Passed 0.52 sec
|
Start 72: ma_pagecache_consist_64kWR
|
72/86 Test #72: ma_pagecache_consist_64kWR ....... Passed 4.81 sec
|
Start 73: ma_pagecache_rwconsist_1k
|
73/86 Test #73: ma_pagecache_rwconsist_1k ........ Passed 0.00 sec
|
Start 74: ma_pagecache_rwconsist2_1k
|
74/86 Test #74: ma_pagecache_rwconsist2_1k ....... Passed 0.00 sec
|
Start 75: pfs_instr_class
|
75/86 Test #75: pfs_instr_class .................. Passed 0.01 sec
|
Start 76: pfs_instr_class-oom
|
76/86 Test #76: pfs_instr_class-oom .............. Passed 0.00 sec
|
Start 77: pfs_instr
|
77/86 Test #77: pfs_instr ........................ Passed 0.00 sec
|
Start 78: pfs_instr-oom
|
78/86 Test #78: pfs_instr-oom .................... Passed 0.01 sec
|
Start 79: pfs_account-oom
|
79/86 Test #79: pfs_account-oom .................. Passed 0.00 sec
|
Start 80: pfs_host-oom
|
80/86 Test #80: pfs_host-oom ..................... Passed 0.01 sec
|
Start 81: pfs_timer
|
81/86 Test #81: pfs_timer ........................ Passed 25.04 sec
|
Start 82: pfs_user-oom
|
82/86 Test #82: pfs_user-oom ..................... Passed 0.03 sec
|
Start 83: pfs_noop
|
83/86 Test #83: pfs_noop ......................... Passed 0.01 sec
|
Start 84: pfs
|
84/86 Test #84: pfs .............................. Passed 0.03 sec
|
Start 85: pfs_misc
|
85/86 Test #85: pfs_misc ......................... Passed 0.01 sec
|
Start 86: ed25519
|
86/86 Test #86: ed25519 .......................... Passed 0.01 sec
|
|
|
77% tests passed, 20 tests failed out of 86
|
|
|
Total Test time (real) = 48.89 sec
|
|
|
The following tests FAILED:
|
28 - conc336 (Failed)
|
29 - bulk1 (Failed)
|
30 - performance (Failed)
|
31 - basic-t (Failed)
|
32 - fetch (Failed)
|
33 - charset (Failed)
|
34 - logs (Failed)
|
35 - cursor (Failed)
|
36 - errors (Failed)
|
37 - view (Failed)
|
38 - ps (Failed)
|
39 - ps_bugs (Failed)
|
40 - sp (Failed)
|
41 - result (Failed)
|
42 - connection (Failed)
|
43 - misc (Failed)
|
44 - ps_new (Failed)
|
45 - thread (Failed)
|
46 - features-10_2 (Failed)
|
47 - async (Failed)
|
Errors while running CTest
|
make: *** [Makefile:129: test] Error 8
|
Attachments
Issue Links
- is duplicated by
-
-
- Closed
-
Activity
| Field | Original Value | New Value |
|---|---|---|
| Link |
This issue relates to |
| Link |
This issue is duplicated by |
| Link |
This issue relates to |
This is not a bug report about CUnit tests failing. This is a bug report about 'make tests' failing. Users expect by convetion that running commands like 'make', 'make build', 'make test', 'make install' etc work. And they do work when CMake generates the Makefile and CMake is correctly configured.
Either you decrease the scope of CUnit tests run in 'make test' so it actually does run them, or you extend 'make test' to start a server so that the full set of CUnit passes.
For the record, if built with `cmake -G Ninja` and when running `ninja test` cunit also fails. The mtr is starting/stopping the server, I hope there was some way to make cunit start the server as well so that the last 20 tests could properly run and cunit exit with success so that CI systems and new contributors running cunit would not see errors..
georg, what do you think about all C/C unit tests doing skip_all unless some MTR variable is set in the environment? Like
if (!getenv("MYSQLTEST_VARDIR")) { |
skip_all();
|
exit(0); |
}
|
Something like that?
| Assignee | Georg Richter [ georg ] |
| Fix Version/s | 10.2 [ 14601 ] | |
| Fix Version/s | 10.3 [ 22126 ] | |
| Fix Version/s | 10.4 [ 22408 ] | |
| Fix Version/s | 10.5 [ 23123 ] |
| Priority | Major [ 3 ] | Minor [ 4 ] |
serg What about skipping all tests if we can't connect via specified credentials?
georg, no, please, not that. You had this in the first version of unit tests, and they were silently skipped in mtr for quite a while and nobody noticed, buildbot was green. Tests should fail hard if they are expected to connect, but cannot.
Fixed in Connector/C:
Tests will be skipped unless the following environment variables
were specified:
- MYSQLTEST_VARDIR (will be automatically set when running inside
mtr) - MARIADB_CC_TEST
| Assignee | Georg Richter [ georg ] | Oleksandr Byelkin [ sanja ] |
IMHO it is enough to be sure that server is reachable and it does not matter if it is mysql-test-run or something else, i.e. I propose to just check all server related variables and do not make a new one.
| Assignee | Oleksandr Byelkin [ sanja ] | Georg Richter [ georg ] |
| Workflow | MariaDB v3 [ 122332 ] | MariaDB v4 [ 142893 ] |
| Fix Version/s | 10.2 [ 14601 ] |
| Fix Version/s | 10.3 [ 22126 ] |
| Fix Version/s | 10.4 [ 22408 ] |
| Status | Open [ 1 ] | Needs Feedback [ 10501 ] |
This is a duplicate of
MDEV-12470, the same answer applies. Unit tests of libmariadb need a running server, if you don't meet the precondition of the test, it will fail.