[MDEV-14503] Test mf_iocache fails Created: 2017-11-26  Updated: 2020-12-08  Resolved: 2018-03-08

Status: Closed
Project: MariaDB Server
Component/s: Tests
Affects Version/s: 10.2.10
Fix Version/s: 10.1.31, 10.2.13

Type: Bug Priority: Minor
Reporter: TBK Assignee: Sergei Golubchik
Resolution: Fixed Votes: 1
Labels: packaging, tests
Environment:

Alpine Linux: Edge - musl libc & libressl

Attachments: HTML File APKBUILD     Text File CTestCostData.txt     Text File LastTest.log     Text File LastTestsFailed.log     Text File build.txt    

 Description   

Hi MariaDB team,

I am trying to package version 10.2.10 for Alpine Linux.

As part of the build process "make test" is executed.

Unfortunately the following test fails:
22:mf_iocache

I have attached the build definition file (APKBUILD) and the logs.

 Comments   
Comment by Louis Dupré Bertoni [ 2018-01-28 ]

I have the same problem trying pass this test after compiling version 10.2.12 in an Alpine Linux Docker container.
Is this test very important or not ? Because if I can just ignore it tell me how to please.

Comment by Sergei Golubchik [ 2018-01-28 ]

Judging from the log, it looks like IO_CACHE cannot write to disk. If that's the case, then MariaDB will probably not work at all, or at least it'll break in many places.

I cannot see from the log why it cannot write to disk, though, it'd need some debugging...

Comment by Louis Dupré Bertoni [ 2018-01-28 ]

Hi
Here is the output log:

----------------------------------------------------------
 
 
 
24/62 Testing: mf_iocache
 
24/62 Test: mf_iocache
 
Command: "/tmp/build-mariadb/mariadb-10.2.12/build/unittest/sql/mf_iocache-t"
 
Directory: /tmp/build-mariadb/mariadb-10.2.12/build/unittest/sql
 
"mf_iocache" start time: Jan 28 13:51 UTC
 
Output:
 
----------------------------------------------------------
 
1..29
 
# temp io_cache with encryption
 
ok 1 - open_cached_file, pos_in_file = 0, pos_in_mem = 0
 
ok 2 - small write, pos_in_file = 0, pos_in_mem = 100
 
not ok 3 - large write, pos_in_file = 0, pos_in_mem = 16384
 
ok 4 - reinit with rewind, pos_in_file = 0, pos_in_mem = 250
 
not ok 5 - large write, pos_in_file = 0, pos_in_mem = 16384
 
not ok 6 - flush, pos_in_file = 0, pos_in_mem = 16384
 
ok 7 - reinit READ_CACHE, pos_in_file = 0, pos_in_mem = 0
 
not ok 8 - file must be unreadable
 
ok 9 - small read, pos_in_file = 0, pos_in_mem = 50
 
not ok 10 - large read, pos_in_file = 16384, pos_in_mem = 0
 
# temp io_cache without encryption
 
ok 11 - open_cached_file, pos_in_file = 0, pos_in_mem = 0
 
ok 12 - small write, pos_in_file = 0, pos_in_mem = 100
 
ok 13 - large write, pos_in_file = 16384, pos_in_mem = 300
 
ok 14 - reinit with rewind, pos_in_file = 250, pos_in_mem = 0
 
ok 15 - large write, pos_in_file = 16384, pos_in_mem = 450
 
ok 16 - flush, pos_in_file = 16834, pos_in_mem = 0
 
ok 17 - reinit READ_CACHE, pos_in_file = 0, pos_in_mem = 0
 
ok 18 - file must be readable
 
ok 19 - small read, pos_in_file = 0, pos_in_mem = 50
 
ok 20 - large read, pos_in_file = 16384, pos_in_mem = 250
 
# MDEV-9044 Binlog corruption in Galera
 
ok 21 - open_cached_file, pos_in_file = 0, pos_in_mem = 0
 
ok 22 - first write, pos_in_file = 0, pos_in_mem = 12
 
ok 23 - flush, pos_in_file = 12, pos_in_mem = 0
 
ok 24 - reinit WRITE_CACHE, pos_in_file = 0, pos_in_mem = 0
 
ok 25 - second write, pos_in_file = 0, pos_in_mem = 13
 
ok 26 - reinit READ_CACHE, pos_in_file = 0, pos_in_mem = 0
 
ok 27 - fill, pos_in_file = 0, pos_in_mem = 13
 
ok 28 - reinit READ_CACHE, pos_in_file = 0, pos_in_mem = 0
 
ok 29 - read 'second write'
 
# Failed 5 tests!
 
<end of output>
 
Test time =   0.01 sec
 
----------------------------------------------------------
 
Test Failed."mf_iocache" end time: Jan 28 13:51 UTC
 
"mf_iocache" time elapsed: 00:00:00
 
----------------------------------------------------------

Comment by Sergei Golubchik [ 2018-01-28 ]

Yes, I've seen that already. It matches the attached LastTest.log file. Which means you have exactly the same issue.

Comment by Louis Dupré Bertoni [ 2018-01-28 ]

Could that be due to Docker or the overlay file-system? Or even the host machine which is a VM?

Comment by Sergei Golubchik [ 2018-01-28 ]

I don't know, may be. Could you strace the test? Like

strace -o strace.log unittest/sql/mf_iocache-t

and then attach the strace.log file?

Comment by Louis Dupré Bertoni [ 2018-01-28 ]

I get this:
strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted

I think this has to do with a Docker security feature and it could be the same reason for what the test fails.

Comment by Sergei Golubchik [ 2018-01-28 ]

No, I doubt it's the same reason. As for the strace, I've just found this blog: http://johntellsall.blogspot.de/2016/10/tip-use-strace-to-debug-issues-inside.html

Looks like --security-opt=seccomp:unconfined is the important part.
Even if it won't help, there were many more search results, I just didn't study them all. This seems to be a known issue with known solutions.

Comment by Louis Dupré Bertoni [ 2018-01-28 ]

Ok initially this is what I got with strace:

open("/usr/local/lib/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
 
open("/etc/ld-musl-x86_64.path", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
 
open("/lib/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
 
open("/usr/local/lib/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
 
open("/usr/local/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
 
open("/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
 
open("/usr/local/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

The missing libraries where in /usr/lib so I symlinked them in /usr/local/lib, I tried telling cmake to look into the right directory without success.
Now I have the following:

https://framabin.org/?6e3247917302d98b#b7zDdtOhh1xJ8/EoRid7AhuO/5JUkjtyRpJbNwlIacI=

I don't see any errors anymore, what is the problem?

Comment by Sergei Golubchik [ 2018-01-28 ]

Thanks. For the reference, I'll quote the important part here:

writev(1, [{iov_base="ok 2 - ", iov_len=7}, {iov_base=NULL, iov_len=0}], 2) = 7
 
writev(1, [{iov_base="small write, pos_in_file = 0, po"..., iov_len=46}, {iov_base=NULL, iov_len=0}], 2) = 46
 
writev(1, [{iov_base="", iov_len=0}, {iov_base="\n", iov_len=1}], 2) = 1
 
open("/tmp/tmp.emnCon", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
 
unlink("/tmp/tmp.emnCon")               = 0
 
writev(1, [{iov_base="not ok 3 - ", iov_len=11}, {iov_base=NULL, iov_len=0}], 2) = 11

This is how it looks for me, when the test is working:

write(1, "ok 2 - ", 7)                  = 7
 
write(1, "small write, pos_in_file = 0, po"..., 46) = 46
 
write(1, "\n", 1)                       = 1
 
getpid()                                = 16421
 
open("/tmp/tmp.sk3M8n", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
 
unlink("/tmp/tmp.sk3M8n")               = 0
 
getpid()                                = 16421
 
getpid()                                = 16421
 
open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 4
 
fstat(4, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
 
poll([{fd=4, events=POLLIN}], 1, 10)    = 1 ([{fd=4, revents=POLLIN}])
 
read(4, "\200\301\354g(\26\370C\34\235\201\272\277\203\\g\317\251\204\254'\300\207R\255\243\230\276~\f\244j", 32) = 32
 
close(4)                                = 0
 
getuid()                                = 1000
 
write(3, "\0\0\0\0\0\0\0\0\1a\251\r\210\2\22#\236%\305\323E\36W\fX\337\21B\310D\200>"..., 16408) = 16408
 
write(1, "ok 3 - ", 7)                  = 7
 
write(1, "large write, pos_in_file = 16384"..., 50) = 50

Comment by Sergei Golubchik [ 2018-01-28 ]

As far as I understand, this part with getpid and open("/dev/urandom") comes from this code:

  RAND_METHOD *rand = RAND_OpenSSL();
 
  if (rand == NULL || rand->bytes(buf, num) != 1)
 
    return MY_AES_OPENSSL_ERROR;
 
  return MY_AES_OK;

I suspect that it fails for you (either because of Docker or, more probably, because of musl or something else specific to Alpine) and it makes the whole unit test to fail.

What OpenSSL version do you have?

This issue might be related to MDEV-14343.

Could you try to build MariaDB with yassl? It's -DWITH_SSL=bundled option to cmake. Even if you don't want to run with yassl, it'll help to understand whether this OpenSSL code above is to blame.

Comment by Louis Dupré Bertoni [ 2018-01-28 ]

I am compiling with LibreSSL and this is apparently the cause of the issue since I just tried with GNUTLS and the test passed.
Now I want to now why LibreSSL is causing problems. Is there anything I can do to debug? Because strace does not provides much information or at least I don't know how to use it.

Comment by Sergei Golubchik [ 2018-01-28 ]

Don't bother. The issue will probably disappear in the next release when MDEV-14343 is fixed.
See https://jira.mariadb.org/ for the release schedule.

But it'd be good if you confirm here whether the issue is fixed or not, after trying the next release...

Generated at Thu Feb 08 08:14:05 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.