Details
-
Task
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
None
Description
As noted in MDEV-20377, MemorySanitizer (cmake -DWITH_MSAN=ON) requires that all libraries be instrumented, with the possible exception of libc. Currently, libmariadb insists on linking with GNUTLS instead of WolfSSL, which we can easily build ourselves by specifying -DWITH_SSL=bundled.
An instrumented library is a must-have for WITH_MSAN builds, and it would also improve the coverage of other sanitizers (WITH_ASAN, WITH_UBSAN, WITH_TSAN).
Because libmariadb is normally licensed under the GNU LGPL version 2, which WolfSSL is believed to be incompatible with, we could consider introducing some configuration parameter, similar to the existing cmake flag NOT_FOR_DISTRIBUTION. That flag is already being used in cmake/readline.cmake and cmake/aws_sdk.cmake. However, its purpose appears to allow something GPLv2 incompatible to be built, while here the result should still be GPLv2 compatible while being LGPLv2 incompatible. Perhaps the option should be called NOT_LGPL or similar, and it should apply to Connector/C only, not to the MariaDB server?
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
MDEV-21835 Implement option 'system-wolfssl' in build flag WITH_SSL
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
Activity
serg, having fully instrumented libraries would also benefit other sanitized builds, such as ASAN, UBSAN, TSAN. MemorySanitizer differs from the rest in that it assumes that uninstrumented libraries are doing bad things, while AddressSanitizer assumes the opposite. For example, if we are passing a out-of-bounds pointer to an encryption routine with the current libmariadb, that error would not be caught.
Building GNUTLS feels like a highly nontrivial exercise, with lots of dependencies, using ./configure to build. I do not even know if it all is supposed to build with clang.
Just a note that using WolfSSL in libmariadb would not make the library GPL as we have an exception for WolfSSL to use it as LGPL with the clients and the server.
Even if we would NOT have an exception, supporting WolfSSL in the libmariadb and use it for our own internal buildbot testing would not make the libmariadb GPL was we would not distribute these builds to outsiders.
I tried to compile an MSAN-instrumented GNUTLS as follows, based on what serg had attempted:
cd /mariadb/gnutls28-3.6.12
|
CC=clang-10 CXX=clang++-10 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --with-included-libtasn1 --with-included-unistring --without-p11-kit && make -j$(nproc)
|
For me, it would fail due to missing makeinfo when generating documentation, but not before producing the desired output in lib/.libs.
With this, I tried to invoke a few known-failing tests mentioned in MDEV-20377:
MSAN_OPTIONS=abort_on_error=1 LD_LIBRARY_PATH=/mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib:/mariadb/gnutls28-3.6.12/lib/.libs ./mtr --force --parallel=auto --max-test-fail=0 main.mysql_client_test_nonblock main.mysql_client_test main.mysql_client_test_comp main.tls_version main.ssl_8k_key main.flush_ssl perfschema.hostcache_ipv6_ssl main.ssl_ca main.userstat main.mysql_upgrade_ssl main.ssl_7937 main.openssl_1 main.ssl_connect main.ssl_timeout-9836 main.ssl main.ssl_cipher main.ssl-big main.ssl_compress perfschema.connection_type_notwin main.ssl_timeout perfschema.hostcache_ipv4_ssl
|
For most tests, I am seeing something like the following:
|
10.5 05fa4558e0e82302ece981deabce764491464eb2 |
CURRENT_TEST: perfschema.connection_type_notwin
|
==625405==WARNING: MemorySanitizer: use-of-uninitialized-value
|
#0 0x7f98883bb98f in asn1_get_tag_der /mariadb/gnutls28-3.6.12/lib/minitasn1/decoding.c:181:7
|
#1 0x7f98883bb98f in _asn1_extract_tag_der /mariadb/gnutls28-3.6.12/lib/minitasn1/decoding.c:629:11
|
#2 0x7f98883b51ab in extract_tag_der_recursive /mariadb/gnutls28-3.6.12/lib/minitasn1/decoding.c:716:12
|
#3 0x7f98883b51ab in asn1_der_decoding2 /mariadb/gnutls28-3.6.12/lib/minitasn1/decoding.c:1045:8
|
#4 0x7f98882b6735 in _asn1_strict_der_decode /mariadb/gnutls28-3.6.12/lib/x509/./common.h:288:9
|
#5 0x7f98882b6735 in gnutls_x509_crt_import /mariadb/gnutls28-3.6.12/lib/x509/x509.c:606:6
|
#6 0x7f98882c71dc in gnutls_x509_crt_list_import /mariadb/gnutls28-3.6.12/lib/x509/x509.c:3847:8
|
#7 0x7f98882c6bdf in gnutls_x509_crt_list_import2 /mariadb/gnutls28-3.6.12/lib/x509/x509.c:3727:6
|
#8 0x7f98882e64b0 in gnutls_x509_trust_list_add_trust_mem /mariadb/gnutls28-3.6.12/lib/x509/verify-high2.c:93:7
|
#9 0x7f98882e6f90 in gnutls_x509_trust_list_add_trust_file /mariadb/gnutls28-3.6.12/lib/x509/verify-high2.c:378:6
|
#10 0x7f988818f150 in gnutls_certificate_set_x509_trust_file /mariadb/gnutls28-3.6.12/lib/cert-cred-x509.c:1209:8
|
#11 0x5652ef1a0425 in ma_tls_set_certs /mariadb/10.5m/libmariadb/libmariadb/secure/gnutls.c:1077:16
|
#12 0x5652ef1a0425 in ma_tls_init /mariadb/10.5m/libmariadb/libmariadb/secure/gnutls.c:1136:19
|
#13 0x5652ef155e53 in ma_pvio_tls_init /mariadb/10.5m/libmariadb/libmariadb/ma_tls.c:71:20
|
#14 0x5652ef1554e3 in ma_pvio_start_ssl /mariadb/10.5m/libmariadb/libmariadb/ma_pvio.c:527:21
|
#15 0x5652ef1c29b2 in send_client_reply_packet /mariadb/10.5m/libmariadb/plugins/auth/my_auth.c:309:9
|
#16 0x5652ef1c29b2 in client_mpvio_write_packet /mariadb/10.5m/libmariadb/plugins/auth/my_auth.c:451:12
|
#17 0x5652ef1be783 in native_password_auth_client /mariadb/10.5m/libmariadb/plugins/auth/my_auth.c:92:9
|
#18 0x5652ef1bfe3f in run_plugin_auth /mariadb/10.5m/libmariadb/plugins/auth/my_auth.c:599:8
|
#19 0x5652ef13b44a in mthd_my_real_connect /mariadb/10.5m/libmariadb/libmariadb/mariadb_lib.c:1646:7
|
#20 0x5652ef138211 in mysql_real_connect /mariadb/10.5m/libmariadb/libmariadb/mariadb_lib.c:1295:10
|
#21 0x5652ef0e10ab in wrap_mysql_real_connect(st_mysql*, char const*, char const*, char const*, char const*, unsigned int, char const*, unsigned long) /mariadb/10.5m/client/../tests/nonblock-wrappers.h:165:1
|
#22 0x5652ef0e10ab in connect_n_handle_errors(st_command*, st_mysql*, char const*, char const*, char const*, char const*, int, char const*) /mariadb/10.5m/client/mysqltest.cc:5754:11
|
#23 0x5652ef0e3c41 in do_connect(st_command*) /mariadb/10.5m/client/mysqltest.cc:6072:7
|
#24 0x5652ef106103 in main /mariadb/10.5m/client/mysqltest.cc:9369:9
|
#25 0x7f9887ba9cc9 in __libc_start_main csu/../csu/libc-start.c:308:16
|
#26 0x5652ef02bac9 in _start (/dev/shm/10.5msan/client/mariadb-test+0x82ac9)
|
|
|
SUMMARY: MemorySanitizer: use-of-uninitialized-value /mariadb/gnutls28-3.6.12/lib/minitasn1/decoding.c:181:7 in asn1_get_tag_der
|
ORIGIN: invalid (0). Might be a bug in MemorySanitizer origin tracking.
|
I have never before seen the ORIGIN: invalid (0). Maybe some more dependencies of gnutls should be built with instrumentation?
Of the above tests, only main.ssl_7937,nossl passes for me and 22 tests fail.
A few uninstrumented dependencies are being listed:
LD_LIBRARY_PATH=/mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib:/mariadb/gnutls28-3.6.12/lib/.libs ldd ../client/mysqltest
|
|
ldd output for mysqltest |
linux-vdso.so.1 (0x00007fff345a0000)
|
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f66a8e5e000)
|
libgnutls.so.30 => /mariadb/gnutls28-3.6.12/lib/.libs/libgnutls.so.30 (0x00007f66a8a66000)
|
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f66a8922000)
|
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f66a891c000)
|
libc++.so.1 => /mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib/libc++.so.1 (0x00007f66a8769000)
|
libc++abi.so.1 => /usr/lib/x86_64-linux-gnu/libc++abi.so.1 (0x00007f66a8731000)
|
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f66a8724000)
|
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f66a870a000)
|
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f66a8545000)
|
/lib64/ld-linux-x86-64.so.2 (0x00007f66ab8d9000)
|
libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f66a8524000)
|
libnettle.so.8 => /usr/lib/x86_64-linux-gnu/libnettle.so.8 (0x00007f66a84e4000)
|
libhogweed.so.6 => /usr/lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007f66a849b000)
|
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f66a8416000)
|
libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f66a8294000)
|
The libgnutls.so contains many uninstrumented dependencies:
ldd /mariadb/gnutls28-3.6.12/lib/.libs/libgnutls.so.30
|
|
ldd output for libgnutls.so.30 |
linux-vdso.so.1 (0x00007ffc32bf8000)
|
libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f7828768000)
|
libnettle.so.8 => /usr/lib/x86_64-linux-gnu/libnettle.so.8 (0x00007f7828728000)
|
libhogweed.so.6 => /usr/lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007f78286df000)
|
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f782865c000)
|
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7828497000)
|
/lib64/ld-linux-x86-64.so.2 (0x00007f7828ba8000)
|
libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f7828315000)
|
serg, I suspect that we would need MSAN-instrumented builds of all of libidn2, libgmp, libnettle, libhogweed, libunistring2 as well.
Could it actually be less effort to implement WolfSSL support in libmariadb than to maintain so many MSAN builds of external libraries?
Again, I think it's a totally wrong approach to add a support for a yet another quite different SSL library to C/C, something we cannot even legally distribute, so do it only for MSAN, and MSAN-test this fake chimera of a build that nobody will ever see or use.
The correct fix is to build C/C as we ship it, with the libraries that bintar uses (yes, that means GnuTLS). And compile all that for MSAN.
Due to a request by elenst, I looked at this again. First, I was thinking if we could simply add some calls to unpoison memory that we get from libgnutls28. But, the issue seems to be that within that library, memory is being allocated and initialized. Intercepted functions such as strcmp() would then complain about that uninstrumented memory.
So, I compiled the latest libgnutls28 again:
apt source libgnutls28-dev
|
cd libgnutls28-3.7.2
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --with-included-libtasn1 --with-included-unistring --without-p11-kit
|
automake --add-missing
|
make -j$(nproc)
|
I got an rr replay trace of a failure as follows:
MYSQL_TEST=/dev/shm/rr-mysqltest _RR_TRACE_DIR=/dev/shm \
|
MSAN_OPTIONS=abort_on_error=1 \
|
LD_LIBRARY_PATH=/mariadb/llvm-toolchain-11-11.0.0/libc++msan/lib:/mariadb/gnutls28-3.7.2/lib/.libs \
|
./mtr main.userstat
|
This was using a simple wrapper script that I wrote:
#!/bin/sh
|
exec rr record /dev/shm/10.5msan/client/mysqltest "$@"
|
Based on rr replay /dev/shm/latest-trace, I can see that the problem is an uninstrumented nettle_base64_decode_update() in libnettle8. Fetching that next:
apt source libnettle8
|
cd nettle-3.7.3
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --with-included-libtasn1 --with-included-unistring --without-p11-kit && make -j$(nproc)
|
ln -s libnettle.so libnettle.so.8
|
With also nettle-3.7.3 added to LD_LIBRARY_PATH, we get a little further:
CURRENT_TEST: main.userstat
|
|
|
|
|
Could not execute 'check-testcase' before testcase 'main.userstat' (res: 1):
|
Uninitialized bytes in MemcmpInterceptorCommon at offset 0 inside [0x7fff17e10234, 12)
|
==863624==WARNING: MemorySanitizer: use-of-uninitialized-value
|
#0 0x56103b6e4d0e in bcmp (/dev/shm/10.5msan/client/mariadb-test+0x92d0e)
|
#1 0x7f17289dfcf0 in get_x86_features /mariadb/nettle-3.7.3/fat-x86_64.c:99:11
|
…
|
Now, we will have to instrument a little:
--- nettle-3.7.3/fat-x86_64.c 2021-06-06 23:06:29.000000000 +0300
|
+++ nettle-3.7.3/fat-x86_64.c 2021-10-04 14:45:18.937166872 +0300
|
@@ -61,6 +61,10 @@
|
#define MATCH(s, slen, literal, llen) \
|
((slen) == (llen) && memcmp ((s), (literal), llen) == 0)
|
|
+#if __has_feature(memory_sanitizer)
|
+# include <sanitizer/msan_interface.h>
|
+#endif
|
+
|
static void
|
get_x86_features (struct x86_features *features)
|
{
|
@@ -96,16 +100,25 @@
|
{
|
uint32_t cpuid_data[4];
|
_nettle_cpuid (0, cpuid_data);
|
+#if __has_feature(memory_sanitizer)
|
+ __msan_unpoison(cpuid_data, sizeof cpuid_data);
|
+#endif
|
if (memcmp (cpuid_data + 1, "Genu" "ntel" "ineI", 12) == 0)
|
features->vendor = X86_INTEL;
|
else if (memcmp (cpuid_data + 1, "Auth" "cAMD" "enti", 12) == 0)
|
features->vendor = X86_AMD;
|
|
_nettle_cpuid (1, cpuid_data);
|
+#if __has_feature(memory_sanitizer)
|
+ __msan_unpoison(cpuid_data, sizeof cpuid_data);
|
+#endif
|
if (cpuid_data[2] & 0x02000000)
|
features->have_aesni = 1;
|
|
_nettle_cpuid (7, cpuid_data);
|
+#if __has_feature(memory_sanitizer)
|
+ __msan_unpoison(cpuid_data, sizeof cpuid_data);
|
+#endif
|
if (cpuid_data[1] & 0x20000000)
|
features->have_sha_ni = 1;
|
} |
That is not yet enough:
CURRENT_TEST: main.userstat
|
==865015==WARNING: MemorySanitizer: use-of-uninitialized-value
|
#0 0x7f3b982bc7f7 in wrap_nettle_pk_fixup /mariadb/gnutls28-3.7.2/lib/nettle/pk.c:3410:7
|
#1 0x7f3b98158136 in _gnutls_pk_fixup /mariadb/gnutls28-3.7.2/lib/x509/./../pk.h:49:10
|
…
|
It looks like the next problem might be in libgnutls28 again, or perhaps in one of the following not being instrumented yet:
libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f4c5bbc1000)
|
libhogweed.so.6 => /usr/lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007f4c5baf7000)
|
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f4c5ba74000)
|
libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f4c5b8e7000)
|
It turns out that libhogweed.so.6 is part of nettle-3.7.3, in the .libs subdirectory.
apt source libidn2
|
cd libidn2-2.3.2
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --enable-valgrind-tests=no
|
make -j$(nproc)
|
cd ..
|
apt source libgmp10
|
cd gmp-6.2.1+dfsg
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory"
|
make -j$(nproc)
|
cd ..
|
Now, with all libraries instrumented (except for linux-vdso, libc, ld-linux), we get a little more detail for the previous failure:
MYSQL_TEST=/dev/shm/rr-mysqltest _RR_TRACE_DIR=/dev/shm \
|
MSAN_OPTIONS=abort_on_error=1 \
|
LD_LIBRARY_PATH=/mariadb/llvm-toolchain-11-11.0.0/libc++msan/lib:/mariadb/gnutls28-3.7.2/lib/.libs:/mariadb/nettle-3.7.3/.lib:/mariadb/gmp-6.2.1+dfsg/.libs:/mariadb/libidn2-2.3.2/lib/.libs ./mtr main.userstat
|
CURRENT_TEST: main.userstat
|
==896435==WARNING: MemorySanitizer: use-of-uninitialized-value
|
#0 0x7f0d0acbce36 in __gmpn_gcdext_lehmer_n /mariadb/gmp-6.2.1+dfsg/mpn/gcdext_lehmer.c:194:11
|
#1 0x7f0d0acb7385 in __gmpn_gcdext /mariadb/gmp-6.2.1+dfsg/mpn/gcdext.c:276:22
|
#2 0x7f0d0ac62cd0 in __gmpz_gcdext /mariadb/gmp-6.2.1+dfsg/mpz/gcdext.c:93:11
|
#3 0x7f0d0ac66d1a in __gmpz_invert /mariadb/gmp-6.2.1+dfsg/mpz/invert.c:50:3
|
#4 0x7f0d0b73da31 in wrap_nettle_pk_fixup /mariadb/gnutls28-3.7.2/lib/nettle/pk.c:3413:7
|
The next culprit is 2 more pieces of assembler code. Let us try rebuilding:
cd gmp-6.2.1+dfsg
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --disable-assembly
|
make clean
|
make -j$(nproc)
|
cd ..
|
cd nettle-3.7.3
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --disable-assembler --disable-fat
|
make clean
|
make -j$(nproc)
|
cd ..
|
Note the inconsistent spelling: assembly and assembler. And thanks to disabling "fat", we will not have to modify nettle-3.7.3/fat-x86_64.c anymore. Unfortunately, the resulting library is incomplete:
/usr/bin/ld: ../libnettle.so: undefined reference to `nettle_memxor'
|
/usr/bin/ld: ../libnettle.so: undefined reference to `_nettle_sha256_compress'
|
/usr/bin/ld: ../libhogweed.so: undefined reference to `nettle_memxor'
|
/usr/bin/ld: ../libnettle.so: undefined reference to `_nettle_aes_encrypt'
|
/usr/bin/ld: ../libnettle.so: undefined reference to `nettle_sha1_compress'
|
/usr/bin/ld: ../libnettle.so: undefined reference to `_nettle_sha256_compress'
|
/usr/bin/ld: ../libnettle.so: undefined reference to `_nettle_aes_decrypt'
|
The following patch made the trick:
--- config.h.in 2021-06-06 23:06:29.000000000 +0300
|
+++ config.h.in 2021-10-04 16:09:18.636629855 +0300
|
@@ -45,9 +45,6 @@
|
/* Define to 1 if you have the `gmp' library (-lgmp). */
|
#undef HAVE_LIBGMP
|
|
-/* Define if compiler and linker supports __attribute__ ifunc */
|
-#undef HAVE_LINK_IFUNC
|
-
|
/* Define to 1 if you have the <malloc.h> header file. */
|
#undef HAVE_MALLOC_H
|
|
|
And we got the next error:
CURRENT_TEST: main.userstat
|
==929302==WARNING: MemorySanitizer: use-of-uninitialized-value
|
#0 0x7fb162bd545a in decrypt_packet_tls13 /mariadb/gnutls28-3.7.2/lib/cipher.c:899:27
|
#1 0x7fb162bd545a in _gnutls_decrypt /mariadb/gnutls28-3.7.2/lib/cipher.c:157:7
|
It looks like we have to rebuild gnutls28 once more:
cd gnutls28-3.7.2
|
CC=clang-11 CXX=clang++-11 ./configure C{,XX}FLAGS="-fno-omit-frame-pointer -O2 -g -fsanitize=memory" LDFLAGS="-fsanitize=memory" --with-included-libtasn1 --with-included-unistring --without-p11-kit --disable-hardware-acceleration
|
automake --add-missing
|
make clean
|
make -j$(nproc)
|
..
|
And that was it, for this test:
main.userstat 'innodb' [ pass ] 1345
|
Some genuine errors were caught in mysql_client_test.c. This broke the tests main.mysql_client_test_nonblock main.mysql_client_test main.mysql_client_test_comp.
We still have the following failures:
|
10.5 ead38354e60e0fb241de8abe0ed6a57c14dfb820 |
main.truncate_notembedded w3 [ fail ]
|
Test ended at 2021-10-04 19:52:02
|
|
|
CURRENT_TEST: main.truncate_notembedded
|
mysqltest: At line 21: query 'reap' failed: 1290: The MariaDB server is running with the --max-thread-mem-used=45500 option so it cannot execute this statement
|
|
|
The result from queries just before the failure was:
|
#
|
# MDEV-23365: Assertion `!is_set() || (m_status == DA_OK_BULK &&
|
# is_bulk_op())' failed upon killed TRUNCATE
|
#
|
CREATE TABLE t1 (a INT) ENGINE=MyISAM;
|
LOCK TABLE t1 READ;
|
connect con1,localhost,root,,test;
|
SET SESSION max_session_mem_used= 45500;
|
LOCK TABLE t1 WRITE;cause
|
connection default;
|
SELECT * FROM t1;
|
a
|
UNLOCK TABLES;
|
connection con1;
|
|
|
- saving '/dev/shm/10.5msan/mysql-test/var/3/log/main.truncate_notembedded/' to '/dev/shm/10.5msan/mysql-test/var/log/main.truncate_notembedded/'
|
main.sp w2 [ fail ]
|
Test ended at 2021-10-04 19:52:15
|
|
|
CURRENT_TEST: main.sp
|
/dev/shm/10.5msan/client/mysqltest: Error on delete of '/dev/shm/10.5msan/mysql-test/var/2/tmp//t1.frm' (Errcode: 2 "No such file or directory")
|
/dev/shm/10.5msan/client/mysqltest: Error on delete of '/dev/shm/10.5msan/mysql-test/var/2/tmp//t1.MYD' (Errcode: 2 "No such file or directory")
|
--- /mariadb/10.5m/mysql-test/main/sp.result 2021-10-04 12:13:51.991511056 +0300
|
+++ /mariadb/10.5m/mysql-test/main/sp.reject 2021-10-04 19:52:15.586545147 +0300
|
@@ -8889,7 +8889,7 @@
|
SELECT 1 latin1 latin1_swedish_ci latin1_swedish_ci
|
SELECT VARIABLE_VALUE-@local_mem_used FROM INFORMATION_SCHEMA.SESSION_STATUS WHERE VARIABLE_NAME='MEMORY_USED';
|
VARIABLE_VALUE-@local_mem_used
|
-0
|
+640
|
DROP PROCEDURE sp1;
|
# End of 10.3 tests
|
#
|
|
|
mysqltest: Result length mismatch
|
|
|
- saving '/dev/shm/10.5msan/mysql-test/var/2/log/main.sp/' to '/dev/shm/10.5msan/mysql-test/var/log/main.sp/'
|
Furthermore, the test innodb.alter_large_dml tends to trigger DEBUG_SYNC timeout.
Here is proof that the test innodb.alter_large_dml sometimes passes with MemorySanitizer. This run was only with --parallel=3. Normally, with much more concurrent tests, the test would almost always timeout.
|
10.5 ead38354e60e0fb241de8abe0ed6a57c14dfb820 |
innodb.alter_large_dml 'innodb' w1 [ pass ] 209421
|
The server error log would contain the following:
2021-10-04 19:52:01 0 [Note] InnoDB: 10.5.13 started; log sequence number 47822; transaction id 27
|
…
|
2021-10-04 19:55:32 0 [Note] InnoDB: Shutdown completed; log sequence number 785123071; transaction id 57
|
2021-10-04 19:55:32 0 [Note] Debug sync points hit: 2568158
|
2021-10-04 19:55:32 0 [Note] Debug sync points executed: 8
|
2021-10-04 19:55:32 0 [Note] Debug sync points max active per thread: 3
|
2021-10-04 19:55:32 0 [Note] /dev/shm/10.5msan/sql/mariadbd: Shutdown complete
|
I attached a script build-msan.sh in MDEV-20377 for building all dependencies using the specified version of clang. I tested it on clang version 13 today. Previously I was using version 11 for MemorySanitizer builds.
Starting with clang-14, llvm-symbolizer depends on libgmp. Therefore, it will fail to work if LD_LIBRARY_PATH points to something that includes an MSAN instrumented libgmp. As noted in MDEV-30936, this can be resolved by making the environment variable MSAN_SYMBOLIZER_PATH point to a simple wrapper script:
#!/bin/sh
|
unset LD_LIBRARY_PATH
|
exec llvm-symbolizer-15 "$@"
|
For clang-15, there is another script build-msan15.sh attached to MDEV-20377.
I suspect it'll be much easier to build gnutls or openssl with instrumentation that add a totally new wolfssl support C/C only for the purpose of running MSAN