Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20377

Make WITH_MSAN more usable

    XMLWordPrintable

Details

    Description

      MemorySanitizer is a compile-time instrumentation layer in clang and GCC. Together with AddressSanitizer mostly makes the run-time instrumentation of Valgrind redundant. It is a little more tricky to set up, because running with uninstrumented libraries will lead into false positives.

      No patches are necessary since 10.5 94d0bb4dbeb28a94d1f87fdd55f4297ff3df0157 (see the commit message for instructions); cmake -DWITH_MSAN=ON is supposed to work ‘out of the box’.

      This task will be kept open until all tests pass and MemorySanitizer can be enabled on CI systems.

      How to instrumented libraries

      These instructions are for clang-10. The script build-msan2.sh was developed to resolve MDEV-22083 a.k.a. MDEV-26758.

      mkdir /tmp/build
      cd /tmp/build
      mkdir "$HOME/msan-libs"
      CLANG=10 MSAN_LIBDIR="$HOME/msan-libs" build-msan.sh
      

      Note: to use different clang (tested with clang-8, clang-9, clang-11, clang-13), just replace 10 with the major version of the compiler above.

      For clang-15, the procedure is a little different:

      mkdir /tmp/build
      cd /tmp/build
      mkdir "$HOME/msan-libs"
      CLANG=15 MSAN_LIBDIR="$HOME/msan-libs" build-msan15.sh
      

      How to build MariaDB Server 10.5 or later with the instrumented libraries

      cd /mariadb/10.5
      mkdir build
      cd build
      cmake -DCMAKE_{C_COMPILER=clang,CXX_COMPILER=clang++}-10 \
      -DCMAKE_C_FLAGS='-O2 -Wno-unused-command-line-argument -fdebug-macro' \
      -DCMAKE_CXX_FLAGS='-stdlib=libc++ -O2 -Wno-unused-command-line-argument -fdebug-macro'  \
      -DWITH_EMBEDDED_SERVER=OFF -DWITH_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=Debug \
      -DWITH_INNODB_{BZIP2,LZ4,LZMA,LZO,SNAPPY}=OFF \
      -DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,CONNECT,SPIDER}=NO -DWITH_SAFEMALLOC=OFF \
      -DWITH_{ZLIB,SSL,PCRE}=bundled \
      -DHAVE_LIBAIO_H=0 -DCMAKE_DISABLE_FIND_PACKAGE_{URING,LIBAIO}=1 \
      -DWITH_MSAN=ON \
      -G Ninja ..
      ninja
      

      Note: -march=native -mtune=native is optional since the second fix of MDEV-20386

      How to run tests

      cd mysql-test
      LD_LIBRARY_PATH="$HOME"/msan-libs ./mtr main.1st
      LD_LIBRARY_PATH="$HOME"/msan-libs MSAN_OPTIONS=abort_on_error=1:poison_in_dtor=1 ./mtr --big-test --parallel=auto --force --retry=0
      

      Note: It may be wise to omit MSAN_OPTIONS=abort_on_error=1 except when running code in a debugger. On some occasions, it may cause truncation of the diagnostic messages.

      Note: The llvm-symbolizer in clang 14 or later will refuse to load if LD_LIBRARY_PATH includes an MSAN-instrumented libgmp.so. To get nice resolved stack traces, you must point the environment variable MSAN_SYMBOLIZER_PATH to a script like the following:

      #!/bin/sh
      unset LD_LIBRARY_PATH
      exec llvm-symbolizer-15 "$@"
      

      The poison_in_dtor=1 (MDEV-30936, MDEV-30942) is an old option that was enabled by default in clang 15.

      Attachments

        1. 10.5-msan.patch
          3 kB
        2. build-msan.sh
          1 kB
        3. build-msan15.sh
          3 kB
        4. build-msan16.sh
          3 kB
        5. build-msan18.sh
          3 kB
        6. build-msan2.sh
          3 kB

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.