Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20377

Make WITH_MSAN more usable

    XMLWordPrintable

    Details

      Description

      MemorySanitizer is a compile-time instrumentation layer in clang and GCC. Together with AddressSanitizer mostly makes the run-time instrumentation of Valgrind redundant. It is a little more tricky to set up, because running with uninstrumented libraries will lead into false positives.

      No patches are necessary since 10.5 94d0bb4dbeb28a94d1f87fdd55f4297ff3df0157 (see the commit message for instructions); cmake -DWITH_MSAN=ON is supposed to work ‘out of the box’.

      This task will be kept open until all tests pass and MemorySanitizer can be enabled on CI systems.

      How to build an instrumented C++ runtime library

      These instructions are for the currently latest compiler, clang-10.

      sudo apt install clang-10 clang++-10 libc++-10-dev libc++abi-10-dev
      		 
      cd /mariadb
      		 
      apt source libc++-10-dev
      		 
      cd llvm-toolchain-10-10.0.0
      		 
      mkdir libc++msan
      		 
      cd libc++msan
      		 
      cmake ../libcxx -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=Memory -DCMAKE_C_COMPILER=clang-10 -DCMAKE_CXX_COMPILER=clang++-10
      		 
      make -j$(nproc)

      Note: to use an older clang (I have successfully used clang-8 and clang-9), just replace 10 with the major version of the compiler above.

      How to build MariaDB Server 10.5 with the instrumented libraries

      cd /mariadb/10.5
      		 
      mkdir build
      		 
      cd build
      		 
      cmake -DCMAKE_{C_COMPILER=clang,CXX_COMPILER=clang++}-10 \
      		 
      -DCMAKE_C_FLAGS='-O2 -Wno-unused-command-line-argument -fdebug-macro' \
      		 
      -DCMAKE_CXX_FLAGS='-stdlib=libc++ -O2 -Wno-unused-command-line-argument -fdebug-macro'  \
      		 
      -DWITH_EMBEDDED_SERVER=OFF -DWITH_UNIT_TESTS=OFF -DCMAKE_BUILD_TYPE=Debug \
      		 
      -DWITH_INNODB_{BZIP2,LZ4,LZMA,LZO,SNAPPY}=OFF \
      		 
      -DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,CONNECT,SPIDER}=NO -DWITH_SAFEMALLOC=OFF \
      		 
      -DWITH_{ZLIB,SSL,PCRE}=bundled \
      		 
      -DHAVE_LIBAIO_H=0 \
      		 
      -DWITH_MSAN=ON \
      		 
      -G Ninja ..
      		 
      ninja

      Note: -march=native -mtune=native is optional since the second fix of MDEV-20386

      How to run tests

      cd mysql-test
      		 
      LD_LIBRARY_PATH=/mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib ./mtr main.1st
      		 
      LD_LIBRARY_PATH=/mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib MSAN_OPTIONS=abort_on_error=1 ./mtr --big-test --parallel=auto --force --retry=0

      Note: It may be wise to omit MSAN_OPTIONS=abort_on_error=1 except when running code in a debugger. On some occasions, it may cause truncation of the diagnostic messages.

        Attachments

          Issue Links

          includes

          Task - A task that needs to be done. MDBF-136 MSAN - create instrumented libcrack

          • Major - Major loss of function.
          • To Do

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20386 Replace inline asm with compiler-builtin intrinsic functions

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20388 Allegedly uninitialized values left by MyCTX_nopad

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22059 MSAN report at replicate_ignore_table_grant

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22688 MSAN use-of-uninitialized-value in READ_INFO::read_xml()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22689 MSAN use-of-uninitialized-value in decode_bytes()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22690 MSAN use-of-uninitialized-value in optimizer_trace

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22691 MSAN use-of-uninitialized-value in test maria.maria-recovery2

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22692 MSAN use-of-uninitialized-value in pagecache_fwrite()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. CONC-483 Workaround for MSAN bug needed

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20309 InnoDB encryption accesses memory outside of allocated block

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22070 MSAN use-of-uninitialized-value in encryption.innodb-redo-badkey

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22071 MSAN use-of-uninitialized-value in build_frm_image()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22072 MSAN use-of-uninitialized-value in wf_incremental_finalize

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22073 MSAN use-of-uninitialized-value in collect_statistics_for_table()

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22083 Allow libmariadb to be built with WolfSSL when WITH_SSL=bundled

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22244 Field::error_generated_column_function_is_not_allowed: Conditional jump or move depends on uninitialised value

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-24514 WITH_MSAN is disabling WOLFSSL_AESNI acceleration

          • Major - Major loss of function.
          • Open

          Task - A task that needs to be done. MDEV-14024 PCRE2

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20310 valgrind bugs found in 10.5

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20383 Use of uninitialized value in Datafile::find_space_id() for ROW_FORMAT=COMPRESSED

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-22060 MSAN use-of-uninitialized-value in main.query_cache_innodb

          • Major - Major loss of function.
          • Closed

            Activity

              People

              Assignee:
              marko Marko Mäkelä
              Reporter:
              marko Marko Mäkelä
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: