Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-24514

WITH_MSAN is disabling WOLFSSL_AESNI acceleration

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Won't Fix
    • 10.4(EOL), 10.5, 10.6
    • N/A
    • Encryption

    Description

      WolfSSL is the WITH_SSL=bundled library since MDEV-18351. For the AMD64 architecture, the library includes some assembler code. That code is disabled in WITH_MSAN builds, because MemorySanitizer would consider any data that is computed by uninstrumented code to be uninitialized. With the following patch, we would have a massive amount of failures when following the steps outlined in MDEV-20377 to compile and run a WITH_MSAN executable:

      diff --git a/extra/wolfssl/CMakeLists.txt b/extra/wolfssl/CMakeLists.txt
      index c99fb155dd6..f3b1c2af2c3 100644
      --- a/extra/wolfssl/CMakeLists.txt
      +++ b/extra/wolfssl/CMakeLists.txt
      @@ -17,16 +17,11 @@ ELSEIF(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|amd64")
         IF(CMAKE_C_COMPILER_ID MATCHES GNU  AND CMAKE_C_COMPILER_VERSION VERSION_LESS 4.9)
           MESSAGE_ONCE(NO_INTEL_ASSEMBLY "Disable Intel assembly for WolfSSL - compiler is too old")
         ELSE()
      -    IF(WITH_MSAN)
      -      MESSAGE_ONCE(MSAN_CANT_HANDLE_IT
      -	"Disable Intel assembly for WolfSSL - MSAN can't handle it")
      -    ELSE()
      -      MY_CHECK_C_COMPILER_FLAG(-maes)
      -      MY_CHECK_C_COMPILER_FLAG(-msse4)
      -      MY_CHECK_C_COMPILER_FLAG(-mpclmul)
      -      IF(have_C__maes AND have_C__msse4 AND have_C__mpclmul)
      -        SET(WOLFSSL_INTELASM ON)
      -      ENDIF()
      +    MY_CHECK_C_COMPILER_FLAG(-maes)
      +    MY_CHECK_C_COMPILER_FLAG(-msse4)
      +    MY_CHECK_C_COMPILER_FLAG(-mpclmul)
      +    IF(have_C__maes AND have_C__msse4 AND have_C__mpclmul)
      +      SET(WOLFSSL_INTELASM ON)
           ENDIF()
           MY_CHECK_C_COMPILER_FLAG(-mrdrnd)
           MY_CHECK_C_COMPILER_FLAG(-mrdseed)
      

      Here are two examples of failures:

      10.5 1bf9acceef252000618a137853638c612339024b

      2021-01-02 12:24:07 0 [Note] InnoDB: Encrypting redo log: 10485760 bytes; LSN=48382
      2021-01-02 12:24:07 0 [Note] InnoDB: Starting to delete and rewrite log file.
      2021-01-02 12:24:07 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 10485760 bytes
      Uninitialized bytes in __interceptor_pwrite64 at offset 500 inside [0x7fce3670e000, 512)
      ==315544==WARNING: MemorySanitizer: use-of-uninitialized-value
          #0 0x5562f26baf94 in SyncFileIO::execute(IORequest const&) /mariadb/10.5m/storage/innobase/os/os0file.cc:742:13
      

      10.5 1bf9acceef252000618a137853638c612339024b

      Uninitialized bytes in __interceptor_pwrite64 at offset 16378 inside [0x728000040000, 16384)
      ==315509==WARNING: MemorySanitizer: use-of-uninitialized-value
      Uninitialized bytes in __interceptor_pwrite64 at offset 16378 inside [0x728000044000, 16384)
          #0 0x5603a7d0112a in tpool::simulated_aio::simulated_aio_callback(void*) /mariadb/10.5m/tpool/aio_simulated.cc:148:16
      

      After applying an additional patch, we will get a little more information:

      diff --git a/storage/innobase/os/os0file.cc b/storage/innobase/os/os0file.cc
      index ac362e50404..b1626285196 100644
      --- a/storage/innobase/os/os0file.cc
      +++ b/storage/innobase/os/os0file.cc
      @@ -3224,6 +3224,7 @@ os_file_pwrite(
       
       	const bool monitor = MONITOR_IS_ON(MONITOR_OS_PENDING_WRITES);
       	MONITOR_ATOMIC_INC_LOW(MONITOR_OS_PENDING_WRITES, monitor);
      +	MEM_CHECK_DEFINED(buf, n);
       	ssize_t	n_bytes = os_file_io(type, file, const_cast<byte*>(buf),
       				     n, offset, err);
       	MONITOR_ATOMIC_DEC_LOW(MONITOR_OS_PENDING_WRITES, monitor);
      

      Uninitialized bytes in __msan_check_mem_is_initialized at offset 500 inside [0x7f731195e000, 512)
      ==320701==WARNING: MemorySanitizer: use-of-uninitialized-value
          #0 0x556a7492c2bf in os_file_pwrite(IORequest const&, int, unsigned char const*, unsigned long, unsigned long, dberr_t*) /mariadb/10.5m/storage/innobase/os/os0file.cc:3227:2
      ...
        Uninitialized value was stored to memory at
          #0 0x556a724ca909 in __msan_memcpy (/dev/shm/10.5msan/sql/mariadbd+0x716909)
          #1 0x556a74903869 in log_crypt(unsigned char*, unsigned long, unsigned long, log_crypt_t) /mariadb/10.5m/storage/innobase/log/log0crypt.cc:211:3
       
        Uninitialized value was stored to memory at
          #0 0x556a7427a056 in MyCTX_nopad::finish(unsigned char*, unsigned int*) /mariadb/10.5m/mysys_ssl/my_crypt.cc:158:15
       
        Uninitialized value was created by an allocation of 'mask' in the stack frame of function '_ZN11MyCTX_nopad6finishEPhPj'
          #0 0x556a74279960 in MyCTX_nopad::finish(unsigned char*, unsigned int*) /mariadb/10.5m/mysys_ssl/my_crypt.cc:133
      

      All uninstrumented assembler code is identified by the use of the macro XASM_LINK in the file wolfcrypt/src/aes.c of the submodule extra/wolfssl/wolfssl. WolfSSL pull request #3611 fixes that problem for me.

      Attachments

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.