[MDEV-24514] WITH_MSAN is disabling WOLFSSL_AESNI acceleration Created: 2021-01-02  Updated: 2021-02-23  Resolved: 2021-02-23

Status: Closed
Project: MariaDB Server
Component/s: Encryption
Affects Version/s: 10.4, 10.5, 10.6
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Won't Fix Votes: 0
Labels: MSAN

Issue Links:
Relates
relates to MDEV-18531 Use WolfSSL instead of YaSSL as "bund... Closed
relates to MDEV-20377 Make WITH_MSAN more usable Closed
relates to MDEV-20386 Replace inline asm with compiler-buil... Closed

 Description   

WolfSSL is the WITH_SSL=bundled library since MDEV-18351. For the AMD64 architecture, the library includes some assembler code. That code is disabled in WITH_MSAN builds, because MemorySanitizer would consider any data that is computed by uninstrumented code to be uninitialized. With the following patch, we would have a massive amount of failures when following the steps outlined in MDEV-20377 to compile and run a WITH_MSAN executable:

diff --git a/extra/wolfssl/CMakeLists.txt b/extra/wolfssl/CMakeLists.txt
index c99fb155dd6..f3b1c2af2c3 100644
--- a/extra/wolfssl/CMakeLists.txt
+++ b/extra/wolfssl/CMakeLists.txt
@@ -17,16 +17,11 @@ ELSEIF(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|amd64")
   IF(CMAKE_C_COMPILER_ID MATCHES GNU  AND CMAKE_C_COMPILER_VERSION VERSION_LESS 4.9)
     MESSAGE_ONCE(NO_INTEL_ASSEMBLY "Disable Intel assembly for WolfSSL - compiler is too old")
   ELSE()
-    IF(WITH_MSAN)
-      MESSAGE_ONCE(MSAN_CANT_HANDLE_IT
-	"Disable Intel assembly for WolfSSL - MSAN can't handle it")
-    ELSE()
-      MY_CHECK_C_COMPILER_FLAG(-maes)
-      MY_CHECK_C_COMPILER_FLAG(-msse4)
-      MY_CHECK_C_COMPILER_FLAG(-mpclmul)
-      IF(have_C__maes AND have_C__msse4 AND have_C__mpclmul)
-        SET(WOLFSSL_INTELASM ON)
-      ENDIF()
+    MY_CHECK_C_COMPILER_FLAG(-maes)
+    MY_CHECK_C_COMPILER_FLAG(-msse4)
+    MY_CHECK_C_COMPILER_FLAG(-mpclmul)
+    IF(have_C__maes AND have_C__msse4 AND have_C__mpclmul)
+      SET(WOLFSSL_INTELASM ON)
     ENDIF()
     MY_CHECK_C_COMPILER_FLAG(-mrdrnd)
     MY_CHECK_C_COMPILER_FLAG(-mrdseed)

Here are two examples of failures:

10.5 1bf9acceef252000618a137853638c612339024b

2021-01-02 12:24:07 0 [Note] InnoDB: Encrypting redo log: 10485760 bytes; LSN=48382
2021-01-02 12:24:07 0 [Note] InnoDB: Starting to delete and rewrite log file.
2021-01-02 12:24:07 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 10485760 bytes
Uninitialized bytes in __interceptor_pwrite64 at offset 500 inside [0x7fce3670e000, 512)
==315544==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5562f26baf94 in SyncFileIO::execute(IORequest const&) /mariadb/10.5m/storage/innobase/os/os0file.cc:742:13

10.5 1bf9acceef252000618a137853638c612339024b

Uninitialized bytes in __interceptor_pwrite64 at offset 16378 inside [0x728000040000, 16384)
==315509==WARNING: MemorySanitizer: use-of-uninitialized-value
Uninitialized bytes in __interceptor_pwrite64 at offset 16378 inside [0x728000044000, 16384)
    #0 0x5603a7d0112a in tpool::simulated_aio::simulated_aio_callback(void*) /mariadb/10.5m/tpool/aio_simulated.cc:148:16

After applying an additional patch, we will get a little more information:

diff --git a/storage/innobase/os/os0file.cc b/storage/innobase/os/os0file.cc
index ac362e50404..b1626285196 100644
--- a/storage/innobase/os/os0file.cc
+++ b/storage/innobase/os/os0file.cc
@@ -3224,6 +3224,7 @@ os_file_pwrite(
 
 	const bool monitor = MONITOR_IS_ON(MONITOR_OS_PENDING_WRITES);
 	MONITOR_ATOMIC_INC_LOW(MONITOR_OS_PENDING_WRITES, monitor);
+	MEM_CHECK_DEFINED(buf, n);
 	ssize_t	n_bytes = os_file_io(type, file, const_cast<byte*>(buf),
 				     n, offset, err);
 	MONITOR_ATOMIC_DEC_LOW(MONITOR_OS_PENDING_WRITES, monitor);

Uninitialized bytes in __msan_check_mem_is_initialized at offset 500 inside [0x7f731195e000, 512)
==320701==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x556a7492c2bf in os_file_pwrite(IORequest const&, int, unsigned char const*, unsigned long, unsigned long, dberr_t*) /mariadb/10.5m/storage/innobase/os/os0file.cc:3227:2
...
  Uninitialized value was stored to memory at
    #0 0x556a724ca909 in __msan_memcpy (/dev/shm/10.5msan/sql/mariadbd+0x716909)
    #1 0x556a74903869 in log_crypt(unsigned char*, unsigned long, unsigned long, log_crypt_t) /mariadb/10.5m/storage/innobase/log/log0crypt.cc:211:3
 
  Uninitialized value was stored to memory at
    #0 0x556a7427a056 in MyCTX_nopad::finish(unsigned char*, unsigned int*) /mariadb/10.5m/mysys_ssl/my_crypt.cc:158:15
 
  Uninitialized value was created by an allocation of 'mask' in the stack frame of function '_ZN11MyCTX_nopad6finishEPhPj'
    #0 0x556a74279960 in MyCTX_nopad::finish(unsigned char*, unsigned int*) /mariadb/10.5m/mysys_ssl/my_crypt.cc:133

All uninstrumented assembler code is identified by the use of the macro XASM_LINK in the file wolfcrypt/src/aes.c of the submodule extra/wolfssl/wolfssl. WolfSSL pull request #3611 fixes that problem for me.



 Comments   
Comment by Marko Mäkelä [ 2021-02-23 ]

Because the MSAN instrumentation wrapper around the hand-written assembler code performs much slower than MSAN-instrumented high-level code, we’d better avoid this change.

Generated at Thu Feb 08 09:30:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.