Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27964

The function ENCRYPT() causes SIGSEGV in WITH_MSAN builds

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • 10.5, 10.6, 10.7(EOL), 10.8(EOL)
    • 10.5, 10.6
    • Encryption

    Description

      In both clang-10 and clang-12, a WITH_MSAN build of the server would crash with SIGSEGV if the ENCRYPT() function is evaluated. The problem seems to be related to the MSAN interceptor of the crypt() function. I was unable to reproduce this with a simple test program:

      #include <crypt.h>
      		 
      #include <stdio.h>
      		 
       
      		 
      int main(int argc, char **argv)
      		 
      {
      		 
        printf("crypt: %s\n", crypt(argv[1], "123"));
      		 
        return 0;
      		 
      }


      clang-10 -fsanitize=undefined crypt.c -lcrypt
      		 
      ./a.out foo

      I see that re-entrant versions of the C function crypt() exist, but we are using the original version with LOCK_crypt around it. Perhaps one of the other variants of the function would be intercepted better?

      Note: No memory appears to be uninitialized. The following patch did not trigger anything; the very first sign of trouble was the SIGSEGV.

      diff --git a/sql/item_strfunc.cc b/sql/item_strfunc.cc
      		 
      index 0635556be40..3f9e023c793 100644
      		 
      --- a/sql/item_strfunc.cc
      		 
      +++ b/sql/item_strfunc.cc
      		 
      @@ -2280,6 +2280,8 @@ String *Item_func_encrypt::val_str(String *str)
      		 
           salt_ptr= salt_str->c_ptr_safe();
      		 
         }
      		 
         mysql_mutex_lock(&LOCK_crypt);
      		 
      +  MEM_CHECK_DEFINED(res->c_ptr_safe(), res->length);
      		 
      +  MEM_CHECK_DEFINED(salt_ptr, 2);
      		 
         char *tmp= crypt(res->c_ptr_safe(),salt_ptr);
      		 
         if (!tmp)
      		 
         {

      To work around this problem, I will move some encrypt() test cases from the files main.view and main.func_encrypt to the test main.func_crypt, which will be disabled in cmake -DWITH_MSAN=ON builds for now.

      Attachments

        Issue Links

          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. MDBF-793 Retire MSAN clang-15 builder, upgrade to clang-19

          • Major - Major loss of function.
          • In Progress
          relates to

          Task - A task that needs to be done. MDBF-276 Apply changes to the MSAN builder

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-20377 Make WITH_MSAN more usable

          • Major - Major loss of function.
          • Closed

          Activity

            danblack Daniel Black added a comment - - edited

            Test program not crashing on debian12/clang-19

            buildbot@d701c4dff4b2:/build$ clang -fsanitize=undefined test.c -lcrypt
            		 
            buildbot@d701c4dff4b2:/build$ ./a.out foo
            		 
            crypt: 12o7559gAGYWY
            		 
            buildbot@d701c4dff4b2:/build$ rm a.out
            		 
            buildbot@d701c4dff4b2:/build$ clang -fsanitize=memory test.c -lcrypt
            		 
            buildbot@d701c4dff4b2:/build$ ./a.out food
            		 
            crypt: 12jorgcEoPNfk
            		 
            buildbot@d701c4dff4b2:/build$ clang --version
            		 
            Debian clang version 19.1.2 (++20241001023520+d5498c39fe6a-1~exp1~20241001143639.51)
            		 
            Target: x86_64-pc-linux-gnu
            		 
            Thread model: posix
            		 
            InstalledDir: /usr/lib/llvm-19/bin
            		 
            buildbot@d701c4dff4b2:/build$ ls /msan-libs/
            		 
            bin		libc++abi.so.1.0   libfmt.so.9	     libgnutls.so.30.34.3  libnettle.so.8	 libpcre2-posix.so	  libunwind.so.1.0
            		 
            include		libcrack.so	   libfmt.so.9.1.0   libhogweed.so	   libodbc.so		 libpcre2-posix.so.3	  libxml2.so
            		 
            libc++.so	libcrack.so.2	   libgmp.so	     libhogweed.so.6	   libodbc.so.2		 libpcre2-posix.so.3.0.4  libxml2.so.2
            		 
            libc++.so.1	libcrack.so.2.9.0  libgmp.so.10      libidn2.so		   libodbc.so.2.0.0	 libssl.so		  libxml2.so.2.9.14
            		 
            libc++.so.1.0	libcrypto.so	   libgmp.so.10.4.1  libidn2.so.0	   libpcre2-8.so	 libssl.so.3
            		 
            libc++abi.so	libcrypto.so.3	   libgnutls.so      libidn2.so.0.3.8	   libpcre2-8.so.0	 libunwind.so
            		 
            libc++abi.so.1	libfmt.so	   libgnutls.so.30   libnettle.so	   libpcre2-8.so.0.11.2  libunwind.so.1

            danblack Daniel Black added a comment - - edited Test program not crashing on debian12/clang-19 buildbot@d701c4dff4b2:/build$ clang -fsanitize=undefined test.c -lcrypt buildbot@d701c4dff4b2:/build$ ./a.out foo crypt: 12o7559gAGYWY buildbot@d701c4dff4b2:/build$ rm a.out buildbot@d701c4dff4b2:/build$ clang -fsanitize=memory test.c -lcrypt buildbot@d701c4dff4b2:/build$ ./a.out food crypt: 12jorgcEoPNfk buildbot@d701c4dff4b2:/build$ clang --version Debian clang version 19.1.2 (++20241001023520+d5498c39fe6a-1~exp1~20241001143639.51) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/lib/llvm-19/bin buildbot@d701c4dff4b2:/build$ ls /msan-libs/ bin libc++abi.so.1.0 libfmt.so.9 libgnutls.so.30.34.3 libnettle.so.8 libpcre2-posix.so libunwind.so.1.0 include libcrack.so libfmt.so.9.1.0 libhogweed.so libodbc.so libpcre2-posix.so.3 libxml2.so libc++.so libcrack.so.2 libgmp.so libhogweed.so.6 libodbc.so.2 libpcre2-posix.so.3.0.4 libxml2.so.2 libc++.so.1 libcrack.so.2.9.0 libgmp.so.10 libidn2.so libodbc.so.2.0.0 libssl.so libxml2.so.2.9.14 libc++.so.1.0 libcrypto.so libgmp.so.10.4.1 libidn2.so.0 libpcre2-8.so libssl.so.3 libc++abi.so libcrypto.so.3 libgnutls.so libidn2.so.0.3.8 libpcre2-8.so.0 libunwind.so libc++abi.so.1 libfmt.so libgnutls.so.30 libnettle.so libpcre2-8.so.0.11.2 libunwind.so.1

            People

              danblack Daniel Black
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.