Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • N/A
    • None
    • None

    Description

      marko is hitting MSAN issues with C/C due to a bug in MSAN and needs the following workaround applied:

      diff --git a/libmariadb/mariadb_stmt.c b/libmariadb/mariadb_stmt.c
      index 8f5bddc..e15e1bd 100644
      --- a/libmariadb/mariadb_stmt.c
      +++ b/libmariadb/mariadb_stmt.c
      @@ -1,5 +1,5 @@
       /****************************************************************************
      -  Copyright (C) 2012 Monty Program AB
      +  Copyright (C) 2012, 2020, MariaDB Corporation.
       
         This library is free software; you can redistribute it and/or
         modify it under the terms of the GNU Library General Public
      @@ -56,7 +56,12 @@
       #include <mysql/client_plugin.h>
       #include <ma_common.h>
       #include "ma_priv.h"
      -
      +#ifndef __has_feature
      +# define __has_feature(x) 0
      +#endif
      +#if __has_feature(memory_sanitizer)
      +# include <sanitizer/msan_interface.h>
      +#endif
       
       #define UPDATE_STMT_ERROR(stmt)\
       SET_CLIENT_STMT_ERROR((stmt), (stmt)->mysql->net.last_errno, (stmt)->mysql->net.sqlstate, (stmt)->mysql->net.last_error)
      @@ -542,7 +547,13 @@ int store_param(MYSQL_STMT *stmt, int column, unsigned char **p, unsigned long r
           (*p) += 4;
           break;
         case MYSQL_TYPE_DOUBLE:
      +#if __has_feature(memory_sanitizer)         /* QQ: MSAN has double trouble? */
      +    __msan_check_mem_is_initialized(buf, sizeof(double));
      +#endif
           float8store(*p, (*(double *)buf));
      +#if __has_feature(memory_sanitizer)        /* QQ: MSAN has double trouble? */
      +    __msan_unpoison(*p, sizeof(double));
      +#endif
           (*p) += 8;
           break;
         case MYSQL_TYPE_LONGLONG:
      

      Attachments

        Issue Links

          Activity

            Note: The patch that I supplied is not addressing the problem. Hopefully it gives an idea what should be done.

            I believe that the problem could be in MSAN itself. Here is what I saw (see MDEV-20377 how to use WITH_MSAN):

            MSAN_OPTIONS=abort_on_error=1:log_path=/dev/shm/msan LD_LIBRARY_PATH=/mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib ./mtr --parallel=auto --force --big-test main.mysql_client_test_nonblock main.mysql_client_test main.mysql_client_test_comp
            

            In one of the /dev/shm/msan.* files, there would be the following output:

            10.5 1813d92d0c505a1c752f4a9d6e37db6bffe4efdd

            Uninitialized bytes in __interceptor_send at offset 29 inside [0x724000002000, 42)
            ==2469456==WARNING: MemorySanitizer: use-of-uninitialized-value
                #0 0x5593fb8ac68c in ma_send /mariadb/10.5-merge/libmariadb/plugins/pvio/pvio_socket.c:373:9
                #1 0x5593fb8ac68c in pvio_socket_async_write /mariadb/10.5-merge/libmariadb/plugins/pvio/pvio_socket.c:429:6
                #2 0x5593fb83b690 in ma_pvio_write_async /mariadb/10.5-merge/libmariadb/libmariadb/ma_pvio.c:322:10
                #3 0x5593fb83b690 in ma_pvio_write /mariadb/10.5-merge/libmariadb/libmariadb/ma_pvio.c:356:23
                #4 0x5593fb8c1bb7 in ma_net_real_write /mariadb/10.5-merge/libmariadb/libmariadb/ma_net.c:335:17
                #5 0x5593fb8c3501 in ma_net_flush /mariadb/10.5-merge/libmariadb/libmariadb/ma_net.c:166:11
                #6 0x5593fb8c3501 in ma_net_write_command /mariadb/10.5-merge/libmariadb/libmariadb/ma_net.c:244:12
                #7 0x5593fb81173d in mthd_my_send_cmd /mariadb/10.5-merge/libmariadb/libmariadb/mariadb_lib.c:402:7
                #8 0x5593fb8567f1 in mysql_stmt_execute /mariadb/10.5-merge/libmariadb/libmariadb/mariadb_stmt.c:2068:8
                #9 0x5593fb89d24d in mysql_stmt_execute_start_internal /mariadb/10.5-merge/libmariadb/libmariadb/mariadb_async.c:1380:1
                #10 0x5593fb8a55dc in my_context_spawn /mariadb/10.5-merge/libmariadb/libmariadb/ma_context.c:201:3
              Uninitialized value was created by a heap allocation
                #0 0x5593fb508a8d in malloc (/dev/shm/10.5msan/tests/mariadb-client-test+0x66a8d)
            

            I basically tried to adapt the MDEV-22691 fix from the server. It seems that MemorySanitizer has some trouble managing the shadow bits of double variables.

            marko Marko Mäkelä added a comment - Note: The patch that I supplied is not addressing the problem. Hopefully it gives an idea what should be done. I believe that the problem could be in MSAN itself. Here is what I saw (see MDEV-20377 how to use WITH_MSAN ): MSAN_OPTIONS=abort_on_error=1:log_path=/dev/shm/msan LD_LIBRARY_PATH=/mariadb/llvm-toolchain-10-10.0.0/libc++msan/lib ./mtr --parallel=auto --force --big-test main.mysql_client_test_nonblock main.mysql_client_test main.mysql_client_test_comp In one of the /dev/shm/msan.* files, there would be the following output: 10.5 1813d92d0c505a1c752f4a9d6e37db6bffe4efdd Uninitialized bytes in __interceptor_send at offset 29 inside [0x724000002000, 42) ==2469456==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x5593fb8ac68c in ma_send /mariadb/10.5-merge/libmariadb/plugins/pvio/pvio_socket.c:373:9 #1 0x5593fb8ac68c in pvio_socket_async_write /mariadb/10.5-merge/libmariadb/plugins/pvio/pvio_socket.c:429:6 #2 0x5593fb83b690 in ma_pvio_write_async /mariadb/10.5-merge/libmariadb/libmariadb/ma_pvio.c:322:10 #3 0x5593fb83b690 in ma_pvio_write /mariadb/10.5-merge/libmariadb/libmariadb/ma_pvio.c:356:23 #4 0x5593fb8c1bb7 in ma_net_real_write /mariadb/10.5-merge/libmariadb/libmariadb/ma_net.c:335:17 #5 0x5593fb8c3501 in ma_net_flush /mariadb/10.5-merge/libmariadb/libmariadb/ma_net.c:166:11 #6 0x5593fb8c3501 in ma_net_write_command /mariadb/10.5-merge/libmariadb/libmariadb/ma_net.c:244:12 #7 0x5593fb81173d in mthd_my_send_cmd /mariadb/10.5-merge/libmariadb/libmariadb/mariadb_lib.c:402:7 #8 0x5593fb8567f1 in mysql_stmt_execute /mariadb/10.5-merge/libmariadb/libmariadb/mariadb_stmt.c:2068:8 #9 0x5593fb89d24d in mysql_stmt_execute_start_internal /mariadb/10.5-merge/libmariadb/libmariadb/mariadb_async.c:1380:1 #10 0x5593fb8a55dc in my_context_spawn /mariadb/10.5-merge/libmariadb/libmariadb/ma_context.c:201:3 Uninitialized value was created by a heap allocation #0 0x5593fb508a8d in malloc (/dev/shm/10.5msan/tests/mariadb-client-test+0x66a8d) I basically tried to adapt the MDEV-22691 fix from the server . It seems that MemorySanitizer has some trouble managing the shadow bits of double variables.
            marko Marko Mäkelä added a comment - - edited

            I am not seeing a change after the CONC-512 fix was applied. Byte 29 of the 42-byte buffer is allegedly still uninitialized.

            There is also another error, which I filed and fixed as CONC-513:

            3.1 93618b4036d61a4425154c7d8bc41647872b64cd

            Uninitialized bytes in StrstrCheck at offset 58 inside [0x7fff5b6ab880, 60)
            ==38278==WARNING: MemorySanitizer: use-of-uninitialized-value
                #0 0x562d177ea77e in strstr (/dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config+0x2e77e)
                #1 0x562d17855fe4 in mariadb_get_install_location /dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config.c:193:8
                #2 0x562d17855688 in main /dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config.c:214:3
                #3 0x7f1ad1985cc9 in __libc_start_main csu/../csu/libc-start.c:308:16
                #4 0x562d177dc2a9 in _start (/dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config+0x202a9)
             
              Uninitialized value was created by an allocation of '' in the stack frame of function 'mariadb_get_install_location'
                #0 0x562d17855c60 in mariadb_get_install_location /dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config.c:146
             
            SUMMARY: MemorySanitizer: use-of-uninitialized-value (/dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config+0x2e77e) in strstr
            Exiting
            

            Applying the patch in the Description does not make any difference to either error.

            marko Marko Mäkelä added a comment - - edited I am not seeing a change after the CONC-512 fix was applied. Byte 29 of the 42-byte buffer is allegedly still uninitialized. There is also another error, which I filed and fixed as CONC-513 : 3.1 93618b4036d61a4425154c7d8bc41647872b64cd Uninitialized bytes in StrstrCheck at offset 58 inside [0x7fff5b6ab880, 60) ==38278==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x562d177ea77e in strstr (/dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config+0x2e77e) #1 0x562d17855fe4 in mariadb_get_install_location /dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config.c:193:8 #2 0x562d17855688 in main /dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config.c:214:3 #3 0x7f1ad1985cc9 in __libc_start_main csu/../csu/libc-start.c:308:16 #4 0x562d177dc2a9 in _start (/dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config+0x202a9)   Uninitialized value was created by an allocation of '' in the stack frame of function 'mariadb_get_install_location' #0 0x562d17855c60 in mariadb_get_install_location /dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config.c:146   SUMMARY: MemorySanitizer: use-of-uninitialized-value (/dev/shm/10.5msan/libmariadb/mariadb_config/mariadb_config+0x2e77e) in strstr Exiting Applying the patch in the Description does not make any difference to either error.

            I think that fixing MDEV-26761 (test_mdev19838() in tests/mysql_client_test.c) may require some changes in Connector/C. I disabled test_mdev19838() in MemorySanitizer builds for now.

            See MDEV-20377 how to build MSAN-instrumented libraries. For main.mysql_client_test I think that only libc++ is needed. Actually even that can be skipped if you copy the cmake -DWITH_MSAN=ON client executable to a non-MSAN build directory. The problem is in the client side, not in the server.

            marko Marko Mäkelä added a comment - I think that fixing MDEV-26761 ( test_mdev19838() in tests/mysql_client_test.c ) may require some changes in Connector/C. I disabled test_mdev19838() in MemorySanitizer builds for now. See MDEV-20377 how to build MSAN-instrumented libraries. For main.mysql_client_test I think that only libc++ is needed. Actually even that can be skipped if you copy the cmake -DWITH_MSAN=ON client executable to a non-MSAN build directory. The problem is in the client side, not in the server.

            No work-around for MSAN seems to be necessary anymore.

            I was not able to run the server’s ./mtr --suite=unit under MemorySanitizer, though. It complained about some unresolved MSAN symbol.

            marko Marko Mäkelä added a comment - No work-around for MSAN seems to be necessary anymore. I was not able to run the server’s ./mtr --suite=unit under MemorySanitizer, though. It complained about some unresolved MSAN symbol.

            People

              georg Georg Richter
              LinuxJedi Andrew Hutchings (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.