Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19415

multi_source.mdev-9544 , multi_source.info_logs failed in buildbot with AddressSanitizer: heap-use-after-free in strend / get_one_variable

Details

    Description

      http://buildbot.askmonty.org/buildbot/builders/kvm-asan/builds/662

      10.2 1e9c2b23

      ==16830==ERROR: AddressSanitizer: heap-use-after-free on address 0x610000077cb0 at pc 0x558f088eb90e bp 0x7fe6d52bd640 sp 0x7fe6d52bd630
      READ of size 1 at 0x610000077cb0 thread T28
          #0 0x558f088eb90d in strend /home/buildbot/buildbot/build/mariadb-10.2.24/strings/strend.c:45
          #1 0x558f073abacb in get_one_variable(THD*, st_mysql_show_var const*, enum_var_type, enum_mysql_show_type, system_status_var*, charset_info_st const**, char*, unsigned long*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_show.cc:3430
          #2 0x558f073ac937 in show_status_array /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_show.cc:3573
          #3 0x558f073da7f0 in fill_variables(THD*, TABLE_LIST*, Item*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_show.cc:7496
          #4 0x558f073e276b in get_schema_tables_result(JOIN*, enum_schema_table_state) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_show.cc:8425
          #5 0x558f072e4e38 in JOIN::exec_inner() /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_select.cc:3587
          #6 0x558f072e316b in JOIN::exec() /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_select.cc:3418
          #7 0x558f072e64b8 in mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_select.cc:3818
          #8 0x558f072c5c7e in handle_select(THD*, LEX*, select_result*, unsigned long) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_select.cc:376
          #9 0x558f0724d5ee in execute_sqlcom_select /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:6479
          #10 0x558f0723aab4 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:3537
          #11 0x558f07b98bc5 in sp_instr_stmt::exec_core(THD*, unsigned int*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sp_head.cc:3246
          #12 0x558f07b97848 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sp_head.cc:3009
          #13 0x558f07b984a2 in sp_instr_stmt::execute(THD*, unsigned int*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sp_head.cc:3162
          #14 0x558f07b8de43 in sp_head::execute(THD*, bool) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sp_head.cc:1327
          #15 0x558f07b91c9a in sp_head::execute_procedure(THD*, List<Item>*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sp_head.cc:2116
          #16 0x558f07236f58 in do_execute_sp /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:2956
          #17 0x558f07248a89 in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:5827
          #18 0x558f07255d52 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:8013
          #19 0x558f07231407 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:1832
          #20 0x558f0722e586 in do_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:1386
          #21 0x558f07556367 in do_handle_one_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_connect.cc:1335
          #22 0x558f07555d6f in handle_one_connection /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_connect.cc:1241
          #23 0x558f07d06a93 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.24/storage/perfschema/pfs.cc:1862
          #24 0x7fe6eca366b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
          #25 0x7fe6ebecb82c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10682c)
       
      0x610000077cb0 is located 112 bytes inside of 188-byte region [0x610000077c40,0x610000077cfc)
      freed by thread T33 here:
          #0 0x7fe6ee2142ca in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x982ca)
          #1 0x558f0882acdb in free_memory /home/buildbot/buildbot/build/mariadb-10.2.24/mysys/safemalloc.c:279
          #2 0x558f0882a365 in sf_free /home/buildbot/buildbot/build/mariadb-10.2.24/mysys/safemalloc.c:197
          #3 0x558f087faed8 in my_free /home/buildbot/buildbot/build/mariadb-10.2.24/mysys/my_malloc.c:218
          #4 0x558f07766fb5 in mysql_close_free_options /home/buildbot/buildbot/build/mariadb-10.2.24/sql-common/client.c:3828
          #5 0x558f07767afe in mysql_close /home/buildbot/buildbot/build/mariadb-10.2.24/sql-common/client.c:3991
          #6 0x558f070b67e4 in handle_slave_io /home/buildbot/buildbot/build/mariadb-10.2.24/sql/slave.cc:4651
          #7 0x558f07d06a93 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.24/storage/perfschema/pfs.cc:1862
          #8 0x7fe6eca366b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
       
      previously allocated by thread T33 here:
          #0 0x7fe6ee214602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
          #1 0x558f08829d36 in sf_malloc /home/buildbot/buildbot/build/mariadb-10.2.24/mysys/safemalloc.c:118
          #2 0x558f087fa63f in my_malloc /home/buildbot/buildbot/build/mariadb-10.2.24/mysys/my_malloc.c:101
          #3 0x558f087fb1a4 in my_strdup /home/buildbot/buildbot/build/mariadb-10.2.24/mysys/my_malloc.c:241
          #4 0x558f0776adbb in mysql_options /home/buildbot/buildbot/build/mariadb-10.2.24/sql-common/client.c:4337
          #5 0x558f070c214a in connect_to_master /home/buildbot/buildbot/build/mariadb-10.2.24/sql/slave.cc:6717
          #6 0x558f070c19a5 in safe_connect /home/buildbot/buildbot/build/mariadb-10.2.24/sql/slave.cc:6647
          #7 0x558f070b4ff4 in handle_slave_io /home/buildbot/buildbot/build/mariadb-10.2.24/sql/slave.cc:4336
          #8 0x558f07d06a93 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.24/storage/perfschema/pfs.cc:1862
          #9 0x7fe6eca366b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
       
      Thread T28 created by T0 here:
          #0 0x7fe6ee1b2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
          #1 0x558f07d06e80 in spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.2.24/storage/perfschema/pfs.cc:1912
          #2 0x558f0703d38e in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.2.24/include/mysql/psi/mysql_thread.h:1239
          #3 0x558f07051995 in create_thread_to_handle_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/mysqld.cc:6482
          #4 0x558f07052095 in create_new_thread /home/buildbot/buildbot/build/mariadb-10.2.24/sql/mysqld.cc:6552
          #5 0x558f070530d8 in handle_connections_sockets() /home/buildbot/buildbot/build/mariadb-10.2.24/sql/mysqld.cc:6827
          #6 0x558f07050ee0 in mysqld_main(int, char**) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/mysqld.cc:6101
          #7 0x558f0703b79f in main /home/buildbot/buildbot/build/mariadb-10.2.24/sql/main.cc:25
          #8 0x7fe6ebde582f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
       
      Thread T33 created by T28 here:
          #0 0x7fe6ee1b2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
          #1 0x558f07d06e80 in spawn_thread_v1 /home/buildbot/buildbot/build/mariadb-10.2.24/storage/perfschema/pfs.cc:1912
          #2 0x558f0709e39b in inline_mysql_thread_create /home/buildbot/buildbot/build/mariadb-10.2.24/include/mysql/psi/mysql_thread.h:1239
          #3 0x558f070a2de6 in start_slave_thread(unsigned int, void* (*)(void*), st_mysql_mutex*, st_mysql_mutex*, st_mysql_cond*, unsigned int volatile*, unsigned long volatile*, Master_info*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/slave.cc:931
          #4 0x558f070a36d8 in start_slave_threads(THD*, bool, bool, Master_info*, char const*, char const*, int) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/slave.cc:1053
          #5 0x558f072bb561 in start_slave(THD*, Master_info*, bool) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_repl.cc:3117
          #6 0x558f075424a3 in Master_info_index::start_all_slaves(THD*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/rpl_mi.cc:1644
          #7 0x558f0723d71d in mysql_execute_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:4174
          #8 0x558f07255d52 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:8013
          #9 0x558f07231407 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:1832
          #10 0x558f0722e586 in do_command(THD*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_parse.cc:1386
          #11 0x558f07556367 in do_handle_one_connection(CONNECT*) /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_connect.cc:1335
          #12 0x558f07555d6f in handle_one_connection /home/buildbot/buildbot/build/mariadb-10.2.24/sql/sql_connect.cc:1241
          #13 0x558f07d06a93 in pfs_spawn_thread /home/buildbot/buildbot/build/mariadb-10.2.24/storage/perfschema/pfs.cc:1862
          #14 0x7fe6eca366b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
       
      SUMMARY: AddressSanitizer: heap-use-after-free /home/buildbot/buildbot/build/mariadb-10.2.24/strings/strend.c:45 strend
      Shadow bytes around the buggy address:
        0x0c2080006f40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c2080006f50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c2080006f60: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
        0x0c2080006f70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
        0x0c2080006f80: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
      =>0x0c2080006f90: fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd
        0x0c2080006fa0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
        0x0c2080006fb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
        0x0c2080006fc0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
        0x0c2080006fd0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
        0x0c2080006fe0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07 
        Heap left redzone:       fa
        Heap right redzone:      fb
        Freed heap region:       fd
        Stack left redzone:      f1
        Stack mid redzone:       f2
        Stack right redzone:     f3
        Stack partial redzone:   f4
        Stack after return:      f5
        Stack use after scope:   f8
        Global redzone:          f9
        Global init order:       f6
        Poisoned by user:        f7
        Container overflow:      fc
        Array cookie:            ac
        Intra object redzone:    bb
        ASan internal:           fe
      ==16830==ABORTING
      190327  9:40:07 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
       
      To report this bug, see https://mariadb.com/kb/en/reporting-bugs
       
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
       
      Server version: 10.2.24-MariaDB-debug-log
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=1
      max_threads=153
      thread_count=8
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63102 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
       
      Thread pointer: 0x62a00008a270
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7fe6d52c3c70 thread_stack 0x5b000
      /usr/lib/x86_64-linux-gnu/libasan.so.2(+0x4a077)[0x7fe6ee1c6077]
      /home/buildbot/buildbot/build/mariadb-10.2.24/sql/mysqld(my_print_stacktrace+0xab)[0x558f0880a61f]
      /home/buildbot/buildbot/build/mariadb-10.2.24/sql/mysqld(handle_fatal_signal+0x8da)[0x558f077fa7b8]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe6eca40390]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fe6ebdfa428]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fe6ebdfc02a]
      /usr/lib/x86_64-linux-gnu/libasan.so.2(+0xaddc9)[0x7fe6ee229dc9]
      /usr/lib/x86_64-linux-gnu/libasan.so.2(+0xa0769)[0x7fe6ee21c769]
      /usr/lib/x86_64-linux-gnu/libasan.so.2(+0xa55d2)[0x7fe6ee2215d2]
      /usr/lib/x86_64-linux-gnu/libasan.so.2(__asan_report_error+0x416)[0x7fe6ee21b6e6]
      /usr/lib/x86_64-linux-gnu/libasan.so.2(__asan_report_load1+0x33)[0x7fe6ee21cb13]
      strings/strend.c:45(strend)[0x558f088eb90e]
      sql/sql_show.cc:3430(get_one_variable(THD*, st_mysql_show_var const*, enum_var_type, enum_mysql_show_type, system_status_var*, charset_info_st const**, char*, unsigned long*))[0x558f073abacc]
      sql/sql_show.cc:3573(show_status_array(THD*, char const*, st_mysql_show_var*, enum_var_type, system_status_var*, char const*, TABLE*, bool, Item*))[0x558f073ac938]
      sql/sql_show.cc:7496(fill_variables(THD*, TABLE_LIST*, Item*))[0x558f073da7f1]
      sql/sql_show.cc:8425(get_schema_tables_result(JOIN*, enum_schema_table_state))[0x558f073e276c]
      sql/sql_select.cc:3586(JOIN::exec_inner())[0x558f072e4e39]
      sql/sql_select.cc:3419(JOIN::exec())[0x558f072e316c]
      sql/sql_select.cc:3820(mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*))[0x558f072e64b9]
      sql/sql_select.cc:376(handle_select(THD*, LEX*, select_result*, unsigned long))[0x558f072c5c7f]
      sql/sql_parse.cc:6479(execute_sqlcom_select(THD*, TABLE_LIST*))[0x558f0724d5ef]
      sql/sql_parse.cc:3537(mysql_execute_command(THD*))[0x558f0723aab5]
      sql/sp_head.cc:3246(sp_instr_stmt::exec_core(THD*, unsigned int*))[0x558f07b98bc6]
      sql/sp_head.cc:3009(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*))[0x558f07b97849]
      sql/sp_head.cc:3162(sp_instr_stmt::execute(THD*, unsigned int*))[0x558f07b984a3]
      sql/sp_head.cc:1327(sp_head::execute(THD*, bool))[0x558f07b8de44]
      sql/sp_head.cc:2116(sp_head::execute_procedure(THD*, List<Item>*))[0x558f07b91c9b]
      sql/sql_parse.cc:2956(do_execute_sp(THD*, sp_head*))[0x558f07236f59]
      sql/sql_parse.cc:5827(mysql_execute_command(THD*))[0x558f07248a8a]
      sql/sql_parse.cc:8013(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x558f07255d53]
      sql/sql_parse.cc:1834(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x558f07231408]
      sql/sql_parse.cc:1386(do_command(THD*))[0x558f0722e587]
      sql/sql_connect.cc:1335(do_handle_one_connection(CONNECT*))[0x558f07556368]
      sql/sql_connect.cc:1242(handle_one_connection)[0x558f07555d70]
      perfschema/pfs.cc:1864(pfs_spawn_thread)[0x558f07d06a94]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7fe6eca366ba]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fe6ebecb82d]
       
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x62500027b188): SELECT * FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES     WHERE variable_name NOT IN ('timestamp', 'innodb_file_format_max')      AND variable_name not like "Last_IO_Err*"      AND variable_name != 'INNODB_IBUF_MAX_SIZE'      AND variable_name != 'INNODB_USE_NATIVE_AIO'      AND variable_name != 'INNODB_BUFFER_POOL_LOAD_AT_STARTUP'      AND variable_name not like 'GTID%POS'      AND variable_name != 'GTID_BINLOG_STATE'    ORDER BY variable_name
      Connection ID (thread ID): 18
      Status: NOT_KILLED
      

      Attachments

        Issue Links

          Activity

            Yes, I think that the MemorySanitizer in recent clang can catch heap-use-after-free, by declaring freed memory as uninitialized. There is also MSAN_OPTIONS=poison_in_dtor=1 which was recently enabled by default; see MDEV-30942.

            Here is one more recent failure:

            bb-10.9-release 6f91631bae0bc6dfc527ec3c172e9fa9f83a1161

            multi_source.mdev-9544 'innodb'          w20 [ fail ]
                    Test ended at 2023-08-02 18:56:47
             
            CURRENT_TEST: multi_source.mdev-9544
             
             
            The server [mysqld.4 - pid: 35689, winpid: 35689, exit: 256] crashed while running 'check testcase after test'
            …
            2023-08-02 18:56:43 11 [Note] Master 'm1': Error reading relay log event: slave SQL thread was killed
            2023-08-02 18:56:43 11 [Note] Master 'm1': Slave SQL thread exiting, replication stopped in log 'mysqld-bin.000001' at position 1256, master: 127.0.0.1:16300
            2023-08-02 18:56:43 10 [Note] Master 'm1': Slave I/O thread exiting, read up to log 'mysqld-bin.000001', position 1256, master 127.0.0.1:16300
            ==35746==WARNING: MemorySanitizer: use-of-uninitialized-value
                #0 0x55e36abca6b6 in strend /home/buildbot/amd64-debian-11-msan/build/strings/strend.c:45:3
                #1 0x55e368204382 in get_one_variable(THD*, st_mysql_show_var const*, enum_var_type, enum_mysql_show_type, system_status_var*, charset_info_st const**, char*, unsigned long*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc
                #2 0x55e36822abfc in show_status_array(THD*, char const*, st_mysql_show_var*, enum_var_type, system_status_var*, char const*, TABLE*, bool, Item*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc:3897:14
                #3 0x55e3682296ff in fill_variables(THD*, TABLE_LIST*, Item*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc:7996:8
                #4 0x55e368234bec in get_schema_tables_result(JOIN*, enum_schema_table_state) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc:8941:11
                #5 0x55e3680e48e6 in JOIN::exec_inner() /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:4806:7
                #6 0x55e3680e208a in JOIN::exec() /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:4627:3
                #7 0x55e368055fd2 in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:5107:9
                #8 0x55e368054b75 in handle_select(THD*, LEX*, select_result*, unsigned long) /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:584:10
                #9 0x55e367f3d91f in execute_sqlcom_select(THD*, TABLE_LIST*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:6288:12
                #10 0x55e367f1deff in mysql_execute_command(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:3958:12
                #11 0x55e367bc941d in sp_instr_stmt::exec_core(THD*, unsigned int*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:3854:12
                #12 0x55e367bc47d7 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:3579:17
                #13 0x55e367bc742d in sp_instr_stmt::execute(THD*, unsigned int*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:3760:25
                #14 0x55e367ba47ad in sp_head::execute(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:1456:20
                #15 0x55e367bad0de in sp_head::execute_procedure(THD*, List<Item>*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:2443:5
                #16 0x55e367f17f0c in do_execute_sp(THD*, sp_head*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:3034:16
                #17 0x55e367f17077 in Sql_cmd_call::execute(THD*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:3280:9
                #18 0x55e367f1dc01 in mysql_execute_command(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:6024:26
                #19 0x55e367f06db9 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:8060:18
                #20 0x55e367efd570 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:1894:7
                #21 0x55e367f084e2 in do_command(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:1407:17
                #22 0x55e368542e86 in do_handle_one_connection(CONNECT*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_connect.cc:1416:11
                #23 0x55e368542489 in handle_one_connection /home/buildbot/amd64-debian-11-msan/build/sql/sql_connect.cc:1318:5
                #24 0x55e36979fede in pfs_spawn_thread /home/buildbot/amd64-debian-11-msan/build/storage/perfschema/pfs.cc:2201:3
                #25 0x7fcb764eeea6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7ea6) (BuildId: 255e355c207aba91a59ae1f808e3b4da443abf0c)
                #26 0x7fcb75ef6a2e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfba2e) (BuildId: e15ec78d51a522023f9cfc58dc284f379d81860b)
             
              Uninitialized value was created by a heap deallocation
                #0 0x55e36795672c in free (/home/buildbot/amd64-debian-11-msan/build/sql/mariadbd+0x86872c) (BuildId: d58deefb908e1a6bd640a02927dc493177d22787)
                #1 0x55e368a6fba6 in mysql_close_free_options /home/buildbot/amd64-debian-11-msan/build/sql-common/client.c:3290:3
            

            The reason why I got interested in this test was that I got this local test failure yesterday for a 10.6 based branch:

            multi_source.mdev-9544 'innodb'          w30 [ pass ]   2264
             
            MTR's internal check of the test case 'multi_source.mdev-9544' failed.
            …
            -CHARACTER_SETS_DIR	/mariadb/10.6/sql/share/charsets/
            +CHARACTER_SETS_DIR	????????????????????????????????????????
            

            This would not be flagged as a failure by mtr, so I would guess it to be rather common on buildbot as well.

            marko Marko Mäkelä added a comment - Yes, I think that the MemorySanitizer in recent clang can catch heap-use-after-free, by declaring freed memory as uninitialized. There is also MSAN_OPTIONS=poison_in_dtor=1 which was recently enabled by default; see MDEV-30942 . Here is one more recent failure : bb-10.9-release 6f91631bae0bc6dfc527ec3c172e9fa9f83a1161 multi_source.mdev-9544 'innodb' w20 [ fail ] Test ended at 2023-08-02 18:56:47   CURRENT_TEST: multi_source.mdev-9544     The server [mysqld.4 - pid: 35689, winpid: 35689, exit: 256] crashed while running 'check testcase after test' … 2023-08-02 18:56:43 11 [Note] Master 'm1': Error reading relay log event: slave SQL thread was killed 2023-08-02 18:56:43 11 [Note] Master 'm1': Slave SQL thread exiting, replication stopped in log 'mysqld-bin.000001' at position 1256, master: 127.0.0.1:16300 2023-08-02 18:56:43 10 [Note] Master 'm1': Slave I/O thread exiting, read up to log 'mysqld-bin.000001', position 1256, master 127.0.0.1:16300 ==35746==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55e36abca6b6 in strend /home/buildbot/amd64-debian-11-msan/build/strings/strend.c:45:3 #1 0x55e368204382 in get_one_variable(THD*, st_mysql_show_var const*, enum_var_type, enum_mysql_show_type, system_status_var*, charset_info_st const**, char*, unsigned long*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc #2 0x55e36822abfc in show_status_array(THD*, char const*, st_mysql_show_var*, enum_var_type, system_status_var*, char const*, TABLE*, bool, Item*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc:3897:14 #3 0x55e3682296ff in fill_variables(THD*, TABLE_LIST*, Item*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc:7996:8 #4 0x55e368234bec in get_schema_tables_result(JOIN*, enum_schema_table_state) /home/buildbot/amd64-debian-11-msan/build/sql/sql_show.cc:8941:11 #5 0x55e3680e48e6 in JOIN::exec_inner() /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:4806:7 #6 0x55e3680e208a in JOIN::exec() /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:4627:3 #7 0x55e368055fd2 in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:5107:9 #8 0x55e368054b75 in handle_select(THD*, LEX*, select_result*, unsigned long) /home/buildbot/amd64-debian-11-msan/build/sql/sql_select.cc:584:10 #9 0x55e367f3d91f in execute_sqlcom_select(THD*, TABLE_LIST*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:6288:12 #10 0x55e367f1deff in mysql_execute_command(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:3958:12 #11 0x55e367bc941d in sp_instr_stmt::exec_core(THD*, unsigned int*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:3854:12 #12 0x55e367bc47d7 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:3579:17 #13 0x55e367bc742d in sp_instr_stmt::execute(THD*, unsigned int*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:3760:25 #14 0x55e367ba47ad in sp_head::execute(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:1456:20 #15 0x55e367bad0de in sp_head::execute_procedure(THD*, List<Item>*) /home/buildbot/amd64-debian-11-msan/build/sql/sp_head.cc:2443:5 #16 0x55e367f17f0c in do_execute_sp(THD*, sp_head*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:3034:16 #17 0x55e367f17077 in Sql_cmd_call::execute(THD*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:3280:9 #18 0x55e367f1dc01 in mysql_execute_command(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:6024:26 #19 0x55e367f06db9 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:8060:18 #20 0x55e367efd570 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:1894:7 #21 0x55e367f084e2 in do_command(THD*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_parse.cc:1407:17 #22 0x55e368542e86 in do_handle_one_connection(CONNECT*, bool) /home/buildbot/amd64-debian-11-msan/build/sql/sql_connect.cc:1416:11 #23 0x55e368542489 in handle_one_connection /home/buildbot/amd64-debian-11-msan/build/sql/sql_connect.cc:1318:5 #24 0x55e36979fede in pfs_spawn_thread /home/buildbot/amd64-debian-11-msan/build/storage/perfschema/pfs.cc:2201:3 #25 0x7fcb764eeea6 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7ea6) (BuildId: 255e355c207aba91a59ae1f808e3b4da443abf0c) #26 0x7fcb75ef6a2e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfba2e) (BuildId: e15ec78d51a522023f9cfc58dc284f379d81860b)   Uninitialized value was created by a heap deallocation #0 0x55e36795672c in free (/home/buildbot/amd64-debian-11-msan/build/sql/mariadbd+0x86872c) (BuildId: d58deefb908e1a6bd640a02927dc493177d22787) #1 0x55e368a6fba6 in mysql_close_free_options /home/buildbot/amd64-debian-11-msan/build/sql-common/client.c:3290:3 The reason why I got interested in this test was that I got this local test failure yesterday for a 10.6 based branch: multi_source.mdev-9544 'innodb' w30 [ pass ] 2264   MTR's internal check of the test case 'multi_source.mdev-9544' failed. … -CHARACTER_SETS_DIR /mariadb/10.6/sql/share/charsets/ +CHARACTER_SETS_DIR ???????????????????????????????????????? This would not be flagged as a failure by mtr , so I would guess it to be rather common on buildbot as well.

            I encountered this yesterday again:

            10.6 625a150a86ce6da15fb5c3c39924723c9933bc51

            multi_source.mdev-9544 'innodb'          w20 [ pass ]   2275
             
            MTR's internal check of the test case 'multi_source.mdev-9544' failed.
            …
            -CHARACTER_SETS_DIR	/mariadb/10.6/sql/share/charsets/
            +CHARACTER_SETS_DIR	????????????????????????????????????????
            

            marko Marko Mäkelä added a comment - I encountered this yesterday again: 10.6 625a150a86ce6da15fb5c3c39924723c9933bc51 multi_source.mdev-9544 'innodb' w20 [ pass ] 2275   MTR's internal check of the test case 'multi_source.mdev-9544' failed. … -CHARACTER_SETS_DIR /mariadb/10.6/sql/share/charsets/ +CHARACTER_SETS_DIR ????????????????????????????????????????

            This is a real serious issue in the server code that needs to be fixed, and there's already a perfect analysis by Sujatha from 3 years ago.
            I was able to reproduce easily with valgrind by injecting a small sleep and repeat-running the test:

            diff --git a/sql-common/client.c b/sql-common/client.c
            index 687f28cea28..ad40b5ce680 100644
            --- a/sql-common/client.c
            +++ b/sql-common/client.c
            @@ -1748,6 +1748,7 @@ mysql_set_character_set_with_default_collation(MYSQL *mysql)
               if (mysql->options.charset_dir)
                 charsets_dir=mysql->options.charset_dir;
             
            +  my_sleep(rand() % 250000);
               if ((mysql->charset= get_charset_by_csname(mysql->options.charset_name,
                                                          MY_CS_PRIMARY, MYF(MY_WME))))
               {
            

            knielsen Kristian Nielsen added a comment - This is a real serious issue in the server code that needs to be fixed, and there's already a perfect analysis by Sujatha from 3 years ago. I was able to reproduce easily with valgrind by injecting a small sleep and repeat-running the test: diff --git a/sql-common/client.c b/sql-common/client.c index 687f28cea28..ad40b5ce680 100644 --- a/sql-common/client.c +++ b/sql-common/client.c @@ -1748,6 +1748,7 @@ mysql_set_character_set_with_default_collation(MYSQL *mysql) if (mysql->options.charset_dir) charsets_dir=mysql->options.charset_dir; + my_sleep(rand() % 250000); if ((mysql->charset= get_charset_by_csname(mysql->options.charset_name, MY_CS_PRIMARY, MYF(MY_WME)))) {

            I have a fix for this in st-10.5-knielsen, but I'm blocked from pushing by broken branch protection on github. Feel free to pull the branch if desired.

            knielsen Kristian Nielsen added a comment - I have a fix for this in st-10.5-knielsen, but I'm blocked from pushing by broken branch protection on github. Feel free to pull the branch if desired.

            Pushed to 10.5.

            knielsen Kristian Nielsen added a comment - Pushed to 10.5.

            People

              knielsen Kristian Nielsen
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.