Roel Van de Paar added a comment - 2020-05-19 09:25 - edited Issue may be sporadic. Just repeat the testcase a few times if so.

Roel Van de Paar added a comment - 2022-04-22 04:29 Updated test report SET GLOBAL KEYCACHE1.key_cache_segments=7; SET GLOBAL KEYCACHE1.key_buffer_size=1*1024*1024; SET GLOBAL KEYCACHE1.key_buffer_size=0; SET GLOBAL KEYCACHE1.key_buffer_size=1*1024*1024; Leads to: 10.9.0 ef930dcad58ae6c3f334a32bd63e26c65fd66fa6 (Debug) Core was generated by `/test/MD050422-mariadb-10.9.0-linux-x86_64-dbg/bin/mysqld --no-defaults --core-'. Program terminated with signal SIGSEGV, Segmentation fault. #0 flush_all_key_blocks (keycache=0x151b0401a468) at /test/10.9_dbg/mysys/mf_keycache.c:4463 [Current thread is 1 (Thread 0x151b4c147700 (LWP 1966671))] (gdb) bt #0 flush_all_key_blocks (keycache=0x151b0401a468) at /test/10.9_dbg/mysys/mf_keycache.c:4463 #1 prepare_resize_simple_key_cache (keycache=0x151b0401a468, release_lock=release_lock@entry=1 '\001') at /test/10.9_dbg/mysys/mf_keycache.c:709 #2 0x0000560977700e02 in resize_partitioned_key_cache (keycache=0x151b0401a2d8, key_cache_block_size=1024, use_mem=1048576, division_limit=100, age_threshold=300, changed_blocks_hash_size=0) at /test/10.9_dbg/mysys/mf_keycache.c:5279 #3 0x0000560977704b89 in resize_key_cache (keycache=keycache@entry=0x151b0401a0e8, key_cache_block_size=key_cache_block_size@entry=1024, use_mem=use_mem@entry=1048576, division_limit=division_limit@entry=100, age_threshold=age_threshold@entry=300, changed_blocks_hash_size=changed_blocks_hash_size@entry=0) at /test/10.9_dbg/mysys/mf_keycache.c:6067 #4 0x0000560976f0c109 in ha_resize_key_cache (key_cache=key_cache@entry=0x151b0401a0e8) at /test/10.9_dbg/sql/handler.cc:5975 #5 0x0000560976d88bd8 in update_buffer_size (thd=<optimized out>, key_cache=0x151b0401a0e8, offset=<optimized out>, new_value=1048576) at /test/10.9_dbg/sql/sys_vars.inl:1126 #6 0x0000560976d9a614 in Sys_var_keycache::global_update (this=0x5609782769a0 <Sys_key_buffer_size>, thd=0x151b04000db8, var=<optimized out>) at /test/10.9_dbg/sql/sys_vars.inl:1071 #7 0x0000560976aeb2cc in sys_var::update (this=0x5609782769a0 <Sys_key_buffer_size>, thd=0x151b04000db8, var=0x151b040141b0) at /test/10.9_dbg/sql/set_var.cc:207 #8 0x0000560976aeb7f9 in set_var::update (this=<optimized out>, thd=<optimized out>) at /test/10.9_dbg/sql/set_var.cc:863 #9 0x0000560976aecb3f in sql_set_variables (thd=thd@entry=0x151b04000db8, var_list=var_list@entry=0x151b040060d8, free=free@entry=true) at /test/10.9_dbg/sql/set_var.cc:745 #10 0x0000560976bf4a30 in mysql_execute_command (thd=thd@entry=0x151b04000db8, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.9_dbg/sql/sql_parse.cc:5049 #11 0x0000560976bdde23 in mysql_parse (thd=thd@entry=0x151b04000db8, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x151b4c146400) at /test/10.9_dbg/sql/sql_parse.cc:8043 #12 0x0000560976beca23 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x151b04000db8, packet=packet@entry=0x151b0400b889 "SET GLOBAL KEYCACHE1.key_buffer_size=1*1024*1024", packet_length=packet_length@entry=48, blocking=blocking@entry=true) at /test/10.9_dbg/sql/sql_class.h:1362 #13 0x0000560976befe74 in do_command (thd=0x151b04000db8, blocking=blocking@entry=true) at /test/10.9_dbg/sql/sql_parse.cc:1407 #14 0x0000560976d6cd2a in do_handle_one_connection (connect=<optimized out>, connect@entry=0x560979b5e568, put_in_cache=put_in_cache@entry=true) at /test/10.9_dbg/sql/sql_connect.cc:1418 #15 0x0000560976d6d32f in handle_one_connection (arg=arg@entry=0x560979b5e568) at /test/10.9_dbg/sql/sql_connect.cc:1312 #16 0x00005609771f77a5 in pfs_spawn_thread (arg=0x560979a9e978) at /test/10.9_dbg/storage/perfschema/pfs.cc:2201 #17 0x0000151b60ee3609 in start_thread (arg=<optimized out>) at pthread_create.c:477 #18 0x0000151b60acf163 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Bug confirmed present in: MariaDB: 10.2.44 (dbg), 10.2.44 (opt), 10.3.35 (dbg), 10.3.35 (opt), 10.4.25 (dbg), 10.4.25 (opt), 10.5.16 (dbg), 10.5.16 (opt), 10.6.8 (dbg), 10.6.8 (opt), 10.7.4 (dbg), 10.7.4 (opt), 10.8.3 (dbg), 10.8.3 (opt), 10.9.0 (dbg), 10.9.0 (opt) Bug (or feature/syntax) confirmed not present in: MySQL: 5.5.62 (dbg), 5.5.62 (opt), 5.6.51 (dbg), 5.6.51 (opt), 5.7.37 (dbg), 5.7.37 (opt), 8.0.28 (dbg), 8.0.28 (opt)

Roel Van de Paar added a comment - 2022-04-22 04:36 Further input from ASAN: SET GLOBAL KEYCACHE1.key_cache_segments=7; SET GLOBAL KEYCACHE1.key_buffer_size=1*1024*1024; SET GLOBAL KEYCACHE1.key_buffer_size=0; SET GLOBAL KEYCACHE1.key_buffer_size=1*1024*1024; Leads to: 10.9.0 161fd2d29cc2f8390fa3bf7e739c52bc8d5c39df (Optimized) ==2093643==ERROR: AddressSanitizer: heap-use-after-free on address 0x62c000076ab8 at pc 0x55d383ae4a4c bp 0x15089e6ff480 sp 0x15089e6ff470 READ of size 8 at 0x62c000076b28 thread T13 #0 0x5618d1119238 in flush_all_key_blocks /test/10.9_dbg_san/mysys/mf_keycache.c:4456 #1 0x5618d1119238 in prepare_resize_simple_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:709 #2 0x5618d1119e0e in resize_partitioned_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:5279 #3 0x5618d1125067 in resize_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:6067 #4 0x5618ce2817d7 in ha_resize_key_cache(st_key_cache*) /test/10.9_dbg_san/sql/handler.cc:5991 #5 0x5618cd55c06c in update_buffer_size /test/10.9_dbg_san/sql/sys_vars.inl:1126 #6 0x5618cd598e77 in Sys_var_keycache::global_update(THD*, set_var*) /test/10.9_dbg_san/sql/sys_vars.inl:1071 #7 0x5618cc1ffaac in sys_var::update(THD*, set_var*) /test/10.9_dbg_san/sql/set_var.cc:207 #8 0x5618cc202b5c in set_var::update(THD*) /test/10.9_dbg_san/sql/set_var.cc:863 #9 0x5618cc20b555 in sql_set_variables(THD*, List<set_var_base>*, bool) /test/10.9_dbg_san/sql/set_var.cc:745 #10 0x5618cc9fc7ff in mysql_execute_command(THD*, bool) /test/10.9_dbg_san/sql/sql_parse.cc:5049 #11 0x5618cc948728 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.9_dbg_san/sql/sql_parse.cc:8043 #12 0x5618cc9be44e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.9_dbg_san/sql/sql_parse.cc:1910 #13 0x5618cc9d4fa9 in do_command(THD*, bool) /test/10.9_dbg_san/sql/sql_parse.cc:1407 #14 0x5618cd4a1c4b in do_handle_one_connection(CONNECT*, bool) /test/10.9_dbg_san/sql/sql_connect.cc:1418 #15 0x5618cd4a4ae5 in handle_one_connection /test/10.9_dbg_san/sql/sql_connect.cc:1312 #16 0x5618cf9fdc62 in pfs_spawn_thread /test/10.9_dbg_san/storage/perfschema/pfs.cc:2201 #17 0x153d597b9608 in start_thread /build/glibc-sMfBJT/glibc-2.31/nptl/pthread_create.c:477 #18 0x153d58a2e162 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f162)   0x62c000076b28 is located 26920 bytes inside of 28972-byte region [0x62c000070200,0x62c00007732c) freed by thread T13 here: #0 0x5618cbf59e4f in __interceptor_free (/test/UBASAN_MD090422-mariadb-10.9.0-linux-x86_64-dbg/bin/mariadbd+0x849de4f) #1 0x5618d119c88b in free_memory /test/10.9_dbg_san/mysys/safemalloc.c:297 #2 0x5618d119d33a in sf_free /test/10.9_dbg_san/mysys/safemalloc.c:203 #3 0x5618d115dc02 in my_free /test/10.9_dbg_san/mysys/my_malloc.c:211 #4 0x5618d110a516 in end_simple_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:970 #5 0x5618d110ac9f in end_partitioned_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:5376 #6 0x5618d1119f34 in resize_partitioned_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:5274 #7 0x5618d1125067 in resize_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:6067 #8 0x5618ce2817d7 in ha_resize_key_cache(st_key_cache*) /test/10.9_dbg_san/sql/handler.cc:5991 #9 0x5618cd55bc31 in update_buffer_size /test/10.9_dbg_san/sql/sys_vars.inl:1105 #10 0x5618cd598e77 in Sys_var_keycache::global_update(THD*, set_var*) /test/10.9_dbg_san/sql/sys_vars.inl:1071 #11 0x5618cc1ffaac in sys_var::update(THD*, set_var*) /test/10.9_dbg_san/sql/set_var.cc:207 #12 0x5618cc202b5c in set_var::update(THD*) /test/10.9_dbg_san/sql/set_var.cc:863 #13 0x5618cc20b555 in sql_set_variables(THD*, List<set_var_base>*, bool) /test/10.9_dbg_san/sql/set_var.cc:745 #14 0x5618cc9fc7ff in mysql_execute_command(THD*, bool) /test/10.9_dbg_san/sql/sql_parse.cc:5049 #15 0x5618cc948728 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.9_dbg_san/sql/sql_parse.cc:8043 #16 0x5618cc9be44e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.9_dbg_san/sql/sql_parse.cc:1910 #17 0x5618cc9d4fa9 in do_command(THD*, bool) /test/10.9_dbg_san/sql/sql_parse.cc:1407 #18 0x5618cd4a1c4b in do_handle_one_connection(CONNECT*, bool) /test/10.9_dbg_san/sql/sql_connect.cc:1418 #19 0x5618cd4a4ae5 in handle_one_connection /test/10.9_dbg_san/sql/sql_connect.cc:1312 #20 0x5618cf9fdc62 in pfs_spawn_thread /test/10.9_dbg_san/storage/perfschema/pfs.cc:2201 #21 0x153d597b9608 in start_thread /build/glibc-sMfBJT/glibc-2.31/nptl/pthread_create.c:477   previously allocated by thread T13 here: #0 0x5618cbf5a248 in malloc (/test/UBASAN_MD090422-mariadb-10.9.0-linux-x86_64-dbg/bin/mariadbd+0x849e248) #1 0x5618d119cb76 in sf_malloc /test/10.9_dbg_san/mysys/safemalloc.c:126 #2 0x5618d115d4e1 in my_malloc /test/10.9_dbg_san/mysys/my_malloc.c:90 #3 0x5618d1135998 in my_multi_malloc_large /test/10.9_dbg_san/mysys/mulalloc.c:95 #4 0x5618d110b9c2 in init_simple_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:557 #5 0x5618d110ea97 in init_partitioned_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:5142 #6 0x5618d110cdc5 in init_key_cache_internal /test/10.9_dbg_san/mysys/mf_keycache.c:5950 #7 0x5618d1124f16 in init_key_cache /test/10.9_dbg_san/mysys/mf_keycache.c:6010 #8 0x5618ce28133d in ha_init_key_cache /test/10.9_dbg_san/sql/handler.cc:5964 #9 0x5618cd55bf0b in update_buffer_size /test/10.9_dbg_san/sql/sys_vars.inl:1124 #10 0x5618cd598e77 in Sys_var_keycache::global_update(THD*, set_var*) /test/10.9_dbg_san/sql/sys_vars.inl:1071 #11 0x5618cc1ffaac in sys_var::update(THD*, set_var*) /test/10.9_dbg_san/sql/set_var.cc:207 #12 0x5618cc202b5c in set_var::update(THD*) /test/10.9_dbg_san/sql/set_var.cc:863 #13 0x5618cc20b555 in sql_set_variables(THD*, List<set_var_base>*, bool) /test/10.9_dbg_san/sql/set_var.cc:745 #14 0x5618cc9fc7ff in mysql_execute_command(THD*, bool) /test/10.9_dbg_san/sql/sql_parse.cc:5049 #15 0x5618cc948728 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.9_dbg_san/sql/sql_parse.cc:8043 #16 0x5618cc9be44e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.9_dbg_san/sql/sql_parse.cc:1910 #17 0x5618cc9d4fa9 in do_command(THD*, bool) /test/10.9_dbg_san/sql/sql_parse.cc:1407 #18 0x5618cd4a1c4b in do_handle_one_connection(CONNECT*, bool) /test/10.9_dbg_san/sql/sql_connect.cc:1418 #19 0x5618cd4a4ae5 in handle_one_connection /test/10.9_dbg_san/sql/sql_connect.cc:1312 #20 0x5618cf9fdc62 in pfs_spawn_thread /test/10.9_dbg_san/storage/perfschema/pfs.cc:2201 #21 0x153d597b9608 in start_thread /build/glibc-sMfBJT/glibc-2.31/nptl/pthread_create.c:477   Thread T13 created by T0 here: #0 0x5618cbe87285 in __interceptor_pthread_create (/test/UBASAN_MD090422-mariadb-10.9.0-linux-x86_64-dbg/bin/mariadbd+0x83cb285) #1 0x5618cfa0d18c in my_thread_create /test/10.9_dbg_san/storage/perfschema/my_thread.h:52 #2 0x5618cfa0d18c in pfs_spawn_thread_v1 /test/10.9_dbg_san/storage/perfschema/pfs.cc:2252 #3 0x5618cbfb38ac in inline_mysql_thread_create /test/10.9_dbg_san/include/mysql/psi/mysql_thread.h:1139 #4 0x5618cbfb38ac in create_thread_to_handle_connection(CONNECT*) /test/10.9_dbg_san/sql/mysqld.cc:5980 #5 0x5618cbfc8d86 in create_new_thread(CONNECT*) /test/10.9_dbg_san/sql/mysqld.cc:6039 #6 0x5618cbfc9561 in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /test/10.9_dbg_san/sql/mysqld.cc:6101 #7 0x5618cbfcb146 in handle_connections_sockets() /test/10.9_dbg_san/sql/mysqld.cc:6225 #8 0x5618cbfd129c in mysqld_main(int, char**) /test/10.9_dbg_san/sql/mysqld.cc:5875 #9 0x5618cbf9b80a in main /test/10.9_dbg_san/sql/main.cc:34 #10 0x153d589330b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x240b2)   SUMMARY: AddressSanitizer: heap-use-after-free /test/10.9_dbg_san/mysys/mf_keycache.c:4456 in flush_all_key_blocks Shadow bytes around the buggy address: 0x0c5880006d10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006d20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006d30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006d40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006d50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd =>0x0c5880006d60: fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd 0x0c5880006d70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006d80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006d90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006da0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c5880006db0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==2093639==ABORTING Setup: Compiled with GCC >=7.5.0 (I use GCC 9.4.0) and: -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON -DWITH_RAPID=OFF -DWSREP_LIB_WITH_ASAN=ON Set before execution: export ASAN_OPTIONS=quarantine_size_mb=512:atexit=1:detect_invalid_pointer_pairs=3:dump_instruction_bytes=1:abort_on_error=1 Bug confirmed present in: MariaDB: 10.2.44 (dbg), 10.2.44 (opt), 10.3.35 (dbg), 10.3.35 (opt), 10.4.25 (dbg), 10.4.25 (opt), 10.5.16 (dbg), 10.5.16 (opt), 10.6.8 (dbg), 10.6.8 (opt), 10.7.4 (dbg), 10.7.4 (opt), 10.8.3 (dbg), 10.8.3 (opt), 10.9.0 (dbg), 10.9.0 (opt)

Oleksandr Byelkin added a comment - 2022-06-23 12:30 Problem is that keycache is shutdown by assigning size 0, and most structures are freed, but on reize the flush ignore the shutdown status of the cache

Igor Babaev (Inactive) added a comment - 2022-06-24 07:15 Sanja, Please change the comment for something like this: Take into account that in preparation of a simple key cache for resizing no disk blocks might be assigned to it. (Please check spelling anyway.) And mention the reviewer. After this you can push.