Nayuta Yanagisawa (Inactive) added a comment - 2022-06-08 11:21 09177eadc39ae1e777ad473970456cb9dd9c3993 (ASAN) ==69742==ERROR: AddressSanitizer: heap-use-after-free on address 0x61e000010a30 at pc 0x7fc514b2d2d8 bp 0x7fc514e7f1d0 sp 0x7fc514e7f1c0 READ of size 4 at 0x61e000010a30 thread T27 #0 0x7fc514b2d2d7 in ha_spider::external_lock(THD*, int) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/ha_spider.cc:1316 #1 0x5622862a5bf1 in handler::ha_external_lock(THD*, int) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/handler.cc:6526 #2 0x5622865a5fe0 in lock_external /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/lock.cc:393 #3 0x5622865a5775 in mysql_lock_tables(THD*, st_mysql_lock*, unsigned int) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/lock.cc:338 #4 0x5622859f3e64 in mysql_ha_read(THD*, TABLE_LIST*, enum_ha_read_modes, char const*, List<Item>*, ha_rkey_function, Item*, unsigned long long, unsigned long long) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_handler.cc:821 #5 0x562285ac8a34 in mysql_execute_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:5676 #6 0x562285ad80e7 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:7995 #7 0x562285aae178 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:1857 #8 0x562285aaabfd in do_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:1378 #9 0x562285eb0f83 in do_handle_one_connection(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_connect.cc:1420 #10 0x562285eb0700 in handle_one_connection /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_connect.cc:1316 #11 0x562286b5903c in pfs_spawn_thread /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/perfschema/pfs.cc:1869 #12 0x7fc52b988b42 (/lib/x86_64-linux-gnu/libc.so.6+0x94b42) #13 0x7fc52ba1a9ff (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff)   0x61e000010a30 is located 432 bytes inside of 2676-byte region [0x61e000010880,0x61e0000112f4) freed by thread T27 here: #0 0x7fc52c0d6517 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127 #1 0x5622877274e7 in free_memory /home/nayuta_mariadb/repo/mariadb-server/10.4/mysys/safemalloc.c:279 #2 0x562287726a8c in sf_free /home/nayuta_mariadb/repo/mariadb-server/10.4/mysys/safemalloc.c:197 #3 0x5622876f4c7d in my_free /home/nayuta_mariadb/repo/mariadb-server/10.4/mysys/my_malloc.c:222 #4 0x7fc514afb61b in spider_free_mem(st_spider_transaction*, void*, unsigned long) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_malloc.cc:188 #5 0x7fc514a23856 in spider_free_conn(st_spider_conn*) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_conn.cc:1252 #6 0x7fc514a1cd51 in spider_free_conn_from_trx(st_spider_transaction*, st_spider_conn*, bool, bool, int*) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_conn.cc:315 #7 0x7fc51495d5d3 in spider_free_trx_conn(st_spider_transaction*, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_trx.cc:117 #8 0x7fc51497ad87 in spider_rollback(handlerton*, THD*, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_trx.cc:3564 #9 0x56228627fa0c in ha_rollback_trans(THD*, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/handler.cc:1942 #10 0x562285eee55e in trans_rollback_stmt(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/transaction.cc:496 #11 0x5622859f58f8 in mysql_ha_read(THD*, TABLE_LIST*, enum_ha_read_modes, char const*, List<Item>*, ha_rkey_function, Item*, unsigned long long, unsigned long long) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_handler.cc:1008 #12 0x562285ac8a34 in mysql_execute_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:5676 #13 0x562285ad80e7 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:7995 #14 0x562285aae178 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:1857 #15 0x562285aaabfd in do_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:1378 #16 0x562285eb0f83 in do_handle_one_connection(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_connect.cc:1420 #17 0x562285eb0700 in handle_one_connection /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_connect.cc:1316 #18 0x562286b5903c in pfs_spawn_thread /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/perfschema/pfs.cc:1869 #19 0x7fc52b988b42 (/lib/x86_64-linux-gnu/libc.so.6+0x94b42)   previously allocated by thread T27 here: #0 0x7fc52c0d6867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145 #1 0x562287726433 in sf_malloc /home/nayuta_mariadb/repo/mariadb-server/10.4/mysys/safemalloc.c:118 #2 0x5622876f4144 in my_malloc /home/nayuta_mariadb/repo/mariadb-server/10.4/mysys/my_malloc.c:101 #3 0x7fc514afbd7f in spider_bulk_alloc_mem(st_spider_transaction*, unsigned int, char const*, char const*, unsigned long, unsigned long, ...) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_malloc.cc:236 #4 0x7fc514a1d9a7 in spider_create_conn(st_spider_share*, ha_spider*, int, int, unsigned int, int*) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_conn.cc:478 #5 0x7fc514a22783 in spider_get_conn(st_spider_share*, int, char*, st_spider_transaction*, ha_spider*, bool, bool, unsigned int, int*) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_conn.cc:1080 #6 0x7fc514a8e0c0 in spider_get_share(char const*, TABLE*, THD*, ha_spider*, int*) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/spd_table.cc:5052 #7 0x7fc514b22cad in ha_spider::open(char const*, int, unsigned int) /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/ha_spider.cc:360 #8 0x562286286731 in handler::ha_open(TABLE*, char const*, int, unsigned int, st_mem_root*, List<String>*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/handler.cc:2811 #9 0x562285dddeb6 in open_table_from_share(THD*, TABLE_SHARE*, st_mysql_const_lex_string const*, unsigned int, unsigned int, unsigned int, TABLE*, bool, List<String>*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/table.cc:4096 #10 0x56228591fb25 in open_table(THD*, TABLE_LIST*, Open_table_context*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_base.cc:2108 #11 0x562285929562 in open_and_process_table /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_base.cc:3907 #12 0x56228592c14d in open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_base.cc:4388 #13 0x562285900e11 in open_tables(THD*, TABLE_LIST**, unsigned int*, unsigned int) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_base.h:473 #14 0x5622859f05b9 in mysql_ha_open(THD*, TABLE_LIST*, SQL_HANDLER*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_handler.cc:348 #15 0x562285ac8636 in mysql_execute_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:5661 #16 0x562285ad80e7 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:7995 #17 0x562285aae178 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:1857 #18 0x562285aaabfd in do_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_parse.cc:1378 #19 0x562285eb0f83 in do_handle_one_connection(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_connect.cc:1420 #20 0x562285eb0700 in handle_one_connection /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/sql_connect.cc:1316 #21 0x562286b5903c in pfs_spawn_thread /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/perfschema/pfs.cc:1869 #22 0x7fc52b988b42 (/lib/x86_64-linux-gnu/libc.so.6+0x94b42)   Thread T27 created by T0 here: #0 0x7fc52c07a685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216 #1 0x562286b5942d in spawn_thread_v1 /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/perfschema/pfs.cc:1919 #2 0x5622857a5e3f in inline_mysql_thread_create /home/nayuta_mariadb/repo/mariadb-server/10.4/include/mysql/psi/mysql_thread.h:1275 #3 0x5622857be5ef in create_thread_to_handle_connection(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/mysqld.cc:6282 #4 0x5622857beda4 in create_new_thread(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/mysqld.cc:6352 #5 0x5622857bf293 in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/mysqld.cc:6450 #6 0x5622857c0161 in handle_connections_sockets() /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/mysqld.cc:6608 #7 0x5622857bdca9 in mysqld_main(int, char**) /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/mysqld.cc:5940 #8 0x5622857a408c in main /home/nayuta_mariadb/repo/mariadb-server/10.4/sql/main.cc:25 #9 0x7fc52b91dd8f (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f)   SUMMARY: AddressSanitizer: heap-use-after-free /home/nayuta_mariadb/repo/mariadb-server/10.4/storage/spider/ha_spider.cc:1316 in ha_spider::external_lock(THD*, int) Shadow bytes around the buggy address: 0x0c3c7fffa0f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c3c7fffa100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c3c7fffa110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c3c7fffa120: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c3c7fffa130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd =>0x0c3c7fffa140: fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd 0x0c3c7fffa150: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c3c7fffa160: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c3c7fffa170: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c3c7fffa180: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c3c7fffa190: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==69742==ABORTING

Roel Van de Paar added a comment - 2023-02-24 07:00 - edited Please also test any fixes with: INSTALL PLUGIN Spider SONAME 'ha_spider.so' ; CREATE TABLE t (c INT KEY ,c2 INT ) ENGINE=Spider PARTITION BY LIST (c) (PARTITION p VALUES IN (1,2)); LOCK TABLES t AS a1 WRITE,t AS a4 READ ,t3 AS a0 READ ; HANDLER t OPEN ; HANDLER t READ NEXT ;

Roel Van de Paar added a comment - 2023-02-24 07:04 Both testcases do not seem to produce the heap-use-after-free anymore on a 10.11 ASAN debug build at revision d186cb180e424fb4e166959145b3bccb5e7f5164

Yuchen Pei added a comment - 2023-10-12 01:03 - edited I could not reproduce either 12701 (ER_SPIDER_REMOTE_SERVER_GONE_AWAY_NUM) or ASAN heap-use-after-free for 10.4 6400b199acdaabf50b99cbd5ae70caac783f7e15 or 11.0 5e2d08b5e89ec600f46021d99beeb2635eef4f45. In both cases I get 1429 (ER_CONNECT_TO_FOREIGN_DATA_SOURCE) at HANDLER t READ FIRST; , which is probably caused by spider connecting to remote server using default connection values, and there's no server at the default socket /tmp/mysql.sock . I also tried the new test case in a previous comment [1] that contains a LOCK TABLES statement at the same 10.4 and 11.0 commits, and both cases pass with an --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE at the LOCK and HANDLER t READ NEXT; statements.

Roel Van de Paar added a comment - 2023-10-14 03:46 - edited Confirmed that the ASAN heap-use-after-free is gone in all versions/build types. The Got error 12701 when reading table 't' error in the error log is still present in 10.4 to 11.3 (rev 905c3d61e18ae6222d0d195c43d335046eec65d9 tested, 27 Sep build)

Roel Van de Paar added a comment - 2023-10-14 03:48 - edited The original test can be used unaltered in MTR. MTR will indeed output Error 1429 Unable to connect to foreign data source: localhost however, check the error log in var/log/mysqld.1.err and it will have the 12701 output somewhere in the log towards the end: 11.3.0 905c3d61e18ae6222d0d195c43d335046eec65d9 2023-10-14 14:46:57 4 [ERROR] mysql_ha_read: Got error 12701 when reading table 't' Both CLI and MTR have same outcome: 1429 in client/MTR, 12701 in error log.

Yuchen Pei added a comment - 2023-10-18 02:12 The ASAN issue no longer exists. Splitted out MDEV-32506 regarding error 12701.