[MDEV-30191] SIGSEGV & heap-use-after-free in spider_db_print_item_type, SIGABRT in __cxa_pure_virtual/spider_db_print_item_type, Got error 128 "Out of memory in engine", 56/112 memory not freed, and Assertion `fixed()' failed in Item_sp_variable::val_str on SP call Created: 2022-12-10  Updated: 2023-01-19  Resolved: 2023-01-19

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Spider, Stored routines
Affects Version/s: 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11
Fix Version/s: 10.11.2, 11.0.1, 10.5.19, 10.6.12, 10.7.8, 10.8.7, 10.9.5, 10.10.3

Type: Bug Priority: Critical
Reporter: Roel Van de Paar Assignee: Yuchen Pei
Resolution: Fixed Votes: 1
Labels: ASAN, corruption, not-10.4, regression-10.5

Issue Links:
Duplicate
Issue split
split from MDEV-27438 Spider: crash in spider_sys_open_tabl... Open
Problem/Incident
is caused by MDEV-19002 Partition performance optimization Stalled
Relates
relates to MDEV-26541 Undefined symbol: _ZTI12ha_partition ... Closed

 Description   

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
CREATE USER Spider@localhost IDENTIFIED BY 'PWD123';
CREATE SERVER s FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '../socket.sock',DATABASE'',USER 'Spider',PASSWORD 'PWD123');
CREATE TABLE t (c INT) ENGINE=InnoDB;
CREATE TABLE ts (c INT) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "s",TABLE "t"';
DELIMITER $$
CREATE PROCEDURE sp() BEGIN
DECLARE v1 DATE; SELECT c FROM ts;
WHILE EXISTS (SELECT 1 FROM ts WHERE c>v1 AND c<=v1) DO SELECT st.c; END WHILE;
WHILE EXISTS (SELECT 1 FROM ts WHERE c<v1 AND EXISTS (SELECT 1 FROM t WHERE ts.c=t.c)) DO SELECT ts.c; DELETE ts FROM ts; END WHILE; 
END $$
DELIMITER ;
CALL sp();

Leads to:

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Optimized)

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-opt/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x145774da7700 (LWP 1274163))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000014578d6c9859 in __GI_abort () at abort.c:79
#2  0x000014578da88911 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x000014578da9438c in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x000014578da943f7 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x000014578da95155 in __cxa_pure_virtual () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x0000145774cafb15 in spider_db_print_item_type (item=0x1457340b0a38, field=0x0, spider=0x14573407d0f0, str=0x145734084770, alias=0x0, alias_length=0, dbton_id=0, use_fields=false, fields=0x0) at /test/10.11_opt/storage/spider/spd_db_conn.cc:7384
#7  0x0000145774d19f88 in spider_mbase_handler::append_condition (this=0x145734084710, str=0x145734084770, alias=0x0, alias_length=0, start_where=false, sql_type=<optimized out>) at /test/10.11_opt/storage/spider/spd_db_mysql.cc:11040
#8  0x0000145774cf2661 in ha_spider::append_condition_sql_part (this=0x14573407d0f0, alias=0x0, alias_length=0, sql_type=sql_type@entry=1, test_flg=test_flg@entry=false) at /test/10.11_opt/storage/spider/ha_spider.cc:10977
#9  0x0000145774cb0461 in spider_db_append_condition (spider=spider@entry=0x14573407d0f0, alias=alias@entry=0x0, alias_length=alias_length@entry=0, test_flg=test_flg@entry=false) at /test/10.11_opt/storage/spider/spd_db_conn.cc:8248
#10 0x0000145774cf62e9 in ha_spider::rnd_next_internal (this=0x14573407d0f0, buf=0x14573404cdc8 "\377") at /test/10.11_opt/storage/spider/ha_spider.cc:5669
#11 0x000055f4b2ac08b6 in handler::ha_rnd_next (this=0x14573407d0f0, buf=0x14573404cdc8 "\377") at /test/10.11_opt/sql/handler.cc:3414
#12 0x000055f4b27a67e4 in rr_sequential (info=0x1457340b0378) at /test/10.11_opt/sql/records.h:82
#13 0x000055f4b28bfd72 in sub_select (end_of_records=false, join_tab=0x1457340b02b0, join=0x14573407b120) at /test/10.11_opt/sql/sql_select.cc:21841
#14 sub_select (join=0x14573407b120, join_tab=0x1457340b02b0, end_of_records=<optimized out>) at /test/10.11_opt/sql/sql_select.cc:21771
#15 0x000055f4b28eec51 in do_select (procedure=<optimized out>, join=0x14573407b120) at /test/10.11_opt/sql/sql_select.cc:21387
#16 JOIN::exec_inner (this=0x14573407b120) at /test/10.11_opt/sql/sql_select.cc:4824
#17 0x000055f4b28ef018 in JOIN::exec (this=0x14573407b120) at /test/10.11_opt/sql/sql_select.cc:4602
#18 0x000055f4b2b93186 in subselect_single_select_engine::exec (this=0x145734070b58) at /test/10.11_opt/sql/item_subselect.cc:4101
#19 0x000055f4b2b927dc in Item_subselect::exec (this=0x1457340709b8) at /test/10.11_opt/sql/item_subselect.cc:811
#20 0x000055f4b2b92535 in Item_exists_subselect::val_bool (this=0x1457340709b8) at /test/10.11_opt/sql/item_subselect.cc:1853
#21 0x000055f4b27c2033 in sp_instr_jump_if_not::exec_core (this=0x145734070be0, thd=<optimized out>, nextp=0x145774da5484) at /test/10.11_opt/sql/sp_head.cc:4161
#22 0x000055f4b27c96f6 in sp_lex_keeper::reset_lex_and_exec_core (this=0x145734070c38, thd=0x145734000c58, nextp=<optimized out>, open_tables=<optimized out>, instr=0x145734070be0) at /test/10.11_opt/sql/sp_head.cc:3582
#23 0x000055f4b27c3d36 in sp_head::execute (this=0x14573405cf90, thd=0x145734000c58, merge_da_on_success=true) at /test/10.11_opt/sql/sp_head.cc:1459
#24 0x000055f4b27c575a in sp_head::execute_procedure (this=0x14573405cf90, thd=thd@entry=0x145734000c58, args=0x145734005bd8) at /test/10.11_opt/sql/sp_head.cc:2446
#25 0x000055f4b286eb97 in do_execute_sp (thd=thd@entry=0x145734000c58, sp=sp@entry=0x14573405cf90) at /test/10.11_opt/sql/sql_parse.cc:3026
#26 0x000055f4b28736e6 in Sql_cmd_call::execute (this=0x1457340107f8, thd=0x145734000c58) at /test/10.11_opt/sql/sql_parse.cc:3271
#27 0x000055f4b28794d6 in mysql_execute_command (thd=0x145734000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.11_opt/sql/sql_parse.cc:5999
#28 0x000055f4b2869da5 in mysql_parse (rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>, thd=0x145734000c58) at /test/10.11_opt/sql/sql_parse.cc:7998
#29 mysql_parse (thd=0x145734000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.11_opt/sql/sql_parse.cc:7920
#30 0x000055f4b2875aea in dispatch_command (command=COM_QUERY, thd=0x145734000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.11_opt/sql/sql_class.h:1346
#31 0x000055f4b28778d2 in do_command (thd=0x145734000c58, blocking=blocking@entry=true) at /test/10.11_opt/sql/sql_parse.cc:1407
#32 0x000055f4b2990e8f in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55f4b5316d88, put_in_cache=put_in_cache@entry=true) at /test/10.11_opt/sql/sql_connect.cc:1415
#33 0x000055f4b299116d in handle_one_connection (arg=0x55f4b5316d88) at /test/10.11_opt/sql/sql_connect.cc:1317
#34 0x000014578dbda609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#35 0x000014578d7c6133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Debug)

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000014deaa3edefb in spider_db_print_item_type (item=0x14de700ec0e0, 
    field=field@entry=0x0, spider=0x14de70099450, 
    str=str@entry=0x14de700a5b40, alias=alias@entry=0x0, 
    alias_length=alias_length@entry=0, dbton_id=0, use_fields=false, 
    fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:7384
7384	  switch (item->type())
[Current thread is 1 (Thread 0x14deaa4fe700 (LWP 1803669))]
(gdb) bt
#0  0x000014deaa3edefb in spider_db_print_item_type (item=0x14de700ec0e0, field=field@entry=0x0, spider=0x14de70099450, str=str@entry=0x14de700a5b40, alias=alias@entry=0x0, alias_length=alias_length@entry=0, dbton_id=0, use_fields=false, fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:7384
#1  0x000014deaa46c133 in spider_mbase_handler::append_condition (this=this@entry=0x14de700a5ae0, str=str@entry=0x14de700a5b40, alias=alias@entry=0x0, alias_length=alias_length@entry=0, start_where=false, sql_type=sql_type@entry=1) at /test/10.11_dbg/storage/spider/spd_db_mysql.cc:11040
#2  0x000014deaa46c287 in spider_mbase_handler::append_condition_part (this=0x14de700a5ae0, alias=0x0, alias_length=0, sql_type=1, test_flg=<optimized out>) at /test/10.11_dbg/storage/spider/spd_db_mysql.cc:10996
#3  0x000014deaa43a2da in ha_spider::append_condition_sql_part (this=0x14de70099450, alias=0x0, alias_length=0, sql_type=sql_type@entry=1, test_flg=test_flg@entry=false) at /test/10.11_dbg/storage/spider/ha_spider.cc:10977
#4  0x000014deaa3ee65c in spider_db_append_condition (spider=spider@entry=0x14de70099450, alias=alias@entry=0x0, alias_length=alias_length@entry=0, test_flg=test_flg@entry=false) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:8248
#5  0x000014deaa43f43c in ha_spider::rnd_next_internal (this=this@entry=0x14de70099450, buf=buf@entry=0x14de70061748 "\377") at /test/10.11_dbg/storage/spider/ha_spider.cc:5669
#6  0x000014deaa4400f8 in ha_spider::rnd_next (this=0x14de70099450, buf=0x14de70061748 "\377") at /test/10.11_dbg/storage/spider/ha_spider.cc:5963
#7  0x0000562911e1148a in handler::ha_rnd_next (this=0x14de70099450, buf=0x14de70061748 "\377") at /test/10.11_dbg/sql/handler.cc:3414
#8  0x0000562911a44cbf in rr_sequential (info=0x14de700eb958) at /test/10.11_dbg/sql/records.h:82
#9  0x0000562911b9ef37 in READ_RECORD::read_record (this=0x14de700eb958) at /test/10.11_dbg/sql/records.h:81
#10 join_init_read_record (tab=0x14de700eb890) at /test/10.11_dbg/sql/sql_select.cc:22838
#11 0x0000562911b85e81 in sub_select (join=0x14de70096440, join_tab=0x14de700eb890, end_of_records=false) at /test/10.11_dbg/sql/sql_select.cc:21841
#12 0x0000562911bbaf0d in do_select (procedure=<optimized out>, join=0x14de70096440) at /test/10.11_dbg/sql/sql_select.cc:21387
#13 JOIN::exec_inner (this=this@entry=0x14de70096440) at /test/10.11_dbg/sql/sql_select.cc:4824
#14 0x0000562911bbb3f6 in JOIN::exec (this=0x14de70096440) at /test/10.11_dbg/sql/sql_select.cc:4602
#15 0x0000562911f1b20c in subselect_single_select_engine::exec (this=0x14de7008be78) at /test/10.11_dbg/sql/item_subselect.cc:4101
#16 0x0000562911f1a85a in Item_subselect::exec (this=0x14de7008bcd8) at /test/10.11_dbg/sql/item_subselect.cc:811
#17 0x0000562911f1974d in Item_exists_subselect::val_bool (this=0x14de7008bcd8) at /test/10.11_dbg/sql/item_subselect.cc:1853
#18 0x0000562911a68aff in sp_instr_jump_if_not::exec_core (this=0x14de7008bf00, thd=<optimized out>, nextp=0x14deaa4fc364) at /test/10.11_dbg/sql/sp_head.cc:4161
#19 0x0000562911a72012 in sp_lex_keeper::reset_lex_and_exec_core (this=0x14de7008bf58, thd=0x14de70000d48, nextp=<optimized out>, open_tables=open_tables@entry=true, instr=<optimized out>) at /test/10.11_dbg/sql/sp_head.cc:3582
#20 0x0000562911a72b53 in sp_instr_jump_if_not::execute (this=<optimized out>, thd=<optimized out>, nextp=<optimized out>) at /test/10.11_dbg/sql/sp_head.cc:4143
#21 0x0000562911a6ac8e in sp_head::execute (this=this@entry=0x14de700782b0, thd=thd@entry=0x14de70000d48, merge_da_on_success=merge_da_on_success@entry=true) at /test/10.11_dbg/sql/sp_head.cc:1459
#22 0x0000562911a6cf23 in sp_head::execute_procedure (this=0x14de700782b0, thd=thd@entry=0x14de70000d48, args=0x14de70005e88) at /test/10.11_dbg/sql/sp_head.cc:2446
#23 0x0000562911b26973 in do_execute_sp (thd=thd@entry=0x14de70000d48, sp=sp@entry=0x14de700782b0) at /test/10.11_dbg/sql/sql_parse.cc:3026
#24 0x0000562911b2bece in Sql_cmd_call::execute (this=0x14de700131c8, thd=0x14de70000d48) at /test/10.11_dbg/sql/sql_parse.cc:3271
#25 0x0000562911b38a03 in mysql_execute_command (thd=thd@entry=0x14de70000d48, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.11_dbg/sql/sql_parse.cc:5999
#26 0x0000562911b215a6 in mysql_parse (thd=thd@entry=0x14de70000d48, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14deaa4fd300) at /test/10.11_dbg/sql/sql_parse.cc:7998
#27 0x0000562911b2eae1 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14de70000d48, packet=packet@entry=0x14de7000adf9 "", packet_length=packet_length@entry=9, blocking=blocking@entry=true) at /test/10.11_dbg/sql/sql_class.h:1346
#28 0x0000562911b30f1f in do_command (thd=0x14de70000d48, blocking=blocking@entry=true) at /test/10.11_dbg/sql/sql_parse.cc:1407
#29 0x0000562911c8bb27 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x5629148fd928, put_in_cache=put_in_cache@entry=true) at /test/10.11_dbg/sql/sql_connect.cc:1415
#30 0x0000562911c8bff6 in handle_one_connection (arg=0x5629148fd928) at /test/10.11_dbg/sql/sql_connect.cc:1317
#31 0x000014dec7785609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#32 0x000014dec7371133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Bug confirmed present in:
MariaDB: 10.5.19 (dbg), 10.5.19 (opt), 10.6.12 (dbg), 10.6.12 (opt), 10.7.8 (dbg), 10.7.8 (opt), 10.8.7 (dbg), 10.8.7 (opt), 10.9.5 (dbg), 10.9.5 (opt), 10.10.3 (dbg), 10.10.3 (opt), 10.11.2 (dbg), 10.11.2 (opt)

Bug (or feature/syntax) confirmed not present in:
MariaDB: 10.4.28 (dbg), 10.4.28 (opt)

Observed UniqueID's for this testcase accross versions:

SIGABRT|__cxa_pure_virtual|spider_db_print_item_type|spider_mbase_handler::append_condition|ha_spider::append_condition_sql_part
SIGSEGV|spider_db_print_item_type|spider_mbase_handler::append_condition|spider_mbase_handler::append_condition_part|ha_spider::append_condition_sql_part



 Comments   
Comment by Roel Van de Paar [ 2022-12-10 ]

Additional stack (in debug) + memory-not-freed (in optimized) with this testcase:
* Note that the only text/clause removed from the testcase above is AND c<=v1
* Note that the InnoDB table is being referenced directly
* ASAN did not show any ASAN-observed issue for this testcase (though the mariadbd-observed 'memory not freed' persisted in the error log)
* UBSAN could not be tested due to MDEV-26541

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
CREATE USER Spider@localhost IDENTIFIED BY 'PWD123';
CREATE SERVER s FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '../socket.sock',DATABASE'',USER 'Spider',PASSWORD 'PWD123');
CREATE TABLE t (c INT) ENGINE=InnoDB;
CREATE TABLE ts (c INT) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "s",TABLE "t"';
DELIMITER $$
CREATE PROCEDURE sp() BEGIN
DECLARE v1 DATE; SELECT c FROM ts;
WHILE EXISTS (SELECT 1 FROM ts WHERE c>v1) DO SELECT st.c; END WHILE;
WHILE EXISTS (SELECT 1 FROM ts WHERE c<v1 AND EXISTS (SELECT 1 FROM t WHERE ts.c=t.c)) DO SELECT ts.c; DELETE ts FROM ts; END WHILE; 
END $$
DELIMITER ;
CALL sp();

Leads to:
(The memory loss size (112) is consistent in all versions 10.5-10.11, optimized builds)

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Optimized)

2022-12-10 13:51:13 0 [Note] /test/MD291122-mariadb-10.11.2-linux-x86_64-opt/bin/mysqld: Shutdown complete
 
Warning: Memory not freed: 112

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Debug)

mysqld: /test/10.11_dbg/sql/item.cc:1697: virtual String* Item_sp_variable::val_str(String*): Assertion `fixed()' failed.

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Debug)

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-dbg/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x14bda805f700 (LWP 2293929))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x000014bdbedc7859 in __GI_abort () at abort.c:79
#2  0x000014bdbedc7729 in __assert_fail_base (fmt=0x14bdbef5d588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x5569bb03a087 "fixed()", file=0x5569bb1ef9c4 "/test/10.11_dbg/sql/item.cc", line=1697, function=<optimized out>) at assert.c:92
#3  0x000014bdbedd8fd6 in __GI___assert_fail (assertion=assertion@entry=0x5569bb03a087 "fixed()", file=file@entry=0x5569bb1ef9c4 "/test/10.11_dbg/sql/item.cc", line=line@entry=1697, function=function@entry=0x5569bb1f01b8 "virtual String* Item_sp_variable::val_str(String*)") at assert.c:101
#4  0x00005569ba798bf5 in Item_sp_variable::val_str (this=this@entry=0x14bd6c07bf60, sp=0x14bda805bfe8) at /test/10.11_dbg/sql/item.h:1072
#5  0x000014bda214550f in spider_db_open_item_string (item=item@entry=0x14bd6c07bf60, field=field@entry=0x0, spider=spider@entry=0x14bd6c099330, str=str@entry=0x14bd6c0a5960, alias=alias@entry=0x0, alias_length=alias_length@entry=0, dbton_id=0, use_fields=false, fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:7906
#6  0x000014bda21460a8 in spider_db_print_item_type (item=0x14bd6c07bf60, field=field@entry=0x0, spider=spider@entry=0x14bd6c099330, str=str@entry=0x14bd6c0a5960, alias=alias@entry=0x0, alias_length=alias_length@entry=0, dbton_id=0, use_fields=false, fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:7409
#7  0x000014bda21b9506 in spider_db_mbase_util::open_item_func (this=0x14bda220b150 <spider_db_mysql_utility>, item_func=<optimized out>, spider=0x14bd6c099330, str=<optimized out>, alias=0x0, alias_length=0, use_fields=false, fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_mysql.cc:6636
#8  0x000014bda2144cca in spider_db_open_item_func (item_func=item_func@entry=0x14bd6c07de20, spider=spider@entry=0x14bd6c099330, str=str@entry=0x14bd6c0a5960, alias=alias@entry=0x0, alias_length=alias_length@entry=0, dbton_id=dbton_id@entry=0, use_fields=false, fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:7556
#9  0x000014bda2145f3f in spider_db_print_item_type (item=0x14bd6c07de20, field=field@entry=0x0, spider=0x14bd6c099330, str=str@entry=0x14bd6c0a5960, alias=alias@entry=0x0, alias_length=alias_length@entry=0, dbton_id=0, use_fields=false, fields=0x0) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:7387
#10 0x000014bda21c4133 in spider_mbase_handler::append_condition (this=this@entry=0x14bd6c0a5900, str=str@entry=0x14bd6c0a5960, alias=alias@entry=0x0, alias_length=alias_length@entry=0, start_where=false, sql_type=sql_type@entry=1) at /test/10.11_dbg/storage/spider/spd_db_mysql.cc:11040
#11 0x000014bda21c4287 in spider_mbase_handler::append_condition_part (this=0x14bd6c0a5900, alias=0x0, alias_length=0, sql_type=1, test_flg=<optimized out>) at /test/10.11_dbg/storage/spider/spd_db_mysql.cc:10996
#12 0x000014bda21922da in ha_spider::append_condition_sql_part (this=0x14bd6c099330, alias=0x0, alias_length=0, sql_type=sql_type@entry=1, test_flg=test_flg@entry=false) at /test/10.11_dbg/storage/spider/ha_spider.cc:10977
#13 0x000014bda214665c in spider_db_append_condition (spider=spider@entry=0x14bd6c099330, alias=alias@entry=0x0, alias_length=alias_length@entry=0, test_flg=test_flg@entry=false) at /test/10.11_dbg/storage/spider/spd_db_conn.cc:8248
#14 0x000014bda219743c in ha_spider::rnd_next_internal (this=this@entry=0x14bd6c099330, buf=buf@entry=0x14bd6c061628 "\377") at /test/10.11_dbg/storage/spider/ha_spider.cc:5669
#15 0x000014bda21980f8 in ha_spider::rnd_next (this=0x14bd6c099330, buf=0x14bd6c061628 "\377") at /test/10.11_dbg/storage/spider/ha_spider.cc:5963
#16 0x00005569ba78348a in handler::ha_rnd_next (this=0x14bd6c099330, buf=0x14bd6c061628 "\377") at /test/10.11_dbg/sql/handler.cc:3414
#17 0x00005569ba3b6cbf in rr_sequential (info=0x14bd6c0eb7e8) at /test/10.11_dbg/sql/records.h:82
#18 0x00005569ba510f37 in READ_RECORD::read_record (this=0x14bd6c0eb7e8) at /test/10.11_dbg/sql/records.h:81
#19 join_init_read_record (tab=0x14bd6c0eb720) at /test/10.11_dbg/sql/sql_select.cc:22838
#20 0x00005569ba4f7e81 in sub_select (join=0x14bd6c096320, join_tab=0x14bd6c0eb720, end_of_records=false) at /test/10.11_dbg/sql/sql_select.cc:21841
#21 0x00005569ba52cf0d in do_select (procedure=<optimized out>, join=0x14bd6c096320) at /test/10.11_dbg/sql/sql_select.cc:21387
#22 JOIN::exec_inner (this=this@entry=0x14bd6c096320) at /test/10.11_dbg/sql/sql_select.cc:4824
#23 0x00005569ba52d3f6 in JOIN::exec (this=0x14bd6c096320) at /test/10.11_dbg/sql/sql_select.cc:4602
#24 0x00005569ba88d20c in subselect_single_select_engine::exec (this=0x14bd6c08c048) at /test/10.11_dbg/sql/item_subselect.cc:4101
#25 0x00005569ba88c85a in Item_subselect::exec (this=0x14bd6c08bea8) at /test/10.11_dbg/sql/item_subselect.cc:811
#26 0x00005569ba88b74d in Item_exists_subselect::val_bool (this=0x14bd6c08bea8) at /test/10.11_dbg/sql/item_subselect.cc:1853
#27 0x00005569ba3daaff in sp_instr_jump_if_not::exec_core (this=0x14bd6c08c0d0, thd=<optimized out>, nextp=0x14bda805d364) at /test/10.11_dbg/sql/sp_head.cc:4161
#28 0x00005569ba3e4012 in sp_lex_keeper::reset_lex_and_exec_core (this=0x14bd6c08c128, thd=0x14bd6c000d48, nextp=<optimized out>, open_tables=open_tables@entry=true, instr=<optimized out>) at /test/10.11_dbg/sql/sp_head.cc:3582
#29 0x00005569ba3e4b53 in sp_instr_jump_if_not::execute (this=<optimized out>, thd=<optimized out>, nextp=<optimized out>) at /test/10.11_dbg/sql/sp_head.cc:4143
#30 0x00005569ba3dcc8e in sp_head::execute (this=this@entry=0x14bd6c078190, thd=thd@entry=0x14bd6c000d48, merge_da_on_success=merge_da_on_success@entry=true) at /test/10.11_dbg/sql/sp_head.cc:1459
#31 0x00005569ba3def23 in sp_head::execute_procedure (this=0x14bd6c078190, thd=thd@entry=0x14bd6c000d48, args=0x14bd6c005e88) at /test/10.11_dbg/sql/sp_head.cc:2446
#32 0x00005569ba498973 in do_execute_sp (thd=thd@entry=0x14bd6c000d48, sp=sp@entry=0x14bd6c078190) at /test/10.11_dbg/sql/sql_parse.cc:3026
#33 0x00005569ba49dece in Sql_cmd_call::execute (this=0x14bd6c0131c8, thd=0x14bd6c000d48) at /test/10.11_dbg/sql/sql_parse.cc:3271
#34 0x00005569ba4aaa03 in mysql_execute_command (thd=thd@entry=0x14bd6c000d48, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false) at /test/10.11_dbg/sql/sql_parse.cc:5999
#35 0x00005569ba4935a6 in mysql_parse (thd=thd@entry=0x14bd6c000d48, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14bda805e300) at /test/10.11_dbg/sql/sql_parse.cc:7998
#36 0x00005569ba4a0ae1 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14bd6c000d48, packet=packet@entry=0x14bd6c00adf9 "", packet_length=packet_length@entry=9, blocking=blocking@entry=true) at /test/10.11_dbg/sql/sql_class.h:1346
#37 0x00005569ba4a2f1f in do_command (thd=0x14bd6c000d48, blocking=blocking@entry=true) at /test/10.11_dbg/sql/sql_parse.cc:1407
#38 0x00005569ba5fdb27 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x5569bd6b8408, put_in_cache=put_in_cache@entry=true) at /test/10.11_dbg/sql/sql_connect.cc:1415
#39 0x00005569ba5fdff6 in handle_one_connection (arg=0x5569bd6b8408) at /test/10.11_dbg/sql/sql_connect.cc:1317
#40 0x000014bdbf2d8609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#41 0x000014bdbeec4133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Bug confirmed present in:
MariaDB: 10.5.19 (dbg), 10.5.19 (opt), 10.6.12 (dbg), 10.6.12 (opt), 10.7.8 (dbg), 10.7.8 (opt), 10.8.7 (dbg), 10.8.7 (opt), 10.9.5 (dbg), 10.9.5 (opt), 10.10.3 (dbg), 10.10.3 (opt), 10.11.2 (dbg), 10.11.2 (opt)

Bug (or feature/syntax) confirmed not present in:
MariaDB: 10.4.28 (dbg), 10.4.28 (opt)

Observed UniqueID's for this testcase accross versions:

MEMORY_NOT_FREED|Warning: Memory not freed
fixed()|SIGABRT|Item_sp_variable::val_str|spider_db_open_item_string|spider_db_print_item_type|spider_db_mbase_util::open_item_func
fixed|SIGABRT|Item_sp_variable::val_str|spider_db_open_item_string|spider_db_print_item_type|spider_db_mbase_util::open_item_func

Comment by Roel Van de Paar [ 2022-12-10 ]

Possibly related to MDEV-29447.
Please also check MDEV-29357, MDEV-29173 and MDEV-27966 which have similar asserts and refer to *val_str.

Comment by Roel Van de Paar [ 2022-12-10 ]

Again making only a small change from the original testcase: DATE to INT for the DECLARE in the SP, leads to a different outcome:

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
CREATE USER Spider@localhost IDENTIFIED BY 'PWD123';
CREATE SERVER s FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '../socket.sock',DATABASE'',USER 'Spider',PASSWORD 'PWD123');
CREATE TABLE t (c INT) ENGINE=InnoDB;
CREATE TABLE ts (c INT) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "s",TABLE "t"';
DELIMITER $$
CREATE PROCEDURE sp() BEGIN
DECLARE v1 INT; SELECT c FROM ts;
WHILE EXISTS (SELECT 1 FROM ts WHERE c>v1 AND c<=v1) DO SELECT st.c; END WHILE;
WHILE EXISTS (SELECT 1 FROM ts WHERE c<v1 AND EXISTS (SELECT 1 FROM t WHERE ts.c=t.c)) DO SELECT ts.c; DELETE ts FROM ts; END WHILE; 
END $$
DELIMITER ;
CALL sp();

Leads to:

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Debug)

10.11.2-dbg>CALL sp();
Empty set (0.005 sec)
 
ERROR 1030 (HY000): Got error 128 "Out of memory in engine" from storage engine SPIDER

Without crashing. The error log, on shutdown, will show 56 bytes lost for all versions 10.5-10.11:

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Debug)

2022-12-10 15:41:29 0 [Note] /test/MD291122-mariadb-10.11.2-linux-x86_64-dbg/bin/mysqld: Shutdown complete
 
Warning: Memory not freed: 56

Comment by Roel Van de Paar [ 2022-12-10 ]

In connection with this comment, noting that the bug reproduces even when --spider_support_xa=OFF (or =0) is used.

Comment by Roel Van de Paar [ 2022-12-10 ]

There is one additional stack which can be produced with this testcase:

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS(SOCKET '../socket.sock',DATABASE'',USER'',PASSWORD'');
CREATE TABLE t(c INT,c2 CHAR,c3 DATE) ENGINE=InnoDB;
CREATE TABLE st(c INT,c2 CHAR,c3 DATE,c4 CHAR) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "srv",TABLE "t"';
DELIMITER $$
CREATE PROCEDURE sp() BEGIN
DECLARE v1 DATE;
SELECT c,c2,c3 FROM st;
WHILE EXISTS(SELECT 1 FROM st WHERE c3>v1 AND c3<=v1 AND NOT EXISTS(SELECT * FROM t a WHERE a.c=st.c)) DO SELECT st.c,st.c2,st.c3; END WHILE;
WHILE EXISTS(SELECT 1 FROM st WHERE c3<v1 AND EXISTS(SELECT * FROM t a WHERE st.c=a.c)) DO SELECT st.c; DELETE st FROM st; END WHILE;
END $$
DELIMITER ;
CALL sp();

Leads to:

10.11.2 936436ef437c73911c18854a8ce8dad1216331b8 (Optimized)

Core was generated by `/test/MD291122-mariadb-10.11.2-linux-x86_64-opt/bin/mysqld --no-defaults --core'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000150028852b0f in spider_db_print_item_type (item=0x14fff40b52a8, 
    field=0x0, spider=0x14fff40809f0, str=0x14fff4088560, alias=0x0, 
    alias_length=0, dbton_id=0, use_fields=false, fields=0x0)
    at /test/10.11_opt/storage/spider/spd_db_conn.cc:7384
7384	  switch (item->type())
[Current thread is 1 (Thread 0x15002894a700 (LWP 3355515))]
(gdb) bt
#0  0x0000150028852b0f in spider_db_print_item_type (item=0x14fff40b52a8, field=0x0, spider=0x14fff40809f0, str=0x14fff4088560, alias=0x0, alias_length=0, dbton_id=0, use_fields=false, fields=0x0) at /test/10.11_opt/storage/spider/spd_db_conn.cc:7384
#1  0x00001500288bcf88 in spider_mbase_handler::append_condition (this=0x14fff4088500, str=0x14fff4088560, alias=0x0, alias_length=0, start_where=false, sql_type=<optimized out>) at /test/10.11_opt/storage/spider/spd_db_mysql.cc:11040
#2  0x0000150028895661 in ha_spider::append_condition_sql_part (this=0x14fff40809f0, alias=0x0, alias_length=0, sql_type=sql_type@entry=1, test_flg=test_flg@entry=false) at /test/10.11_opt/storage/spider/ha_spider.cc:10977
#3  0x0000150028853461 in spider_db_append_condition (spider=spider@entry=0x14fff40809f0, alias=alias@entry=0x0, alias_length=alias_length@entry=0, test_flg=test_flg@entry=false) at /test/10.11_opt/storage/spider/spd_db_conn.cc:8248
#4  0x00001500288992e9 in ha_spider::rnd_next_internal (this=0x14fff40809f0, buf=0x14fff404a3c8 "\377") at /test/10.11_opt/storage/spider/ha_spider.cc:5669
#5  0x0000562d7a90d8b6 in handler::ha_rnd_next (this=0x14fff40809f0, buf=0x14fff404a3c8 "\377") at /test/10.11_opt/sql/handler.cc:3414
#6  0x0000562d7a5f37e4 in rr_sequential (info=0x14fff407f178) at /test/10.11_opt/sql/records.h:82
#7  0x0000562d7a70cd72 in sub_select (end_of_records=false, join_tab=0x14fff407f0b0, join=0x14fff40b3db0) at /test/10.11_opt/sql/sql_select.cc:21841
#8  sub_select (join=0x14fff40b3db0, join_tab=0x14fff407f0b0, end_of_records=<optimized out>) at /test/10.11_opt/sql/sql_select.cc:21771
#9  0x0000562d7a73bc51 in do_select (procedure=<optimized out>, join=0x14fff40b3db0) at /test/10.11_opt/sql/sql_select.cc:21387
#10 JOIN::exec_inner (this=0x14fff40b3db0) at /test/10.11_opt/sql/sql_select.cc:4824
#11 0x0000562d7a73c018 in JOIN::exec (this=0x14fff40b3db0) at /test/10.11_opt/sql/sql_select.cc:4602
#12 0x0000562d7a9e0186 in subselect_single_select_engine::exec (this=0x14fff40710b0) at /test/10.11_opt/sql/item_subselect.cc:4101
#13 0x0000562d7a9df7dc in Item_subselect::exec (this=0x14fff4070f10) at /test/10.11_opt/sql/item_subselect.cc:811
#14 0x0000562d7a9df535 in Item_exists_subselect::val_bool (this=0x14fff4070f10) at /test/10.11_opt/sql/item_subselect.cc:1853
#15 0x0000562d7a60f033 in sp_instr_jump_if_not::exec_core (this=0x14fff40710f0, thd=<optimized out>, nextp=0x150028948484) at /test/10.11_opt/sql/sp_head.cc:4161
#16 0x0000562d7a6166f6 in sp_lex_keeper::reset_lex_and_exec_core (this=0x14fff4071148, thd=0x14fff4000c58, nextp=<optimized out>, open_tables=<optimized out>, instr=0x14fff40710f0) at /test/10.11_opt/sql/sp_head.cc:3582
#17 0x0000562d7a610d36 in sp_head::execute (this=0x14fff405a870, thd=0x14fff4000c58, merge_da_on_success=true) at /test/10.11_opt/sql/sp_head.cc:1459
#18 0x0000562d7a61275a in sp_head::execute_procedure (this=0x14fff405a870, thd=thd@entry=0x14fff4000c58, args=0x14fff4005bd8) at /test/10.11_opt/sql/sp_head.cc:2446
#19 0x0000562d7a6bbb97 in do_execute_sp (thd=thd@entry=0x14fff4000c58, sp=sp@entry=0x14fff405a870) at /test/10.11_opt/sql/sql_parse.cc:3026
#20 0x0000562d7a6c06e6 in Sql_cmd_call::execute (this=0x14fff40107f8, thd=0x14fff4000c58) at /test/10.11_opt/sql/sql_parse.cc:3271
#21 0x0000562d7a6c64d6 in mysql_execute_command (thd=0x14fff4000c58, is_called_from_prepared_stmt=<optimized out>) at /test/10.11_opt/sql/sql_parse.cc:5999
#22 0x0000562d7a6b6da5 in mysql_parse (rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>, thd=0x14fff4000c58) at /test/10.11_opt/sql/sql_parse.cc:7998
#23 mysql_parse (thd=0x14fff4000c58, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/10.11_opt/sql/sql_parse.cc:7920
#24 0x0000562d7a6c2aea in dispatch_command (command=COM_QUERY, thd=0x14fff4000c58, packet=<optimized out>, packet_length=<optimized out>, blocking=<optimized out>) at /test/10.11_opt/sql/sql_class.h:1346
#25 0x0000562d7a6c48d2 in do_command (thd=0x14fff4000c58, blocking=blocking@entry=true) at /test/10.11_opt/sql/sql_parse.cc:1407
#26 0x0000562d7a7dde8f in do_handle_one_connection (connect=<optimized out>, connect@entry=0x562d7dc9f4c8, put_in_cache=put_in_cache@entry=true) at /test/10.11_opt/sql/sql_connect.cc:1415
#27 0x0000562d7a7de16d in handle_one_connection (arg=0x562d7dc9f4c8) at /test/10.11_opt/sql/sql_connect.cc:1317
#28 0x0000150041778609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#29 0x0000150041364133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Observed UniqueID's for this testcase accross versions 10.5-10.11:

SIGABRT|__cxa_pure_virtual|spider_db_print_item_type|spider_mbase_handler::append_condition|ha_spider::append_condition_sql_part
SIGSEGV|spider_db_print_item_type|spider_mbase_handler::append_condition|ha_spider::append_condition_sql_part|spider_db_append_condition
SIGSEGV|spider_db_print_item_type|spider_mbase_handler::append_condition|spider_mbase_handler::append_condition_part|ha_spider::append_condition_sql_part

Comment by Roel Van de Paar [ 2022-12-10 ]

The last testcase produces most stacks seen. Here is an MTR version of the same:

--source include/have_innodb.inc
--let $SOCKET=`SELECT @@global.socket`
INSTALL PLUGIN Spider SONAME 'ha_spider.so';
CREATE USER Spider@localhost IDENTIFIED BY '';
GRANT ALL ON test.* TO Spider@localhost;
eval CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS(SOCKET "$SOCKET",DATABASE'test',USER'Spider',PASSWORD'');
CREATE TABLE t(c INT,c2 CHAR,c3 DATE) ENGINE=InnoDB;
CREATE TABLE st(c INT,c2 CHAR,c3 DATE,c4 CHAR) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "srv",TABLE "t"';
--delimiter $$
CREATE PROCEDURE sp() BEGIN
DECLARE v1 DATE;
SELECT c,c2,c3 FROM st;
WHILE EXISTS(SELECT 1 FROM st WHERE c3>v1 AND c3<=v1 AND NOT EXISTS(SELECT * FROM t a WHERE a.c=st.c)) DO SELECT st.c,st.c2,st.c3; END WHILE;
WHILE EXISTS(SELECT 1 FROM st WHERE c3<v1 AND EXISTS(SELECT * FROM t a WHERE st.c=a.c)) DO SELECT st.c; DELETE st FROM st; END WHILE;
END $$
--delimiter ;
CALL sp();

Comment by Roel Van de Paar [ 2022-12-10 ]

Using the testcases, I was able to reproduce a heap-use-after-free in ASAN:

10.11.1 d186cb180e424fb4e166959145b3bccb5e7f5164 (Debug, ASAN)

==2417502==ERROR: AddressSanitizer: heap-use-after-free on address 0x625000369948 at pc 0x1479aee3c94f bp 0x1479af161d90 sp 0x1479af161d80
READ of size 8 at 0x625000369948 thread T12
    #0 0x1479aee3c94e in spider_db_print_item_type(Item*, Field*, ha_spider*, spider_string*, char const*, unsigned int, unsigned int, bool, spider_fields*) /test/10.11_dbg_san/storage/spider/spd_db_conn.cc:7384
    #1 0x1479aef9f950 in spider_mbase_handler::append_condition(spider_string*, char const*, unsigned int, bool, unsigned long) /test/10.11_dbg_san/storage/spider/spd_db_mysql.cc:11040
    #2 0x1479aef9fb3c in spider_mbase_handler::append_condition_part(char const*, unsigned int, unsigned long, bool) /test/10.11_dbg_san/storage/spider/spd_db_mysql.cc:10996
    #3 0x1479aef17864 in ha_spider::append_condition_sql_part(char const*, unsigned int, unsigned long, bool) /test/10.11_dbg_san/storage/spider/ha_spider.cc:10981
    #4 0x1479aee3d66a in spider_db_append_condition(ha_spider*, char const*, unsigned int, bool) /test/10.11_dbg_san/storage/spider/spd_db_conn.cc:8248
    #5 0x1479aef222f7 in ha_spider::rnd_next_internal(unsigned char*) /test/10.11_dbg_san/storage/spider/ha_spider.cc:5671
    #6 0x1479aef24907 in ha_spider::rnd_next(unsigned char*) /test/10.11_dbg_san/storage/spider/ha_spider.cc:5965
    #7 0x560c2b9223a7 in handler::ha_rnd_next(unsigned char*) /test/10.11_dbg_san/sql/handler.cc:3414
    #8 0x560c2ace29fc in rr_sequential(READ_RECORD*) /test/10.11_dbg_san/sql/records.cc:519
    #9 0x560c2b13b2c3 in READ_RECORD::read_record() /test/10.11_dbg_san/sql/records.h:81
    #10 0x560c2b13b2c3 in join_init_read_record(st_join_table*) /test/10.11_dbg_san/sql/sql_select.cc:22810
    #11 0x560c2b0db257 in sub_select(JOIN*, st_join_table*, bool) /test/10.11_dbg_san/sql/sql_select.cc:21813
    #12 0x560c2b19e0f9 in do_select /test/10.11_dbg_san/sql/sql_select.cc:21359
    #13 0x560c2b19e0f9 in JOIN::exec_inner() /test/10.11_dbg_san/sql/sql_select.cc:4823
    #14 0x560c2b19f63b in JOIN::exec() /test/10.11_dbg_san/sql/sql_select.cc:4601
    #15 0x560c2bccc459 in subselect_single_select_engine::exec() /test/10.11_dbg_san/sql/item_subselect.cc:4101
    #16 0x560c2bcc9afe in Item_subselect::exec() /test/10.11_dbg_san/sql/item_subselect.cc:811
    #17 0x560c2bcc5dd4 in Item_exists_subselect::val_bool() /test/10.11_dbg_san/sql/item_subselect.cc:1853
    #18 0x560c2ad44a24 in sp_instr_jump_if_not::exec_core(THD*, unsigned int*) /test/10.11_dbg_san/sql/sp_head.cc:4161
    #19 0x560c2ad65c30 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /test/10.11_dbg_san/sql/sp_head.cc:3582
    #20 0x560c2ad686ea in sp_instr_jump_if_not::execute(THD*, unsigned int*) /test/10.11_dbg_san/sql/sp_head.cc:4143
    #21 0x560c2ad4c7b0 in sp_head::execute(THD*, bool) /test/10.11_dbg_san/sql/sp_head.cc:1459
    #22 0x560c2ad52373 in sp_head::execute_procedure(THD*, List<Item>*) /test/10.11_dbg_san/sql/sp_head.cc:2446
    #23 0x560c2afade89 in do_execute_sp /test/10.11_dbg_san/sql/sql_parse.cc:3026
    #24 0x560c2afc1135 in Sql_cmd_call::execute(THD*) /test/10.11_dbg_san/sql/sql_parse.cc:3271
    #25 0x560c2afe514e in mysql_execute_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:5999
    #26 0x560c2af9c671 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.11_dbg_san/sql/sql_parse.cc:7998
    #27 0x560c2afc9d6e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1894
    #28 0x560c2afd0969 in do_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1407
    #29 0x560c2b47c5b0 in do_handle_one_connection(CONNECT*, bool) /test/10.11_dbg_san/sql/sql_connect.cc:1416
    #30 0x560c2b47d0b3 in handle_one_connection /test/10.11_dbg_san/sql/sql_connect.cc:1318
    #31 0x1479d048c608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
    #32 0x1479d006e132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)
 
0x625000369948 is located 4168 bytes inside of 8208-byte region [0x625000368900,0x62500036a910)
freed by thread T12 here:
    #0 0x560c2abc6d2f in __interceptor_free (/test/ASAN_SPIDER_MD151122-mariadb-10.11.1-linux-x86_64-dbg/bin/mariadbd+0xd57d2f)
    #1 0x560c2cdbf541 in my_free /test/10.11_dbg_san/mysys/my_malloc.c:211
    #2 0x560c2cda8ad0 in root_free /test/10.11_dbg_san/mysys/my_alloc.c:78
    #3 0x560c2cdaa275 in free_root /test/10.11_dbg_san/mysys/my_alloc.c:501
    #4 0x560c2ad4d797 in sp_head::execute(THD*, bool) /test/10.11_dbg_san/sql/sp_head.cc:1548
    #5 0x560c2ad52373 in sp_head::execute_procedure(THD*, List<Item>*) /test/10.11_dbg_san/sql/sp_head.cc:2446
    #6 0x560c2afade89 in do_execute_sp /test/10.11_dbg_san/sql/sql_parse.cc:3026
    #7 0x560c2afc1135 in Sql_cmd_call::execute(THD*) /test/10.11_dbg_san/sql/sql_parse.cc:3271
    #8 0x560c2afe514e in mysql_execute_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:5999
    #9 0x560c2af9c671 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.11_dbg_san/sql/sql_parse.cc:7998
    #10 0x560c2afc9d6e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1894
    #11 0x560c2afd0969 in do_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1407
    #12 0x560c2b47c5b0 in do_handle_one_connection(CONNECT*, bool) /test/10.11_dbg_san/sql/sql_connect.cc:1416
    #13 0x560c2b47d0b3 in handle_one_connection /test/10.11_dbg_san/sql/sql_connect.cc:1318
    #14 0x1479d048c608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
previously allocated by thread T12 here:
    #0 0x560c2abc7128 in __interceptor_malloc (/test/ASAN_SPIDER_MD151122-mariadb-10.11.1-linux-x86_64-dbg/bin/mariadbd+0xd58128)
    #1 0x560c2cdbf301 in my_malloc /test/10.11_dbg_san/mysys/my_malloc.c:90
    #2 0x560c2cda89ff in root_alloc /test/10.11_dbg_san/mysys/my_alloc.c:66
    #3 0x560c2cda970f in alloc_root /test/10.11_dbg_san/mysys/my_alloc.c:332
    #4 0x560c2b0c9159 in Query_arena::alloc(unsigned long) /test/10.11_dbg_san/sql/sql_class.h:1194
    #5 0x560c2b0c9159 in JOIN::get_best_combination() /test/10.11_dbg_san/sql/sql_select.cc:11326
    #6 0x560c2b188222 in JOIN::optimize_stage2() /test/10.11_dbg_san/sql/sql_select.cc:2582
    #7 0x560c2b196703 in JOIN::optimize_inner() /test/10.11_dbg_san/sql/sql_select.cc:2561
    #8 0x560c2b1976fc in JOIN::optimize() /test/10.11_dbg_san/sql/sql_select.cc:1871
    #9 0x560c2bccb6a9 in subselect_single_select_engine::exec() /test/10.11_dbg_san/sql/item_subselect.cc:4017
    #10 0x560c2bcc9afe in Item_subselect::exec() /test/10.11_dbg_san/sql/item_subselect.cc:811
    #11 0x560c2bcc5dd4 in Item_exists_subselect::val_bool() /test/10.11_dbg_san/sql/item_subselect.cc:1853
    #12 0x560c2ad44a24 in sp_instr_jump_if_not::exec_core(THD*, unsigned int*) /test/10.11_dbg_san/sql/sp_head.cc:4161
    #13 0x560c2ad65c30 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /test/10.11_dbg_san/sql/sp_head.cc:3582
    #14 0x560c2ad686ea in sp_instr_jump_if_not::execute(THD*, unsigned int*) /test/10.11_dbg_san/sql/sp_head.cc:4143
    #15 0x560c2ad4c7b0 in sp_head::execute(THD*, bool) /test/10.11_dbg_san/sql/sp_head.cc:1459
    #16 0x560c2ad52373 in sp_head::execute_procedure(THD*, List<Item>*) /test/10.11_dbg_san/sql/sp_head.cc:2446
    #17 0x560c2afade89 in do_execute_sp /test/10.11_dbg_san/sql/sql_parse.cc:3026
    #18 0x560c2afc1135 in Sql_cmd_call::execute(THD*) /test/10.11_dbg_san/sql/sql_parse.cc:3271
    #19 0x560c2afe514e in mysql_execute_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:5999
    #20 0x560c2af9c671 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.11_dbg_san/sql/sql_parse.cc:7998
    #21 0x560c2afc9d6e in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1894
    #22 0x560c2afd0969 in do_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1407
    #23 0x560c2b47c5b0 in do_handle_one_connection(CONNECT*, bool) /test/10.11_dbg_san/sql/sql_connect.cc:1416
    #24 0x560c2b47d0b3 in handle_one_connection /test/10.11_dbg_san/sql/sql_connect.cc:1318
    #25 0x1479d048c608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
Thread T12 created by T0 here:
    #0 0x560c2aaf4165 in pthread_create (/test/ASAN_SPIDER_MD151122-mariadb-10.11.1-linux-x86_64-dbg/bin/mariadbd+0xc85165)
    #1 0x560c2ac1354b in create_thread_to_handle_connection(CONNECT*) /test/10.11_dbg_san/sql/mysqld.cc:6102
    #2 0x560c2ac1b838 in create_new_thread(CONNECT*) /test/10.11_dbg_san/sql/mysqld.cc:6161
    #3 0x560c2ac1bd0d in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /test/10.11_dbg_san/sql/mysqld.cc:6223
    #4 0x560c2ac1cd39 in handle_connections_sockets() /test/10.11_dbg_san/sql/mysqld.cc:6347
    #5 0x560c2ac22a94 in mysqld_main(int, char**) /test/10.11_dbg_san/sql/mysqld.cc:5997
    #6 0x560c2ac086ea in main /test/10.11_dbg_san/sql/main.cc:34
    #7 0x1479cff73082 in __libc_start_main ../csu/libc-start.c:308
 
SUMMARY: AddressSanitizer: heap-use-after-free /test/10.11_dbg_san/storage/spider/spd_db_conn.cc:7384 in spider_db_print_item_type(Item*, Field*, ha_spider*, spider_string*, char const*, unsigned int, unsigned int, bool, spider_fields*)
Shadow bytes around the buggy address:
  0x0c4a800652d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a800652e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a800652f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a80065300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a80065310: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c4a80065320: fd fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd
  0x0c4a80065330: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a80065340: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a80065350: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a80065360: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4a80065370: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==2417502==ABORTING

UniqueID:

ASAN|heap-use-after-free|storage/spider/spd_db_conn.cc|spider_db_print_item_type|spider_mbase_handler::append_condition|spider_mbase_handler::append_condition_part|ha_spider::append_condition_sql_part

Comment by Nayuta Yanagisawa (Inactive) [ 2022-12-12 ]

Slightly simplified MTR corresponding to one in the issue description:

--echo #
--echo # MDEV-30191 SIGSEGV & heap-use-after-free in spider_db_print_item_type, SIGABRT in __cxa_pure_virtual/spider_db_print_item_type, Got error 128 "Out of memory in engine", 56/112 memory not freed, and Assertion `fixed()' failed in Item_sp_variable::val_str on SP call
--echo #
 
--disable_query_log
--disable_result_log
--source ../../t/test_init.inc
--enable_result_log
--enable_query_log
 
--connection child2_1
CREATE DATABASE auto_test_remote;
USE auto_test_remote;
CREATE TABLE tbl_a (c INT);
 
--connection master_1
CREATE DATABASE auto_test_local;
USE auto_test_local;
 
eval CREATE TABLE tbl_a (
    c INT
) $MASTER_1_ENGINE $MASTER_1_CHARSET COMMENT='table "tbl_a", srv "s_2_1"';
CREATE TABLE tbl_b (c INT);
 
--delimiter $$
CREATE PROCEDURE sp() BEGIN
    DECLARE v1 DATE;
    WHILE EXISTS (SELECT 1 FROM tbl_a WHERE c>v1 AND c<=v1) DO
        SELECT 1;
    END WHILE;
    WHILE EXISTS (SELECT 1 FROM tbl_a WHERE c<v1 AND EXISTS (SELECT 1 FROM tbl_b WHERE tbl_a.c=tbl_b.c)) DO
        SELECT 1;
    END WHILE; 
END $$
--delimiter ;
CALL sp();
 
--connection master_1
DROP DATABASE IF EXISTS auto_test_local;
--connection child2_1
DROP DATABASE IF EXISTS auto_test_remote;
 
--disable_query_log
--disable_result_log
--source ../t/test_deinit.inc
--enable_query_log
--enable_result_log

Comment by Roel Van de Paar [ 2022-12-13 ]

Simplified "single while" testcase, with credit & thanks to ralf.gebhardt for the original one!

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '../socket.sock',DATABASE'',USER'',PASSWORD'');
CREATE TABLE t1 (c1 INT,c2 CHAR) ENGINE=InnoDB;
CREATE TABLE t2 (c1 INT,c2 CHAR,c3 DATE) ENGINE=InnoDB;
CREATE TABLE ts (c1 INT,c3 DATE) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "srv",TABLE "t2"';
INSERT INTO t2 VALUES (0,'a',0);
DELIMITER $$
CREATE PROCEDURE sp() BEGIN
DECLARE v1 DATE; 
SELECT MAX(c3) INTO v1 FROM ts a;
WHILE EXISTS(SELECT * FROM ts WHERE c3=1 AND NOT EXISTS (SELECT 1 FROM t1 WHERE t1.c2=ts.c3)) DO SELECT 1; END WHILE;
END $$
DELIMITER ;
CALL sp();

Comment by Nayuta Yanagisawa (Inactive) [ 2022-12-14 ]

10.5 (851816532b39b4bf04b1d352cf3c28929ec99cf1)

==301431==ERROR: AddressSanitizer: heap-use-after-free on address 0x625000422890 at pc 0x7fd7fdc194af bp 0x7fd7fe59d2d0 sp 0x7fd7fe59d2c0                                
READ of size 8 at 0x625000422890 thread T16                                                                                                                              
    #0 0x7fd7fdc194ae in spider_db_print_item_type(Item*, Field*, ha_spider*, spider_string*, char const*, unsigned int, unsigned int, bool, spider_fields*) /home/nayuta
_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/spider/spd_db_conn.cc:9612                                                                                       
    #1 0x7fd7fde90380 in spider_mbase_handler::append_condition(spider_string*, char const*, unsigned int, bool, unsigned long) /home/nayuta_mariadb/repo/mariadb-server/
bb-10.5-MDEV-30191/storage/spider/spd_db_mysql.cc:11599                                                                                                                  
    #2 0x7fd7fde8fd9b in spider_mbase_handler::append_condition_part(char const*, unsigned int, unsigned long, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDE
V-30191/storage/spider/spd_db_mysql.cc:11555                                                                                                                             
    #3 0x7fd7fddf36b6 in ha_spider::append_condition_sql_part(char const*, unsigned int, unsigned long, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191
/storage/spider/ha_spider.cc:14968                                                                                                                                       
    #4 0x7fd7fdc20559 in spider_db_append_condition(ha_spider*, char const*, unsigned int, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/spid
er/spd_db_conn.cc:10537                                                                                                                                                  
    #5 0x7fd7fddaa714 in ha_spider::rnd_next_internal(unsigned char*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/spider/ha_spider.cc:7452       
    #6 0x7fd7fddaf2a3 in ha_spider::rnd_next(unsigned char*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/spider/ha_spider.cc:7768         
    #7 0x5623731dcace in handler::ha_rnd_next(unsigned char*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/handler.cc:3095                   [425/624]
    #8 0x562373646c9d in rr_sequential(READ_RECORD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/records.cc:519                                      
    #9 0x5623727cd165 in READ_RECORD::read_record() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/records.h:80                                         
    #10 0x562372aef612 in join_init_read_record(st_join_table*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:22008                      
    #11 0x562372ae86c0 in sub_select(JOIN*, st_join_table*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:21041                    
    #12 0x562372ae66e6 in do_select /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:20574                                                  
    #13 0x562372a70cf6 in JOIN::exec_inner() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:4569                                          
    #14 0x562372a6e2c6 in JOIN::exec() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:4349                                                
    #15 0x56237342cf6f in subselect_single_select_engine::exec() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/item_subselect.cc:4054                  
    #16 0x562373407691 in Item_subselect::exec() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/item_subselect.cc:794                                   
    #17 0x56237341289b in Item_exists_subselect::val_bool() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/item_subselect.cc:1811                       
    #18 0x56237273a2db in sp_instr_jump_if_not::exec_core(THD*, unsigned int*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:4079           
    #19 0x562372735fe7 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp
_head.cc:3500                                                                                                                                                            
    #20 0x56237273a188 in sp_instr_jump_if_not::execute(THD*, unsigned int*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:4061             
    #21 0x562372727dd7 in sp_head::execute(THD*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:1438                                   
    #22 0x56237272e6a2 in sp_head::execute_procedure(THD*, List<Item>*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:2450                  
    #23 0x562372991160 in do_execute_sp /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:3089                                                
    #24 0x562372992d62 in Sql_cmd_call::execute(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:3335                                  
    #25 0x5623729a6b23 in mysql_execute_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:6059                                  
    #26 0x5623729b44b3 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:
8089                                                                                                                                                                     
    #27 0x562372989cae in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sq
l_parse.cc:1891                                                                                                                                                          
    #28 0x5623729865b0 in do_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:1375                                             
    #29 0x562372de0fb1 in do_handle_one_connection(CONNECT*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_connect.cc:1415                   
    #30 0x562372de0943 in handle_one_connection /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_connect.cc:1317                                      
    #31 0x562373a76f51 in pfs_spawn_thread /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/perfschema/pfs.cc:2201                                    
    #32 0x7fd80ca94b42  (/lib/x86_64-linux-gnu/libc.so.6+0x94b42)                                                                                                        
    #33 0x7fd80cb25bb3 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x125bb3)                                                                                             
                                                                                                                                                                         
0x625000422890 is located 3984 bytes inside of 8176-byte region [0x625000421900,0x6250004238f0)                                                                          
freed by thread T16 here:                                                                                                                                                
    #0 0x7fd80d8b4517 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127                                                                  
    #1 0x5623747248c2 in my_free /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/mysys/my_malloc.c:211                                                       
    #2 0x562374700215 in free_root /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/mysys/my_alloc.c:416                                                      
    #3 0x56237272853a in sp_head::execute(THD*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:1525                                    
    #4 0x56237272e6a2 in sp_head::execute_procedure(THD*, List<Item>*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:2450                   
    #5 0x562372991160 in do_execute_sp /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:3089                                                 
    #6 0x562372992d62 in Sql_cmd_call::execute(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:3335                                   
    #7 0x5623729a6b23 in mysql_execute_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:6059                                   
    #8 0x5623729b44b3 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:8
089                                                                                                                                                                      
    #9 0x562372989cae in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql
_parse.cc:1891                                                                                                                                                           
    #10 0x5623729865b0 in do_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:1375                                             
    #11 0x562372de0fb1 in do_handle_one_connection(CONNECT*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_connect.cc:1415                   
    #12 0x562372de0943 in handle_one_connection /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_connect.cc:1317                                      
    #13 0x562373a76f51 in pfs_spawn_thread /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/perfschema/pfs.cc:2201                                    
    #14 0x7fd80ca94b42  (/lib/x86_64-linux-gnu/libc.so.6+0x94b42)                                                                                                        
                                                                                                                                                                         
previously allocated by thread T16 here:                                                                                                                                     #0 0x7fd80d8b4867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145                                                                
    #1 0x562374723a3b in my_malloc /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/mysys/my_malloc.c:90                                                      
    #2 0x5623746ff165 in alloc_root /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/mysys/my_alloc.c:244                                                     
    #3 0x56237268e7a7 in Query_arena::alloc(unsigned long) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_class.h:1177                              
    #4 0x562372a9c102 in JOIN::get_best_combination() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:10614                                
    #5 0x562372a58a85 in JOIN::optimize_stage2() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:2384                                      
    #6 0x562372a584d4 in JOIN::optimize_inner() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:2363                                       
    #7 0x562372a51608 in JOIN::optimize() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_select.cc:1695                                             
    #8 0x56237342bcd2 in subselect_single_select_engine::exec() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/item_subselect.cc:3970                   
    #9 0x562373407691 in Item_subselect::exec() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/item_subselect.cc:794                                    
    #10 0x56237341289b in Item_exists_subselect::val_bool() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/item_subselect.cc:1811                       
    #11 0x56237273a2db in sp_instr_jump_if_not::exec_core(THD*, unsigned int*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:4079           
    #12 0x562372735fe7 in sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp
_head.cc:3500      
    #13 0x56237273a188 in sp_instr_jump_if_not::execute(THD*, unsigned int*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:4061    [359/624]
    #14 0x562372727dd7 in sp_head::execute(THD*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:1438                                   
    #15 0x56237272e6a2 in sp_head::execute_procedure(THD*, List<Item>*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sp_head.cc:2450                  
    #16 0x562372991160 in do_execute_sp /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:3089                                                
    #17 0x562372992d62 in Sql_cmd_call::execute(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:3335                                  
    #18 0x5623729a6b23 in mysql_execute_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:6059                                  
    #19 0x5623729b44b3 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:
8089                                                                                                                                                                     
    #20 0x562372989cae in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sq
l_parse.cc:1891                                                                                                                                                          
    #21 0x5623729865b0 in do_command(THD*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_parse.cc:1375                                             
    #22 0x562372de0fb1 in do_handle_one_connection(CONNECT*, bool) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_connect.cc:1415                   
    #23 0x562372de0943 in handle_one_connection /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/sql_connect.cc:1317                                      
    #24 0x562373a76f51 in pfs_spawn_thread /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/perfschema/pfs.cc:2201                                    
    #25 0x7fd80ca94b42  (/lib/x86_64-linux-gnu/libc.so.6+0x94b42)                                                                                                        
                                                                                                                                                                         
Thread T16 created by T0 here:                                                                                                                                           
    #0 0x7fd80d858685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216                                                        
    #1 0x562373a729d6 in my_thread_create /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/perfschema/my_thread.h:52                                  
    #2 0x562373a77344 in pfs_spawn_thread_v1 /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/perfschema/pfs.cc:2252                                  
    #3 0x562372669642 in inline_mysql_thread_create /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/include/mysql/psi/mysql_thread.h:1323                    
    #4 0x562372680216 in create_thread_to_handle_connection(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/mysqld.cc:6067                     
    #5 0x5623726808ab in create_new_thread(CONNECT*) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/mysqld.cc:6126                                      
    #6 0x562372680c0d in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/mysqld.cc:6191         
    #7 0x562372681858 in handle_connections_sockets() /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/mysqld.cc:6318                                     
    #8 0x56237267f9ea in mysqld_main(int, char**) /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/mysqld.cc:5713                                         
    #9 0x56237266810c in main /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/sql/main.cc:25                                                                 
    #10 0x7fd80ca29d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f)                                                                                                        
                                                                                                                                                                         
SUMMARY: AddressSanitizer: heap-use-after-free /home/nayuta_mariadb/repo/mariadb-server/bb-10.5-MDEV-30191/storage/spider/spd_db_conn.cc:9612 in spider_db_print_item_typ
e(Item*, Field*, ha_spider*, spider_string*, char const*, unsigned int, unsigned int, bool, spider_fields*)                                                              
Shadow bytes around the buggy address:                                                                                                                                   
  0x0c4a8007c4c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c4d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c4e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c4f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c500: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
=>0x0c4a8007c510: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c520: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c530: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c540: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c550: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
  0x0c4a8007c560: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd                                                                                                        
Shadow byte legend (one shadow byte represents 8 application bytes):                                                                                                     
  Addressable:           00                                                                                                                                              
  Partially addressable: 01 02 03 04 05 06 07                                                                                                                            
  Heap left redzone:       fa                                                                                                                                            
  Freed heap region:       fd                                                                                                                                            
  Stack left redzone:      f1                                                                                                                                            
  Stack mid redzone:       f2                                                                                                                                            
  Stack right redzone:     f3                                                                                                                                            
  Stack after return:      f5                                                                                                                                              Stack use after scope:   f8                                                                                                                                            
  Global redzone:          f9                                                                                                                                            
  Global init order:       f6                                                                                                                                            
  Poisoned by user:        f7                                                                                                                                            
  Container overflow:      fc                                                                                                                                            
  Array cookie:            ac                                                                                                                                            
  Intra object redzone:    bb                                                                                                                                            
  ASan internal:           fe                                                                                                                                            
  Left alloca redzone:     ca                                                                                                                                            
  Right alloca redzone:    cb                                                                                                                                            
  Shadow gap:              cc                                                                                                                                            
==301431==ABORTING                                                                                                                                                       
----------SERVER LOG END------------- 

Comment by Nayuta Yanagisawa (Inactive) [ 2022-12-15 ]

I've confirmed that the first bad commit is https://github.com/MariaDB/server/commit/e954d9de886aebc68c39240304fe97ae88276dbb.

Comment by Yuchen Pei [ 2023-01-05 ]

Indeed, the parent commit to e954d9de886aebc68c39240304fe97ae88276dbb passes the test. Let's call that pre-19002.

I have compared the trace of pre-19002 with that of current 10.5. Pre-19002 ha_spider->condition gets emptied in ha_spider::reset:

pre-19002 8e6e5acef15

int ha_spider::reset()
{
...
  quick_mode = FALSE;
  keyread = FALSE;
  ignore_dup_key = FALSE;
  write_can_replace = FALSE;
...
  while (condition)
  {
    tmp_cond = condition->next;
    spider_free(spider_current_trx, condition, MYF(0));
    condition = tmp_cond;
  }

This removes the old condition that may have been freed before ha_spider::cond_push}}ing the new {{condition

This however does not work with 10.5, because the reset code becomes

pre-19002 8e6e5acef15

int ha_spider::reset()
{
...
  if (
    wide_handler &&
    wide_handler->sql_command != SQLCOM_END
  ) {
    wide_handler->sql_command = SQLCOM_END; // [1]
    wide_handler->between_flg = FALSE;
    wide_handler->idx_bitmap_is_set = FALSE;
    wide_handler->rnd_bitmap_is_set = FALSE;
    wide_handler->quick_mode = FALSE;
    wide_handler->keyread = FALSE;
    wide_handler->ignore_dup_key = FALSE;
    wide_handler->write_can_replace = FALSE;
...
    while (wide_handler->condition)
    {
      tmp_cond = wide_handler->condition->next;
      spider_free(spider_current_trx, wide_handler->condition, MYF(0));
      wide_handler->condition = tmp_cond;
    }

This is because the reset function is actually invoked twice, once for each of the conditions c>v1 AND c<=v1 and c<v1 AND EXISTS (SELECT 1 FROM tbl_b WHERE tbl_a.c=tbl_b.c)).

The first run sets wide_handler->sql_command to SQLCOM_END at [1], and the second run skips the if body because wide_handler->sql_command == SQLCOM_END. This causes ha_spider->wide_handler->condition to stay, and later trying to access the freed-up space.

Comment by Yuchen Pei [ 2023-01-10 ]

simply removing the sql_command bits seems to fix the testcase in this issue and all existing spider testcases still pass:

 storage/spider/ha_spider.cc | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
 
diff --git a/storage/spider/ha_spider.cc b/storage/spider/ha_spider.cc
index 9b3fffcf873..4978c8f7844 100644
--- a/storage/spider/ha_spider.cc
+++ b/storage/spider/ha_spider.cc
@@ -1365,11 +1365,8 @@ int ha_spider::reset()
 #endif
   result_list.direct_distinct = FALSE;
   store_error_num = 0;
-  if (
-    wide_handler &&
-    wide_handler->sql_command != SQLCOM_END
-  ) {
-    wide_handler->sql_command = SQLCOM_END;
+  if (wide_handler)
+  {
     wide_handler->between_flg = FALSE;
     wide_handler->idx_bitmap_is_set = FALSE;
     wide_handler->rnd_bitmap_is_set = FALSE;

Is this a good fix? I am not sure yet. What do you think holyfoot?

Comment by Yuchen Pei [ 2023-01-12 ]

holyfoot PTAL https://github.com/MariaDB/server/commit/c6c005fc5e6

Comment by Alexey Botchkov [ 2023-01-17 ]

ok to push.

Comment by Yuchen Pei [ 2023-01-19 ]

Thanks holyfoot for the review, pushed.

Generated at Thu Feb 08 10:14:23 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.