Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-35011

SIGSEGV in spider_db_mbase::append_lock_tables (or spider_string::length) on SELECT, ASAN: heap-use-after-free in spider_link_get_key

Details

    Description

      Looping this sporadic testcase till it crashes:

      DROP DATABASE test;
      CREATE DATABASE test;
      USE test;
      INSTALL PLUGIN Spider SONAME 'ha_spider.so';
      CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock', DATABASE 'test', USER 'Spider', PASSWORD '');
      CREATE TABLE t1 (c INT PRIMARY KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "t"';
      CREATE TABLE t2 (c INT KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "tm"';
      CREATE TABLE t3 (e INT, f BLOB) ENGINE=Spider;
      SET GLOBAL default_storage_engine=Spider;
      CREATE TABLE t5 (c1 TINYINT NOT NULL);
      XA START 'xa1';
      SHOW CREATE TABLE t1;
      SET spider_semi_table_lock=1;
      SELECT AVG(c1) AS VALUE FROM t1;
      SET GLOBAL table_open_cache=10;
      EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >=any (SELECT b FROM t2);
      INSERT INTO t2 VALUES (0,0,0,'a','b','c','d');
      UPDATE IGNORE t5 SET c1=NULL WHERE c1>1;
      SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2;
      INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C;
      INSERT INTO t3 VALUES (1,0);
      SELECT HEX(c1),HEX (c2) FROM t5;
      SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;
      

      We see after 200-700 repeats:

      CS 11.2.6 e91a79945822def1452787f825e6047c6a64dbd9 (Debug)

      Core was generated by `/test/MD090924-mariadb-11.2.6-linux-x86_64-dbg/bin/mariadbd --no-defaults --max'.
      Program terminated with signal SIGSEGV, Segmentation fault.
      #0  spider_db_mbase::append_lock_tables (this=0x149cb0067890, str=0x149cb00f3670) at /test/11.2_dbg/storage/spider/spd_db_mysql.cc:3572
       
      [Current thread is 1 (LWP 3655217)]
      (gdb) bt
      #0  spider_db_mbase::append_lock_tables (this=0x149cb0067890, str=0x149cb00f3670) at /test/11.2_dbg/storage/spider/spd_db_mysql.cc:3572
      #1  0x0000149d1ffa8120 in spider_mbase_handler::lock_tables (this=0x149cb00f3610, link_idx=0)at /test/11.2_dbg/storage/spider/spd_db_mysql.cc:13877
      #2  0x0000149d1ff2e2ae in spider_db_lock_tables (spider=spider@entry=0x149cb003b030, link_idx=link_idx@entry=0)at /test/11.2_dbg/storage/spider/spd_db_conn.cc:1271
      #3  0x0000149d1ff96ac7 in ha_spider::lock_tables (this=this@entry=0x149cb003b030)at /test/11.2_dbg/storage/spider/ha_spider.cc:11990
      #4  0x0000149d1ff96f61 in ha_spider::external_lock (this=0x149cb003b030, thd=0x149cb0000d58, lock_type=0)at /test/11.2_dbg/storage/spider/ha_spider.cc:941
      #5  0x0000563cf5f27f71 in handler::ha_external_lock (this=0x149cb003b030, thd=thd@entry=0x149cb0000d58, lock_type=lock_type@entry=0)at /test/11.2_dbg/sql/handler.cc:7445
      #6  0x0000563cf6083118 in lock_external (count=1, tables=0x149cb00154e0, thd=0x149cb0000d58) at /test/11.2_dbg/sql/lock.cc:396
      #7  mysql_lock_tables (thd=thd@entry=0x149cb0000d58, sql_lock=sql_lock@entry=0x149cb00154b0, flags=flags@entry=0)at /test/11.2_dbg/sql/lock.cc:341
      #8  0x0000563cf6083f4b in mysql_lock_tables (thd=thd@entry=0x149cb0000d58, tables=tables@entry=0x149cb00154a8, count=count@entry=1, flags=flags@entry=0) at /test/11.2_dbg/sql/lock.cc:304
      #9  0x0000563cf5b89138 in lock_tables (thd=thd@entry=0x149cb0000d58, tables=0x149cb0013d50, count=<optimized out>, flags=flags@entry=0)at /test/11.2_dbg/sql/sql_base.cc:5917
      #10 0x0000563cf5b8af1c in open_and_lock_tables (thd=thd@entry=0x149cb0000d58, options=<optimized out>, tables=<optimized out>, tables@entry=0x149cb0013d50, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x149d24078880)at /test/11.2_dbg/sql/sql_base.cc:5649
      #11 0x0000563cf5bfde10 in open_and_lock_tables (flags=0, derived=true, tables=0x149cb0013d50, thd=0x149cb0000d58)at /test/11.2_dbg/sql/sql_base.h:531
      #12 execute_sqlcom_select (thd=thd@entry=0x149cb0000d58, all_tables=0x149cb0013d50) at /test/11.2_dbg/sql/sql_parse.cc:6089
      #13 0x0000563cf5c0a07a in mysql_execute_command (thd=thd@entry=0x149cb0000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/11.2_dbg/sql/sql_parse.cc:3984
      #14 0x0000563cf5c10d26 in mysql_parse (thd=thd@entry=0x149cb0000d58, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x149d240792a0)at /test/11.2_dbg/sql/sql_parse.cc:7929
      #15 0x0000563cf5c131bd in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x149cb0000d58, packet=packet@entry=0x149cb000b309 "SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2", packet_length=packet_length@entry=60, blocking=blocking@entry=true) at /test/11.2_dbg/sql/sql_class.h:248
      #16 0x0000563cf5c153e3 in do_command (thd=0x149cb0000d58, blocking=blocking@entry=true) at /test/11.2_dbg/sql/sql_parse.cc:1407
      #17 0x0000563cf5d8135c in do_handle_one_connection (connect=<optimized out>, connect@entry=0x563cf888b448, put_in_cache=put_in_cache@entry=true)at /test/11.2_dbg/sql/sql_connect.cc:1439
      #18 0x0000563cf5d8165c in handle_one_connection (arg=arg@entry=0x563cf888b448)at /test/11.2_dbg/sql/sql_connect.cc:1341
      #19 0x0000563cf61c862c in pfs_spawn_thread (arg=0x563cf88a8a38)at /test/11.2_dbg/storage/perfschema/pfs.cc:2201
      #20 0x0000149d2689ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
      #21 0x0000149d26929c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
      

      A stack which was previously fixed in MDEV-29963.

      Additionally, about 60-70% of the time the crash is not in spider_db_mbase::append_lock_tables but in spider_string::length - as stack previously seen in MDEV-29854 on LOCK TABLES whereas here it is on SELECT:

      CS 11.2.6 e91a79945822def1452787f825e6047c6a64dbd9 (Debug)

      Core was generated by `/test/MD090924-mariadb-11.2.6-linux-x86_64-dbg/bin/mariadbd --no-defaults --max'.
      Program terminated with signal SIGSEGV, Segmentation fault.
      #0  spider_string::length (this=0x1) at /test/11.2_dbg/sql/sql_string.h:355
       
      [Current thread is 1 (LWP 3889885)]
      (gdb) bt
      #0  spider_string::length (this=0x1) at /test/11.2_dbg/sql/sql_string.h:355
      #1  0x000014df50152689 in spider_link_get_key (link_for_hash=0x14def41f1008, length=0x14df50567518, not_used=<optimized out>)at /test/11.2_dbg/storage/spider/spd_table.cc:408
      #2  0x000055c416461656 in my_hash_key (first=1 '\001', length=0x14df50567518, record=<optimized out>, hash=0x14def4284b88)at /test/11.2_dbg/mysys/hash.c:197
      #3  hashcmp (hash=hash@entry=0x14def4284b88, pos=pos@entry=0x14def408cad8, key=key@entry=0x14def40fb408 "`test`.`tm`", length=length@entry=11)at /test/11.2_dbg/mysys/hash.c:380
      #4  0x000055c41646188f in my_hash_first_from_hash_value (hash=0x14def4284b88, hash_value=<optimized out>, key=0x14def40fb408 "`test`.`tm`", length=11, current_record=current_record@entry=0x14df505675bc)at /test/11.2_dbg/mysys/hash.c:291
      #5  0x000055c4164618e0 in my_hash_search_using_hash_value (hash=<optimized out>, hash_value=<optimized out>, key=<optimized out>, length=<optimized out>) at /test/11.2_dbg/mysys/hash.c:245
      #6  0x000014df501a2202 in spider_mbase_handler::append_lock_tables_list (this=0x14def4037070, conn=0x14def4317038, link_idx=0, appended=0x14df5056763c)at /test/11.2_dbg/storage/spider/spd_db_mysql.cc:12760
      #7  0x000014df50196484 in ha_spider::append_lock_tables_list (this=this@entry=0x14def4080040)at /test/11.2_dbg/storage/spider/ha_spider.cc:11915
      #8  0x000014df501968b0 in ha_spider::store_lock (this=0x14def4080040, thd=0x14def4000d58, to=0x14def40154d0, lock_type=TL_READ)at /test/11.2_dbg/storage/spider/ha_spider.cc:820
      #9  0x000055c415e0ebb7 in get_lock_data (thd=thd@entry=0x14def4000d58, table_ptr=table_ptr@entry=0x14def40154a8, count=count@entry=1, flags=flags@entry=3) at /test/11.2_dbg/sql/lock.cc:825
      #10 0x000055c415e0ef31 in mysql_lock_tables (thd=thd@entry=0x14def4000d58, tables=tables@entry=0x14def40154a8, count=count@entry=1, flags=flags@entry=0) at /test/11.2_dbg/sql/lock.cc:301
      #11 0x000055c415914138 in lock_tables (thd=thd@entry=0x14def4000d58, tables=0x14def4013d50, count=<optimized out>, flags=flags@entry=0)at /test/11.2_dbg/sql/sql_base.cc:5917
      #12 0x000055c415915f1c in open_and_lock_tables (thd=thd@entry=0x14def4000d58, options=<optimized out>, tables=<optimized out>, tables@entry=0x14def4013d50, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x14df50567880)at /test/11.2_dbg/sql/sql_base.cc:5649
      #13 0x000055c415988e10 in open_and_lock_tables (flags=0, derived=true, tables=0x14def4013d50, thd=0x14def4000d58)at /test/11.2_dbg/sql/sql_base.h:531
      #14 execute_sqlcom_select (thd=thd@entry=0x14def4000d58, all_tables=0x14def4013d50) at /test/11.2_dbg/sql/sql_parse.cc:6089
      #15 0x000055c41599507a in mysql_execute_command (thd=thd@entry=0x14def4000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/11.2_dbg/sql/sql_parse.cc:3984
      #16 0x000055c41599bd26 in mysql_parse (thd=thd@entry=0x14def4000d58, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x14df505682a0)at /test/11.2_dbg/sql/sql_parse.cc:7929
      #17 0x000055c41599e1bd in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14def4000d58, packet=packet@entry=0x14def401f8f9 "SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2", packet_length=packet_length@entry=60, blocking=blocking@entry=true) at /test/11.2_dbg/sql/sql_class.h:248
      #18 0x000055c4159a03e3 in do_command (thd=0x14def4000d58, blocking=blocking@entry=true) at /test/11.2_dbg/sql/sql_parse.cc:1407
      #19 0x000055c415b0c35c in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55c418e37ce8, put_in_cache=put_in_cache@entry=true)at /test/11.2_dbg/sql/sql_connect.cc:1439
      #20 0x000055c415b0c65c in handle_one_connection (arg=arg@entry=0x55c418e37ce8)at /test/11.2_dbg/sql/sql_connect.cc:1341
      #21 0x000055c415f5362c in pfs_spawn_thread (arg=0x55c418e70f68)at /test/11.2_dbg/storage/perfschema/pfs.cc:2201
      #22 0x000014df52c9ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
      #23 0x000014df52d29c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
      

      Bug confirmed present in:
      MariaDB: 10.5.27 (dbg), 10.5.27 (opt), 10.6.20 (dbg), 10.6.20 (opt), 10.11.10 (dbg), 10.11.10 (opt), 11.2.6 (dbg), 11.2.6 (opt), 11.4.4 (dbg), 11.4.4 (opt), 11.6.2 (dbg), 11.6.2 (opt), 11.7.0 (dbg), 11.7.0 (opt)

      Attachments

        Issue Links

          Activity

            First MTR testcase creation attempt

            ./mtr --repeat 3000 test
            

            --source plugin/spider/spider/include/init_spider.inc
            DROP DATABASE test;
            CREATE DATABASE test;
            USE test;
            CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock', DATABASE 'test', USER 'Spider', PASSWORD '');
            CREATE TABLE t1 (c INT PRIMARY KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "t"';
            CREATE TABLE t2 (c INT KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "tm"';
            CREATE TABLE t3 (e INT, f BLOB) ENGINE=Spider;
            SET GLOBAL default_storage_engine=Spider;
            CREATE TABLE t5 (c1 TINYINT NOT NULL);
            XA START 'xa1';
            SHOW CREATE TABLE t1;
            SET spider_semi_table_lock=1;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT AVG(c1) AS VALUE FROM t1;
            SET GLOBAL table_open_cache=10;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >=any (SELECT b FROM t2);
            --error ER_WRONG_VALUE_COUNT_ON_ROW
            INSERT INTO t2 VALUES (0,0,0,'a','b','c','d');
            UPDATE IGNORE t5 SET c1=NULL WHERE c1>1;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            INSERT INTO t3 VALUES (1,0);
            --error ER_BAD_FIELD_ERROR
            SELECT HEX(c1),HEX (c2) FROM t5;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;
            

            Leads to, after about 3-5 minutes, to a stack observed in MDEV-34849:

            SIGSEGV|spider_db_connect|spider_db_conn_queue_action|spider_db_before_query|spider_db_set_names_internal
            

            Roel Roel Van de Paar added a comment - First MTR testcase creation attempt . /mtr --repeat 3000 test --source plugin/spider/spider/include/init_spider.inc DROP DATABASE test; CREATE DATABASE test; USE test; CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock' , DATABASE 'test' , USER 'Spider' , PASSWORD '' ); CREATE TABLE t1 (c INT PRIMARY KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "t"' ; CREATE TABLE t2 (c INT KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "tm"' ; CREATE TABLE t3 (e INT , f BLOB) ENGINE=Spider; SET GLOBAL default_storage_engine=Spider; CREATE TABLE t5 (c1 TINYINT NOT NULL ); XA START 'xa1' ; SHOW CREATE TABLE t1; SET spider_semi_table_lock=1; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT AVG (c1) AS VALUE FROM t1; SET GLOBAL table_open_cache=10; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >= any ( SELECT b FROM t2); --error ER_WRONG_VALUE_COUNT_ON_ROW INSERT INTO t2 VALUES (0,0,0, 'a' , 'b' , 'c' , 'd' ); UPDATE IGNORE t5 SET c1= NULL WHERE c1>1; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE INSERT INTO t3 VALUES (1,0); --error ER_BAD_FIELD_ERROR SELECT HEX(c1),HEX (c2) FROM t5; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2; Leads to, after about 3-5 minutes, to a stack observed in MDEV-34849 : SIGSEGV|spider_db_connect|spider_db_conn_queue_action|spider_db_before_query|spider_db_set_names_internal
            Roel Roel Van de Paar added a comment - - edited

            Second attempt

            ./mtr --repeat 3000 test
            

            --source include/have_innodb.inc
            --source plugin/spider/spider/include/init_spider.inc
            #SET spider_same_server_link=on;
            eval CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (HOST "127.0.0.1", DATABASE "test", USER "root", PORT $MASTER_MYPORT);
            DROP DATABASE test;
            CREATE DATABASE test;
            USE test;
            #CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock', DATABASE 'test', USER 'Spider', PASSWORD '');
            CREATE TABLE t1 (c INT PRIMARY KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "t"';
            CREATE TABLE t2 (c INT KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "tm"';
            CREATE TABLE t3 (e INT, f BLOB) ENGINE=Spider;
            SET GLOBAL default_storage_engine=Spider;
            CREATE TABLE t5 (c1 TINYINT NOT NULL);
            XA START 'xa1';
            SHOW CREATE TABLE t1;
            SET spider_semi_table_lock=1;
            --error 12702, 12720
            SELECT AVG(c1) AS VALUE FROM t1;
            SET GLOBAL table_open_cache=10;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >=any (SELECT b FROM t2);
            --error ER_WRONG_VALUE_COUNT_ON_ROW
            INSERT INTO t2 VALUES (0,0,0,'a','b','c','d');
            UPDATE IGNORE t5 SET c1=NULL WHERE c1>1;
            --error 12702, 12720
            SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2;
            --error 12702, 12720, 1054
            INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            INSERT INTO t3 VALUES (1,0);
            --error ER_BAD_FIELD_ERROR
            SELECT HEX(c1),HEX (c2) FROM t5;
            --error 1146, ER_PARSE_ERROR, 12720
            SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;
            

            Shows a number of interesting issues:
            1. Varying spider_same_server_link active or not (remove the #) results in different errors. Somewhat to be expected, but it may highlight some of the "not reproducible in MTR" issues we've been seeing lately.
            2. Sometimes errors are sporadic and differ, for example the 1054 vs 12702 on the INSERT and 1146 vs ER_PARSE_ERROR on the final SELECT. This shows [data] inconsistency issues seems concerning.

            Roel Roel Van de Paar added a comment - - edited Second attempt . /mtr --repeat 3000 test --source include/have_innodb.inc --source plugin/spider/spider/include/init_spider.inc # SET spider_same_server_link= on ; eval CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (HOST "127.0.0.1" , DATABASE "test" , USER "root" , PORT $MASTER_MYPORT); DROP DATABASE test; CREATE DATABASE test; USE test; # CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock' , DATABASE 'test' , USER 'Spider' , PASSWORD '' ); CREATE TABLE t1 (c INT PRIMARY KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "t"' ; CREATE TABLE t2 (c INT KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "tm"' ; CREATE TABLE t3 (e INT , f BLOB) ENGINE=Spider; SET GLOBAL default_storage_engine=Spider; CREATE TABLE t5 (c1 TINYINT NOT NULL ); XA START 'xa1' ; SHOW CREATE TABLE t1; SET spider_semi_table_lock=1; --error 12702, 12720 SELECT AVG (c1) AS VALUE FROM t1; SET GLOBAL table_open_cache=10; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >= any ( SELECT b FROM t2); --error ER_WRONG_VALUE_COUNT_ON_ROW INSERT INTO t2 VALUES (0,0,0, 'a' , 'b' , 'c' , 'd' ); UPDATE IGNORE t5 SET c1= NULL WHERE c1>1; --error 12702, 12720 SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2; --error 12702, 12720, 1054 INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE INSERT INTO t3 VALUES (1,0); --error ER_BAD_FIELD_ERROR SELECT HEX(c1),HEX (c2) FROM t5; --error 1146, ER_PARSE_ERROR, 12720 SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2; Shows a number of interesting issues: 1. Varying spider_same_server_link active or not (remove the # ) results in different errors. Somewhat to be expected, but it may highlight some of the "not reproducible in MTR" issues we've been seeing lately. 2. Sometimes errors are sporadic and differ, for example the 1054 vs 12702 on the INSERT and 1146 vs ER_PARSE_ERROR on the final SELECT. This shows [data] inconsistency issues seems concerning.
            Roel Roel Van de Paar added a comment - - edited

            Third attempt, success with:

            ./mtr --repeat 3000 test
            

            --source include/have_innodb.inc
            INSTALL PLUGIN Spider SONAME 'ha_spider.so';
            CREATE USER spider@localhost IDENTIFIED BY 'pwd';
            GRANT ALL ON test.* TO spider@localhost;
            DROP DATABASE test;
            CREATE DATABASE test;
            USE test;
            --let $SOCKET= `SELECT @@global.socket`
            eval CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET "$SOCKET",DATABASE 'test',USER 'Spider',PASSWORD 'pwd');
            CREATE TABLE t1 (c INT PRIMARY KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "t"';
            CREATE TABLE t2 (c INT KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "tm"';
            CREATE TABLE t3 (e INT, f BLOB) ENGINE=Spider;
            SET GLOBAL default_storage_engine=Spider;
            CREATE TABLE t5 (c1 TINYINT NOT NULL);
            XA START 'xa1';
            SHOW CREATE TABLE t1;
            SET spider_semi_table_lock=1;
            --error 12702, 12720, ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT AVG(c1) AS VALUE FROM t1;
            SET GLOBAL table_open_cache=10;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >=any (SELECT b FROM t2);
            --error ER_WRONG_VALUE_COUNT_ON_ROW
            INSERT INTO t2 VALUES (0,0,0,'a','b','c','d');
            UPDATE IGNORE t5 SET c1=NULL WHERE c1>1;
            --error 12702, 12720, ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2;
            --error 12702, 12720, 1054, ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            INSERT INTO t3 VALUES (1,0);
            --error ER_BAD_FIELD_ERROR
            SELECT HEX(c1),HEX (c2) FROM t5;
            --error 1146, ER_PARSE_ERROR, 12720, ER_CONNECT_TO_FOREIGN_DATA_SOURCE, 1054, ER_BAD_FIELD_ERROR
            SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;
            

            1. Produced the SIGSEGV in spider_string::length bug on 10.5 e886c2ba02ac021c648f84aa8f910af4fb4fb4bb (Debug) , 11.2 e91a79945822def1452787f825e6047c6a64dbd9 (Debug) and 11.2 e91a79945822def1452787f825e6047c6a64dbd9 (Debug).
            2. Needed a 'ER_BAD_FIELD_ERROR' error addition for a random sporadic failure on the final query for 11.7 5bbda9711131845ae6b4315a268b4d1710943a85 (Optimized), and 1054 (same error) for 10.5 e886c2ba02ac021c648f84aa8f910af4fb4fb4bb (Optimized)
            3. Sporadically produced the SIGSEGV in spider_db_connect from MDEV-34849 on 10.6 c630e23a186c7ecfe0afac21163cb4fa2cdc5f7a (Debug)
            4. Did not reproduce SIGSEGV in spider_db_mbase::append_lock_tables yet (which looks to be more sporadic than the SIGSEGV in spider_string::length)
            5. General runtime before/between failures is 3-8 minutes on a fast instance.

            Roel Roel Van de Paar added a comment - - edited Third attempt, success with: . /mtr --repeat 3000 test --source include/have_innodb.inc INSTALL PLUGIN Spider SONAME 'ha_spider.so' ; CREATE USER spider@localhost IDENTIFIED BY 'pwd' ; GRANT ALL ON test.* TO spider@localhost; DROP DATABASE test; CREATE DATABASE test; USE test; --let $SOCKET= `SELECT @@global.socket` eval CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET "$SOCKET" , DATABASE 'test' , USER 'Spider' , PASSWORD 'pwd' ); CREATE TABLE t1 (c INT PRIMARY KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "t"' ; CREATE TABLE t2 (c INT KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "tm"' ; CREATE TABLE t3 (e INT , f BLOB) ENGINE=Spider; SET GLOBAL default_storage_engine=Spider; CREATE TABLE t5 (c1 TINYINT NOT NULL ); XA START 'xa1' ; SHOW CREATE TABLE t1; SET spider_semi_table_lock=1; --error 12702, 12720, ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT AVG (c1) AS VALUE FROM t1; SET GLOBAL table_open_cache=10; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >= any ( SELECT b FROM t2); --error ER_WRONG_VALUE_COUNT_ON_ROW INSERT INTO t2 VALUES (0,0,0, 'a' , 'b' , 'c' , 'd' ); UPDATE IGNORE t5 SET c1= NULL WHERE c1>1; --error 12702, 12720, ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2; --error 12702, 12720, 1054, ER_CONNECT_TO_FOREIGN_DATA_SOURCE INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE INSERT INTO t3 VALUES (1,0); --error ER_BAD_FIELD_ERROR SELECT HEX(c1),HEX (c2) FROM t5; --error 1146, ER_PARSE_ERROR, 12720, ER_CONNECT_TO_FOREIGN_DATA_SOURCE, 1054, ER_BAD_FIELD_ERROR SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2; 1. Produced the SIGSEGV in spider_string::length bug on 10.5 e886c2ba02ac021c648f84aa8f910af4fb4fb4bb (Debug) , 11.2 e91a79945822def1452787f825e6047c6a64dbd9 (Debug) and 11.2 e91a79945822def1452787f825e6047c6a64dbd9 (Debug). 2. Needed a 'ER_BAD_FIELD_ERROR' error addition for a random sporadic failure on the final query for 11.7 5bbda9711131845ae6b4315a268b4d1710943a85 (Optimized), and 1054 (same error) for 10.5 e886c2ba02ac021c648f84aa8f910af4fb4fb4bb (Optimized) 3. Sporadically produced the SIGSEGV in spider_db_connect from MDEV-34849 on 10.6 c630e23a186c7ecfe0afac21163cb4fa2cdc5f7a (Debug) 4. Did not reproduce SIGSEGV in spider_db_mbase::append_lock_tables yet (which looks to be more sporadic than the SIGSEGV in spider_string::length) 5. General runtime before/between failures is 3-8 minutes on a fast instance.

            I also saw the following stack on 11.7:

            CS 11.7.0 5bbda9711131845ae6b4315a268b4d1710943a85 (Debug)

            Core was generated by `/test/MD090924-mariadb-11.7.0-linux-x86_64-dbg/bin/mariadbd --no-defaults --bas'.
            Program terminated with signal SIGSEGV, Segmentation fault.
            #0  0x00001528c119e4ba in spider_db_mbase::append_lock_tables (this=0x1528600da4a0, str=0x1528600b3da0)at /test/server_dbg/storage/spider/spd_db_mysql.cc:3596
             
            [Current thread is 1 (LWP 2542252)]
            (gdb) bt
            #0  0x00001528c119e4ba in spider_db_mbase::append_lock_tables (this=0x1528600da4a0, str=0x1528600b3da0)at /test/server_dbg/storage/spider/spd_db_mysql.cc:3596
            #1  0x00001528c11a6244 in spider_mbase_handler::lock_tables (this=0x1528600b3d40, link_idx=0)at /test/server_dbg/storage/spider/spd_db_mysql.cc:13885
            #2  0x00001528c112843e in spider_db_lock_tables (spider=spider@entry=0x15286031d830, link_idx=link_idx@entry=0)at /test/server_dbg/storage/spider/spd_db_conn.cc:1271
            #3  0x00001528c1194bcb in ha_spider::lock_tables (this=this@entry=0x15286031d830)at /test/server_dbg/storage/spider/ha_spider.cc:11990
            #4  0x00001528c1195065 in ha_spider::external_lock (this=0x15286031d830, thd=0x152860000d58, lock_type=0)at /test/server_dbg/storage/spider/ha_spider.cc:941
            #5  0x000055aa8117cc43 in handler::ha_external_lock (this=0x15286031d830, thd=thd@entry=0x152860000d58, lock_type=lock_type@entry=0)at /test/server_dbg/sql/handler.cc:7578
            #6  0x000055aa812dc4c0 in lock_external (count=1, tables=0x15286001ca00, thd=0x152860000d58) at /test/server_dbg/sql/lock.cc:397
            #7  mysql_lock_tables (thd=thd@entry=0x152860000d58, sql_lock=sql_lock@entry=0x15286001c9d0, flags=flags@entry=0)at /test/server_dbg/sql/lock.cc:342
            #8  0x000055aa812dd2f1 in mysql_lock_tables (thd=thd@entry=0x152860000d58, tables=tables@entry=0x15286001c9c8, count=count@entry=1, flags=flags@entry=0) at /test/server_dbg/sql/lock.cc:305
            #9  0x000055aa80dc1639 in lock_tables (thd=thd@entry=0x152860000d58, tables=0x15286001b270, count=<optimized out>, flags=flags@entry=0)at /test/server_dbg/sql/sql_base.cc:5911
            #10 0x000055aa80dcc4b4 in open_and_lock_tables (thd=thd@entry=0x152860000d58, options=<optimized out>, tables=<optimized out>, tables@entry=0x15286001b270, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x1528c1767760)at /test/server_dbg/sql/sql_base.cc:5643
            #11 0x000055aa80e401ce in open_and_lock_tables (flags=0, derived=true, tables=0x15286001b270, thd=0x152860000d58)at /test/server_dbg/sql/sql_base.h:532
            #12 execute_sqlcom_select (thd=thd@entry=0x152860000d58, all_tables=0x15286001b270) at /test/server_dbg/sql/sql_parse.cc:6075
            #13 0x000055aa80e4be3a in mysql_execute_command (thd=thd@entry=0x152860000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/server_dbg/sql/sql_parse.cc:3954
            #14 0x000055aa80e5278c in mysql_parse (thd=thd@entry=0x152860000d58, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x1528c1768260)at /test/server_dbg/sql/sql_parse.cc:7876
            #15 0x000055aa80e54c23 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x152860000d58, packet=packet@entry=0x15286000b329 "SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;", packet_length=packet_length@entry=61, blocking=blocking@entry=true) at /test/server_dbg/sql/sql_class.h:256
            #16 0x000055aa80e56f71 in do_command (thd=0x152860000d58, blocking=blocking@entry=true) at /test/server_dbg/sql/sql_parse.cc:1405
            #17 0x000055aa80fcf218 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55aa840ff208, put_in_cache=put_in_cache@entry=true)at /test/server_dbg/sql/sql_connect.cc:1448
            #18 0x000055aa80fcf524 in handle_one_connection (arg=arg@entry=0x55aa840ff208)at /test/server_dbg/sql/sql_connect.cc:1350
            #19 0x000055aa81423f79 in pfs_spawn_thread (arg=0x55aa840d3878)at /test/server_dbg/storage/perfschema/pfs.cc:2198
            #20 0x00001528c589ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
            #21 0x00001528c5929c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
            

            With this testcase:

            # Options in use: --max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode= --core-file
            INSTALL PLUGIN Spider SONAME 'ha_spider.so';
            CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock', DATABASE 'test', USER 'Spider', PASSWORD '');
            CREATE TABLE t1 (c INT PRIMARY KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "t"';
            CREATE TABLE t2 (c INT KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "tm"';
            CREATE TABLE t3 (e INT, f BLOB) ENGINE=Spider;
            SET GLOBAL default_storage_engine=Spider;
            SET default_storage_engine=DEFAULT;
            CREATE TABLE t5 (c1 TINYINT NOT NULL);
            XA START 'xa1';
            SHOW CREATE TABLE t1;
            SET spider_semi_table_lock=1;
            SELECT AVG(c1) AS VALUE FROM t1;
            SET GLOBAL table_open_cache=FALSE;
            EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >=any (SELECT b FROM t2);
            INSERT INTO t2 VALUES (888,228312,37,'graDUALLy','mineral','creak','FAS');
            UPDATE IGNORE t5 SET c1=NULL WHERE c1>100;
            SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2;
            SELECT * FROM t2 WHERE i > 10 AND i <=18 ORDER BY i;
            INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C;
            CALL sp71_nu (1.00e+40);
            INSERT INTO t3 VALUES (1,0);
            SELECT HEX(c1),HEX (c2) FROM t5;
            SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;
            

            Again highly sporadic.

            Roel Roel Van de Paar added a comment - I also saw the following stack on 11.7: CS 11.7.0 5bbda9711131845ae6b4315a268b4d1710943a85 (Debug) Core was generated by `/test/MD090924-mariadb-11.7.0-linux-x86_64-dbg/bin/mariadbd --no-defaults --bas'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00001528c119e4ba in spider_db_mbase::append_lock_tables (this=0x1528600da4a0, str=0x1528600b3da0)at /test/server_dbg/storage/spider/spd_db_mysql.cc:3596   [Current thread is 1 (LWP 2542252)] (gdb) bt #0 0x00001528c119e4ba in spider_db_mbase::append_lock_tables (this=0x1528600da4a0, str=0x1528600b3da0)at /test/server_dbg/storage/spider/spd_db_mysql.cc:3596 #1 0x00001528c11a6244 in spider_mbase_handler::lock_tables (this=0x1528600b3d40, link_idx=0)at /test/server_dbg/storage/spider/spd_db_mysql.cc:13885 #2 0x00001528c112843e in spider_db_lock_tables (spider=spider@entry=0x15286031d830, link_idx=link_idx@entry=0)at /test/server_dbg/storage/spider/spd_db_conn.cc:1271 #3 0x00001528c1194bcb in ha_spider::lock_tables (this=this@entry=0x15286031d830)at /test/server_dbg/storage/spider/ha_spider.cc:11990 #4 0x00001528c1195065 in ha_spider::external_lock (this=0x15286031d830, thd=0x152860000d58, lock_type=0)at /test/server_dbg/storage/spider/ha_spider.cc:941 #5 0x000055aa8117cc43 in handler::ha_external_lock (this=0x15286031d830, thd=thd@entry=0x152860000d58, lock_type=lock_type@entry=0)at /test/server_dbg/sql/handler.cc:7578 #6 0x000055aa812dc4c0 in lock_external (count=1, tables=0x15286001ca00, thd=0x152860000d58) at /test/server_dbg/sql/lock.cc:397 #7 mysql_lock_tables (thd=thd@entry=0x152860000d58, sql_lock=sql_lock@entry=0x15286001c9d0, flags=flags@entry=0)at /test/server_dbg/sql/lock.cc:342 #8 0x000055aa812dd2f1 in mysql_lock_tables (thd=thd@entry=0x152860000d58, tables=tables@entry=0x15286001c9c8, count=count@entry=1, flags=flags@entry=0) at /test/server_dbg/sql/lock.cc:305 #9 0x000055aa80dc1639 in lock_tables (thd=thd@entry=0x152860000d58, tables=0x15286001b270, count=<optimized out>, flags=flags@entry=0)at /test/server_dbg/sql/sql_base.cc:5911 #10 0x000055aa80dcc4b4 in open_and_lock_tables (thd=thd@entry=0x152860000d58, options=<optimized out>, tables=<optimized out>, tables@entry=0x15286001b270, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x1528c1767760)at /test/server_dbg/sql/sql_base.cc:5643 #11 0x000055aa80e401ce in open_and_lock_tables (flags=0, derived=true, tables=0x15286001b270, thd=0x152860000d58)at /test/server_dbg/sql/sql_base.h:532 #12 execute_sqlcom_select (thd=thd@entry=0x152860000d58, all_tables=0x15286001b270) at /test/server_dbg/sql/sql_parse.cc:6075 #13 0x000055aa80e4be3a in mysql_execute_command (thd=thd@entry=0x152860000d58, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/server_dbg/sql/sql_parse.cc:3954 #14 0x000055aa80e5278c in mysql_parse (thd=thd@entry=0x152860000d58, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x1528c1768260)at /test/server_dbg/sql/sql_parse.cc:7876 #15 0x000055aa80e54c23 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x152860000d58, packet=packet@entry=0x15286000b329 "SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2;", packet_length=packet_length@entry=61, blocking=blocking@entry=true) at /test/server_dbg/sql/sql_class.h:256 #16 0x000055aa80e56f71 in do_command (thd=0x152860000d58, blocking=blocking@entry=true) at /test/server_dbg/sql/sql_parse.cc:1405 #17 0x000055aa80fcf218 in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55aa840ff208, put_in_cache=put_in_cache@entry=true)at /test/server_dbg/sql/sql_connect.cc:1448 #18 0x000055aa80fcf524 in handle_one_connection (arg=arg@entry=0x55aa840ff208)at /test/server_dbg/sql/sql_connect.cc:1350 #19 0x000055aa81423f79 in pfs_spawn_thread (arg=0x55aa840d3878)at /test/server_dbg/storage/perfschema/pfs.cc:2198 #20 0x00001528c589ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447 #21 0x00001528c5929c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 With this testcase: # Options in use : --max_allowed_packet=33554432 --maximum-bulk_insert_buffer_size=1M --maximum-join_buffer_size=1M --maximum-max_heap_table_size=1M --maximum-max_join_size=1M --maximum-myisam_max_sort_file_size=1M --maximum-myisam_mmap_size=1M --maximum-myisam_sort_buffer_size=1M --maximum-optimizer_trace_max_mem_size=1M --maximum-preload_buffer_size=1M --maximum-query_alloc_block_size=1M --maximum-query_prealloc_size=1M --maximum-range_alloc_block_size=1M --maximum-read_buffer_size=1M --maximum-read_rnd_buffer_size=1M --maximum-sort_buffer_size=1M --maximum-tmp_table_size=1M --maximum-transaction_alloc_block_size=1M --maximum-transaction_prealloc_size=1M --log-output=none --sql_mode= --core-file INSTALL PLUGIN Spider SONAME 'ha_spider.so' ; CREATE SERVER srv FOREIGN DATA WRAPPER mysql OPTIONS (SOCKET '../socket.sock' , DATABASE 'test' , USER 'Spider' , PASSWORD '' ); CREATE TABLE t1 (c INT PRIMARY KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "t"' ; CREATE TABLE t2 (c INT KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "tm"' ; CREATE TABLE t3 (e INT , f BLOB) ENGINE=Spider; SET GLOBAL default_storage_engine=Spider; SET default_storage_engine= DEFAULT ; CREATE TABLE t5 (c1 TINYINT NOT NULL ); XA START 'xa1' ; SHOW CREATE TABLE t1; SET spider_semi_table_lock=1; SELECT AVG (c1) AS VALUE FROM t1; SET GLOBAL table_open_cache= FALSE ; EXPLAIN EXTENDED SELECT * FROM t3 WHERE a >= any ( SELECT b FROM t2); INSERT INTO t2 VALUES (888,228312,37, 'graDUALLy' , 'mineral' , 'creak' , 'FAS' ); UPDATE IGNORE t5 SET c1= NULL WHERE c1>100; SELECT * FROM t1 WHERE c2 IS NOT NULL ORDER BY c1,c2 LIMIT 2; SELECT * FROM t2 WHERE i > 10 AND i <=18 ORDER BY i; INSERT INTO t1 SELECT A.a+10* B.a+100* C.a, A.a+10* B.a+100* C.a, 'filler' FROM t1 A, t1 B, t1 C; CALL sp71_nu (1.00e+40); INSERT INTO t3 VALUES (1,0); SELECT HEX(c1),HEX (c2) FROM t5; SELECT * FROM t2 WHERE c1 <=-255 ORDER BY c1,c6 DESC LIMIT 2; Again highly sporadic.
            Roel Roel Van de Paar added a comment - - edited

            All UniqueID's/stacks observed thus far:

            SIGSEGV|spider_db_mbase::append_lock_tables|spider_mbase_handler::lock_tables|spider_db_lock_tables|ha_spider::lock_tables
            SIGSEGV|spider_db_mbase::append_lock_tables|spider_db_mbase::append_lock_tables|spider_mbase_handler::lock_tables|ha_spider::lock_tables
            SIGSEGV|spider_mbase_handler::append_lock_tables_list|ha_spider::append_lock_tables_list|ha_spider::store_lock|ha_spider::store_lock
            SIGSEGV|spider_string::length|spider_link_get_key|my_hash_key|hashcmp
            ASAN|heap-use-after-free|storage/spider/spd_table.cc|spider_link_get_key|my_hash_key|hashcmp|my_hash_first_from_hash_value
            

            Roel Roel Van de Paar added a comment - - edited All UniqueID's/stacks observed thus far: SIGSEGV|spider_db_mbase::append_lock_tables|spider_mbase_handler::lock_tables|spider_db_lock_tables|ha_spider::lock_tables SIGSEGV|spider_db_mbase::append_lock_tables|spider_db_mbase::append_lock_tables|spider_mbase_handler::lock_tables|ha_spider::lock_tables SIGSEGV|spider_mbase_handler::append_lock_tables_list|ha_spider::append_lock_tables_list|ha_spider::store_lock|ha_spider::store_lock SIGSEGV|spider_string::length|spider_link_get_key|my_hash_key|hashcmp ASAN|heap-use-after-free|storage/spider/spd_table.cc|spider_link_get_key|my_hash_key|hashcmp|my_hash_first_from_hash_value
            Roel Roel Van de Paar added a comment - - edited

            A deterministic testcase:

            --source plugin/spider/spider/include/init_spider.inc
            SET spider_same_server_link=on;
            eval CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (HOST "127.0.0.1", DATABASE "test", USER "root", PORT $MASTER_MYPORT);
            CREATE TABLE t2 (c INT KEY,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql", SRV "srv", TABLE "tm"';
            CREATE TABLE t1 (c1 INT, c2 GEOMETRY NOT NULL, SPATIAL INDEX (c2)) ENGINE=Spider;
            XA START '1';
            SET spider_semi_table_lock=1;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT * FROM t1 LIMIT 1;
            --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE
            SELECT a FROM t1 WHERE a > ALL (SELECT * FROM t2);
            --error ER_PARSE_ERROR
            INSERT INTO t2 SELECT * FROM t2;
            HANDLER t2 OPEN;
            --error ER_BAD_FIELD_ERROR
            SELECT f1, f2 FROM t2 FOR UPDATE;
            SET GLOBAL table_open_cache=256;
            --error 0,ER_BAD_FIELD_ERROR
            SELECT HEX(ind),HEX (string1) FROM t2 ORDER BY string1;
            

            Leads to this additional but similar stack on 11.7 opt:

            CS 11.7.0 4016c905cbabea7f29ed282dc2125254c7c0d419 (Optimized)

            Core was generated by `/test/MD141024-mariadb-11.7.0-linux-x86_64-opt/bin/mariadbd --no-defaults --max'.
            Program terminated with signal SIGSEGV, Segmentation fault.
            #0  0x000014c2f93a7f5d in spider_mbase_handler::append_lock_tables_list (this=0x14c2a40cd730, conn=0x14c2a40a29f8, link_idx=0, appended=0x14c2f96e05bc)at /test/11.7_opt/storage/spider/spd_db_mysql.cc:12092
             
            [Current thread is 1 (LWP 1494786)]
            (gdb) bt
            #0  0x000014c2f93a7f5d in spider_mbase_handler::append_lock_tables_list (this=0x14c2a40cd730, conn=0x14c2a40a29f8, link_idx=0, appended=0x14c2f96e05bc)at /test/11.7_opt/storage/spider/spd_db_mysql.cc:12092
            #1  0x000014c2f93a0784 in ha_spider::append_lock_tables_list (this=this@entry=0x14c2a40ca6b0)at /test/11.7_opt/storage/spider/ha_spider.cc:10235
            #2  0x000014c2f93a0d58 in ha_spider::store_lock (lock_type=TL_READ, to=0x14c2a4019d60, thd=0x14c2a4000c68, this=0x14c2a40ca6b0)at /test/11.7_opt/storage/spider/ha_spider.cc:819
            #3  ha_spider::store_lock (this=0x14c2a40ca6b0, thd=0x14c2a4000c68, to=0x14c2a4019d60, lock_type=TL_READ)at /test/11.7_opt/storage/spider/ha_spider.cc:658
            #4  0x000055c8170bbdd4 in get_lock_data (thd=thd@entry=0x14c2a4000c68, table_ptr=table_ptr@entry=0x14c2a4019d38, count=count@entry=1, flags=<optimized out>) at /test/11.7_opt/sql/lock.cc:826
            #5  0x000055c8170bc0c4 in mysql_lock_tables (thd=thd@entry=0x14c2a4000c68, tables=0x14c2a4019d38, count=1, flags=flags@entry=0)at /test/11.7_opt/sql/lock.cc:302
            #6  0x000055c816c682b5 in lock_tables (thd=thd@entry=0x14c2a4000c68, tables=0x14c2a4018b78, count=<optimized out>, flags=flags@entry=0)at /test/11.7_opt/sql/sql_base.cc:5911
            #7  0x000055c816c69f82 in open_and_lock_tables (thd=thd@entry=0x14c2a4000c68, options=<optimized out>, tables=<optimized out>, tables@entry=0x14c2a4018b78, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x14c2f96e07d0)at /test/11.7_opt/sql/sql_base.cc:5643
            #8  0x000055c816cd3885 in open_and_lock_tables (flags=0, derived=true, tables=0x14c2a4018b78, thd=0x14c2a4000c68)at /test/11.7_opt/sql/sql_base.h:532
            #9  execute_sqlcom_select (thd=thd@entry=0x14c2a4000c68, all_tables=0x14c2a4018b78) at /test/11.7_opt/sql/sql_parse.cc:6084
            #10 0x000055c816ce35c9 in mysql_execute_command (thd=thd@entry=0x14c2a4000c68, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/11.7_opt/sql/sql_parse.cc:3954
            #11 0x000055c816ce4a6e in mysql_parse (thd=0x14c2a4000c68, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/11.7_opt/sql/sql_parse.cc:7885
            #12 0x000055c816ce75c5 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14c2a4000c68, packet=packet@entry=0x14c2a4008869 "SELECT HEX(ind),HEX (string1) FROM t2 ORDER BY string1", packet_length=packet_length@entry=54, blocking=blocking@entry=true) at /test/11.7_opt/sql/sql_parse.cc:1991
            #13 0x000055c816ce9218 in do_command (thd=0x14c2a4000c68, blocking=blocking@entry=true) at /test/11.7_opt/sql/sql_parse.cc:1405
            #14 0x000055c816e2286f in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55c81940f538, put_in_cache=put_in_cache@entry=true)at /test/11.7_opt/sql/sql_connect.cc:1448
            #15 0x000055c816e22bf5 in handle_one_connection (arg=arg@entry=0x55c81940f538)at /test/11.7_opt/sql/sql_connect.cc:1350
            #16 0x000055c8171eae3f in pfs_spawn_thread (arg=0x55c819435c58)at /test/11.7_opt/storage/perfschema/pfs.cc:2198
            #17 0x000014c2fc49ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447
            #18 0x000014c2fc529c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
            

            And to the previously seen SIGSEGV|spider_string::length|spider_link_get_key|my_hash_key|hashcmp in 11.7 debug.

            Additionally, with this testcase the issue reproduces in 10.5.27 (dbg), 10.6.20 (dbg), 10.11.10 (dbg), 10.11.10 (opt), 11.2.6 (dbg), 11.4.4 (dbg), 11.4.4 (opt), 11.6.2 (dbg), 11.6.2 (opt), 11.7.0 (dbg), and 11.7.0 (opt), but not in 10.5.27 (opt), 10.6.20 (opt), and 11.2.6 (opt) - an occurence which looks somewhat similar to the version differences seen in (the likely unrelated) MDEV-35375. May be interesting from a merge/previous patch POV?

            Roel Roel Van de Paar added a comment - - edited A deterministic testcase: --source plugin/spider/spider/include/init_spider.inc SET spider_same_server_link= on ; eval CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (HOST "127.0.0.1" , DATABASE "test" , USER "root" , PORT $MASTER_MYPORT); CREATE TABLE t2 (c INT KEY ,c1 BLOB, c2 TEXT) ENGINE=Spider COMMENT= 'WRAPPER "mysql", SRV "srv", TABLE "tm"' ; CREATE TABLE t1 (c1 INT , c2 GEOMETRY NOT NULL , SPATIAL INDEX (c2)) ENGINE=Spider; XA START '1' ; SET spider_semi_table_lock=1; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT * FROM t1 LIMIT 1; --error ER_CONNECT_TO_FOREIGN_DATA_SOURCE SELECT a FROM t1 WHERE a > ALL ( SELECT * FROM t2); --error ER_PARSE_ERROR INSERT INTO t2 SELECT * FROM t2; HANDLER t2 OPEN ; --error ER_BAD_FIELD_ERROR SELECT f1, f2 FROM t2 FOR UPDATE ; SET GLOBAL table_open_cache=256; --error 0,ER_BAD_FIELD_ERROR SELECT HEX(ind),HEX (string1) FROM t2 ORDER BY string1; Leads to this additional but similar stack on 11.7 opt: CS 11.7.0 4016c905cbabea7f29ed282dc2125254c7c0d419 (Optimized) Core was generated by `/test/MD141024-mariadb-11.7.0-linux-x86_64-opt/bin/mariadbd --no-defaults --max'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000014c2f93a7f5d in spider_mbase_handler::append_lock_tables_list (this=0x14c2a40cd730, conn=0x14c2a40a29f8, link_idx=0, appended=0x14c2f96e05bc)at /test/11.7_opt/storage/spider/spd_db_mysql.cc:12092   [Current thread is 1 (LWP 1494786)] (gdb) bt #0 0x000014c2f93a7f5d in spider_mbase_handler::append_lock_tables_list (this=0x14c2a40cd730, conn=0x14c2a40a29f8, link_idx=0, appended=0x14c2f96e05bc)at /test/11.7_opt/storage/spider/spd_db_mysql.cc:12092 #1 0x000014c2f93a0784 in ha_spider::append_lock_tables_list (this=this@entry=0x14c2a40ca6b0)at /test/11.7_opt/storage/spider/ha_spider.cc:10235 #2 0x000014c2f93a0d58 in ha_spider::store_lock (lock_type=TL_READ, to=0x14c2a4019d60, thd=0x14c2a4000c68, this=0x14c2a40ca6b0)at /test/11.7_opt/storage/spider/ha_spider.cc:819 #3 ha_spider::store_lock (this=0x14c2a40ca6b0, thd=0x14c2a4000c68, to=0x14c2a4019d60, lock_type=TL_READ)at /test/11.7_opt/storage/spider/ha_spider.cc:658 #4 0x000055c8170bbdd4 in get_lock_data (thd=thd@entry=0x14c2a4000c68, table_ptr=table_ptr@entry=0x14c2a4019d38, count=count@entry=1, flags=<optimized out>) at /test/11.7_opt/sql/lock.cc:826 #5 0x000055c8170bc0c4 in mysql_lock_tables (thd=thd@entry=0x14c2a4000c68, tables=0x14c2a4019d38, count=1, flags=flags@entry=0)at /test/11.7_opt/sql/lock.cc:302 #6 0x000055c816c682b5 in lock_tables (thd=thd@entry=0x14c2a4000c68, tables=0x14c2a4018b78, count=<optimized out>, flags=flags@entry=0)at /test/11.7_opt/sql/sql_base.cc:5911 #7 0x000055c816c69f82 in open_and_lock_tables (thd=thd@entry=0x14c2a4000c68, options=<optimized out>, tables=<optimized out>, tables@entry=0x14c2a4018b78, derived=derived@entry=true, flags=flags@entry=0, prelocking_strategy=prelocking_strategy@entry=0x14c2f96e07d0)at /test/11.7_opt/sql/sql_base.cc:5643 #8 0x000055c816cd3885 in open_and_lock_tables (flags=0, derived=true, tables=0x14c2a4018b78, thd=0x14c2a4000c68)at /test/11.7_opt/sql/sql_base.h:532 #9 execute_sqlcom_select (thd=thd@entry=0x14c2a4000c68, all_tables=0x14c2a4018b78) at /test/11.7_opt/sql/sql_parse.cc:6084 #10 0x000055c816ce35c9 in mysql_execute_command (thd=thd@entry=0x14c2a4000c68, is_called_from_prepared_stmt=is_called_from_prepared_stmt@entry=false)at /test/11.7_opt/sql/sql_parse.cc:3954 #11 0x000055c816ce4a6e in mysql_parse (thd=0x14c2a4000c68, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at /test/11.7_opt/sql/sql_parse.cc:7885 #12 0x000055c816ce75c5 in dispatch_command (command=command@entry=COM_QUERY, thd=thd@entry=0x14c2a4000c68, packet=packet@entry=0x14c2a4008869 "SELECT HEX(ind),HEX (string1) FROM t2 ORDER BY string1", packet_length=packet_length@entry=54, blocking=blocking@entry=true) at /test/11.7_opt/sql/sql_parse.cc:1991 #13 0x000055c816ce9218 in do_command (thd=0x14c2a4000c68, blocking=blocking@entry=true) at /test/11.7_opt/sql/sql_parse.cc:1405 #14 0x000055c816e2286f in do_handle_one_connection (connect=<optimized out>, connect@entry=0x55c81940f538, put_in_cache=put_in_cache@entry=true)at /test/11.7_opt/sql/sql_connect.cc:1448 #15 0x000055c816e22bf5 in handle_one_connection (arg=arg@entry=0x55c81940f538)at /test/11.7_opt/sql/sql_connect.cc:1350 #16 0x000055c8171eae3f in pfs_spawn_thread (arg=0x55c819435c58)at /test/11.7_opt/storage/perfschema/pfs.cc:2198 #17 0x000014c2fc49ca94 in start_thread (arg=<optimized out>)at ./nptl/pthread_create.c:447 #18 0x000014c2fc529c3c in clone3 ()at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 And to the previously seen SIGSEGV|spider_string::length|spider_link_get_key|my_hash_key|hashcmp in 11.7 debug. Additionally, with this testcase the issue reproduces in 10.5.27 (dbg), 10.6.20 (dbg), 10.11.10 (dbg), 10.11.10 (opt), 11.2.6 (dbg), 11.4.4 (dbg), 11.4.4 (opt), 11.6.2 (dbg), 11.6.2 (opt), 11.7.0 (dbg), and 11.7.0 (opt), but not in 10.5.27 (opt), 10.6.20 (opt), and 11.2.6 (opt) - an occurence which looks somewhat similar to the version differences seen in (the likely unrelated) MDEV-35375 . May be interesting from a merge/previous patch POV?

            The testcase in the last comment also produces an ASAN heap-use-after-free.

            CS 10.5.28 7afee25b08bf801a97ce3246bb604d388572eace (Debug, UBASAN)

            ==3034654==ERROR: AddressSanitizer: heap-use-after-free on address 0x5070000563c8 at pc 0x151b1a2c3dee bp 0x151b1b6fde00 sp 0x151b1b6fddf0
            READ of size 8 at 0x5070000563c8 thread T28
                #0 0x151b1a2c3ded in spider_link_get_key(st_spider_link_for_hash*, unsigned long*, char) /test/10.5_dbg_san/storage/spider/spd_table.cc:415
                #1 0x55ce4e7957f1 in my_hash_key /test/10.5_dbg_san/mysys/hash.c:197
                #2 0x55ce4e7957f1 in hashcmp /test/10.5_dbg_san/mysys/hash.c:372
                #3 0x55ce4e79640c in my_hash_first_from_hash_value /test/10.5_dbg_san/mysys/hash.c:289
                #4 0x55ce4e7965c0 in my_hash_search_using_hash_value /test/10.5_dbg_san/mysys/hash.c:245
                #5 0x151b1a489255 in spider_mbase_handler::append_lock_tables_list(st_spider_conn*, int, int*) /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:12379
                #6 0x151b1a42ea3d in ha_spider::append_lock_tables_list() /test/10.5_dbg_san/storage/spider/ha_spider.cc:11620
                #7 0x151b1a431a67 in ha_spider::store_lock(THD*, st_thr_lock_data**, thr_lock_type) /test/10.5_dbg_san/storage/spider/ha_spider.cc:864
                #8 0x55ce4c7a8348 in get_lock_data(THD*, TABLE**, unsigned int, unsigned int) /test/10.5_dbg_san/sql/lock.cc:812
                #9 0x55ce4c7a99a1 in mysql_lock_tables(THD*, TABLE**, unsigned int, unsigned int) /test/10.5_dbg_san/sql/lock.cc:301
                #10 0x55ce4a15be5e in lock_tables(THD*, TABLE_LIST*, unsigned int, unsigned int) /test/10.5_dbg_san/sql/sql_base.cc:5504
                #11 0x55ce4a168a94 in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/10.5_dbg_san/sql/sql_base.cc:5259
                #12 0x55ce4a4f2e03 in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/10.5_dbg_san/sql/sql_base.h:509
                #13 0x55ce4a4f2e03 in execute_sqlcom_select /test/10.5_dbg_san/sql/sql_parse.cc:6358
                #14 0x55ce4a55a447 in mysql_execute_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:4029
                #15 0x55ce4a588b4d in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:8237
                #16 0x55ce4a5982da in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:1891
                #17 0x55ce4a5a6b32 in do_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:1375
                #18 0x55ce4ae9e470 in do_handle_one_connection(CONNECT*, bool) /test/10.5_dbg_san/sql/sql_connect.cc:1407
                #19 0x55ce4ae9f62f in handle_one_connection /test/10.5_dbg_san/sql/sql_connect.cc:1319
                #20 0x151b44a9ca93 in start_thread nptl/pthread_create.c:447
                #21 0x151b44b29c3b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
             
            0x5070000563c8 is located 56 bytes inside of 80-byte region [0x507000056390,0x5070000563e0)
            freed by thread T28 here:
                #0 0x55ce49d83617 in __interceptor_free (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x8564617)
                #1 0x55ce4e7f2a72 in my_free /test/10.5_dbg_san/mysys/my_malloc.c:213
                #2 0x151b1a371ad4 in spider_free_mem(st_spider_transaction*, void*, unsigned long) /test/10.5_dbg_san/storage/spider/spd_malloc.cc:182
                #3 0x151b1a497667 in spider_mbase_handler::~spider_mbase_handler() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7596
                #4 0x151b1a498136 in spider_mysql_handler::~spider_mysql_handler() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7607
                #5 0x151b1a4981ae in spider_mysql_handler::~spider_mysql_handler() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7607
                #6 0x151b1a3bcd48 in ha_spider::close() /test/10.5_dbg_san/storage/spider/ha_spider.cc:563
                #7 0x55ce4bafcacf in handler::ha_close() /test/10.5_dbg_san/sql/handler.cc:3173
                #8 0x55ce4acb2674 in closefrm(TABLE*) /test/10.5_dbg_san/sql/table.cc:4424
                #9 0x55ce4b53358d in intern_close_table /test/10.5_dbg_san/sql/table_cache.cc:220
                #10 0x55ce4b537deb in tc_purge() /test/10.5_dbg_san/sql/table_cache.cc:314
                #11 0x55ce4af32e4b in fix_table_open_cache /test/10.5_dbg_san/sql/sys_vars.cc:3879
                #12 0x55ce49eaaf3a in sys_var::update(THD*, set_var*) /test/10.5_dbg_san/sql/set_var.cc:208
                #13 0x55ce49eadadf in set_var::update(THD*) /test/10.5_dbg_san/sql/set_var.cc:859
                #14 0x55ce49eb5a87 in sql_set_variables(THD*, List<set_var_base>*, bool) /test/10.5_dbg_san/sql/set_var.cc:746
                #15 0x55ce4a5704f6 in mysql_execute_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:5172
                #16 0x55ce4a588b4d in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:8237
                #17 0x55ce4a5982da in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:1891
                #18 0x55ce4a5a6b32 in do_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:1375
                #19 0x55ce4ae9e470 in do_handle_one_connection(CONNECT*, bool) /test/10.5_dbg_san/sql/sql_connect.cc:1407
                #20 0x55ce4ae9f62f in handle_one_connection /test/10.5_dbg_san/sql/sql_connect.cc:1319
                #21 0x151b44a9ca93 in start_thread nptl/pthread_create.c:447
             
            previously allocated by thread T28 here:
                #0 0x55ce49d83967 in __interceptor_malloc (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x8564967)
                #1 0x55ce4e7f2714 in my_malloc /test/10.5_dbg_san/mysys/my_malloc.c:91
                #2 0x151b1a371f07 in spider_bulk_alloc_mem(st_spider_transaction*, unsigned int, char const*, char const*, unsigned long, unsigned long, ...) /test/10.5_dbg_san/storage/spider/spd_malloc.cc:230
                #3 0x151b1a475dee in spider_mbase_handler::init() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7653
                #4 0x151b1a3295d7 in spider_get_share(char const*, TABLE*, THD*, ha_spider*, int*) /test/10.5_dbg_san/storage/spider/spd_table.cc:5052
                #5 0x151b1a40def8 in ha_spider::open(char const*, int, unsigned int) /test/10.5_dbg_san/storage/spider/ha_spider.cc:310
                #6 0x55ce4bafd4f9 in handler::ha_open(TABLE*, char const*, int, unsigned int, st_mem_root*, List<String>*) /test/10.5_dbg_san/sql/handler.cc:3105
                #7 0x55ce4acf233f in open_table_from_share(THD*, TABLE_SHARE*, st_mysql_const_lex_string const*, unsigned int, unsigned int, unsigned int, TABLE*, bool, List<String>*) /test/10.5_dbg_san/sql/table.cc:4320
                #8 0x55ce4a14b71f in open_table(THD*, TABLE_LIST*, Open_table_context*) /test/10.5_dbg_san/sql/sql_base.cc:2024
                #9 0x55ce4a15fd53 in open_and_process_table /test/10.5_dbg_san/sql/sql_base.cc:3819
                #10 0x55ce4a15fd53 in open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) /test/10.5_dbg_san/sql/sql_base.cc:4303
                #11 0x55ce4a1686b5 in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/10.5_dbg_san/sql/sql_base.cc:5250
                #12 0x55ce4a5660ae in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/10.5_dbg_san/sql/sql_base.h:509
                #13 0x55ce4a5660ae in mysql_execute_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:4742
                #14 0x55ce4a588b4d in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:8237
                #15 0x55ce4a5982da in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:1891
                #16 0x55ce4a5a6b32 in do_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:1375
                #17 0x55ce4ae9e470 in do_handle_one_connection(CONNECT*, bool) /test/10.5_dbg_san/sql/sql_connect.cc:1407
                #18 0x55ce4ae9f62f in handle_one_connection /test/10.5_dbg_san/sql/sql_connect.cc:1319
                #19 0x151b44a9ca93 in start_thread nptl/pthread_create.c:447
             
            Thread T28 created by T0 here:
                #0 0x55ce49d27815 in pthread_create (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x8508815)
                #1 0x55ce49dde350 in create_thread_to_handle_connection(CONNECT*) /test/10.5_dbg_san/sql/mysqld.cc:6116
                #2 0x55ce49de9c8e in create_new_thread(CONNECT*) /test/10.5_dbg_san/sql/mysqld.cc:6175
                #3 0x55ce49dea3b6 in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /test/10.5_dbg_san/sql/mysqld.cc:6240
                #4 0x55ce49deb238 in handle_connections_sockets() /test/10.5_dbg_san/sql/mysqld.cc:6367
                #5 0x55ce49df1f60 in run_main_loop /test/10.5_dbg_san/sql/mysqld.cc:5357
                #6 0x55ce49df1f60 in mysqld_main(int, char**) /test/10.5_dbg_san/sql/mysqld.cc:5768
                #7 0x55ce49dc950a in main /test/10.5_dbg_san/sql/main.cc:25
                #8 0x151b44a2a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
                #9 0x151b44a2a28a in __libc_start_main_impl ../csu/libc-start.c:360
                #10 0x55ce49cf45d4 in _start (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x84d55d4)
             
            SUMMARY: AddressSanitizer: heap-use-after-free /test/10.5_dbg_san/storage/spider/spd_table.cc:415 in spider_link_get_key(st_spider_link_for_hash*, unsigned long*, char)
            Shadow bytes around the buggy address:
              0x0a0e80002c20: 00 00 00 00 00 00 00 00 fa fa fa fa 00 00 00 00
              0x0a0e80002c30: 00 00 00 00 00 fa fa fa fa fa 00 00 00 00 00 00
              0x0a0e80002c40: 00 00 00 fa fa fa fa fa fd fd fd fd fd fd fd fd
              0x0a0e80002c50: fd fa fa fa fa fa fd fd fd fd fd fd fd fd fd fa
              0x0a0e80002c60: fa fa fa fa fd fd fd fd fd fd fd fd fd fa fa fa
            =>0x0a0e80002c70: fa fa fd fd fd fd fd fd fd[fd]fd fd fa fa fa fa
              0x0a0e80002c80: 00 00 00 00 00 00 00 00 00 fa fa fa fa fa 00 00
              0x0a0e80002c90: 00 00 f7 00 f7 00 f7 fa fa fa fa fa 00 00 00 00
              0x0a0e80002ca0: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00
              0x0a0e80002cb0: 00 00 00 fa fa fa fa fa fd fd fd fd fd fd fd fd
              0x0a0e80002cc0: fd fa fa fa fa fa fd fd fd fd fd fd fd fd fd fa
            Shadow byte legend (one shadow byte represents 8 application bytes):
              Addressable:           00
              Partially addressable: 01 02 03 04 05 06 07 
              Heap left redzone:       fa
              Freed heap region:       fd
              Stack left redzone:      f1
              Stack mid redzone:       f2
              Stack right redzone:     f3
              Stack after return:      f5
              Stack use after scope:   f8
              Global redzone:          f9
              Global init order:       f6
              Poisoned by user:        f7
              Container overflow:      fc
              Array cookie:            ac
              Intra object redzone:    bb
              ASan internal:           fe
              Left alloca redzone:     ca
              Right alloca redzone:    cb
              Shadow gap:              cc
            ==3034654==ABORTING
            241109 13:08:14 [ERROR] mysqld got signal 6 ;
            

            FWIW, this stack was previously also seen in MDEV-31357.

            Roel Roel Van de Paar added a comment - The testcase in the last comment also produces an ASAN heap-use-after-free. CS 10.5.28 7afee25b08bf801a97ce3246bb604d388572eace (Debug, UBASAN) ==3034654==ERROR: AddressSanitizer: heap-use-after-free on address 0x5070000563c8 at pc 0x151b1a2c3dee bp 0x151b1b6fde00 sp 0x151b1b6fddf0 READ of size 8 at 0x5070000563c8 thread T28 #0 0x151b1a2c3ded in spider_link_get_key(st_spider_link_for_hash*, unsigned long*, char) /test/10.5_dbg_san/storage/spider/spd_table.cc:415 #1 0x55ce4e7957f1 in my_hash_key /test/10.5_dbg_san/mysys/hash.c:197 #2 0x55ce4e7957f1 in hashcmp /test/10.5_dbg_san/mysys/hash.c:372 #3 0x55ce4e79640c in my_hash_first_from_hash_value /test/10.5_dbg_san/mysys/hash.c:289 #4 0x55ce4e7965c0 in my_hash_search_using_hash_value /test/10.5_dbg_san/mysys/hash.c:245 #5 0x151b1a489255 in spider_mbase_handler::append_lock_tables_list(st_spider_conn*, int, int*) /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:12379 #6 0x151b1a42ea3d in ha_spider::append_lock_tables_list() /test/10.5_dbg_san/storage/spider/ha_spider.cc:11620 #7 0x151b1a431a67 in ha_spider::store_lock(THD*, st_thr_lock_data**, thr_lock_type) /test/10.5_dbg_san/storage/spider/ha_spider.cc:864 #8 0x55ce4c7a8348 in get_lock_data(THD*, TABLE**, unsigned int, unsigned int) /test/10.5_dbg_san/sql/lock.cc:812 #9 0x55ce4c7a99a1 in mysql_lock_tables(THD*, TABLE**, unsigned int, unsigned int) /test/10.5_dbg_san/sql/lock.cc:301 #10 0x55ce4a15be5e in lock_tables(THD*, TABLE_LIST*, unsigned int, unsigned int) /test/10.5_dbg_san/sql/sql_base.cc:5504 #11 0x55ce4a168a94 in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/10.5_dbg_san/sql/sql_base.cc:5259 #12 0x55ce4a4f2e03 in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/10.5_dbg_san/sql/sql_base.h:509 #13 0x55ce4a4f2e03 in execute_sqlcom_select /test/10.5_dbg_san/sql/sql_parse.cc:6358 #14 0x55ce4a55a447 in mysql_execute_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:4029 #15 0x55ce4a588b4d in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:8237 #16 0x55ce4a5982da in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:1891 #17 0x55ce4a5a6b32 in do_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:1375 #18 0x55ce4ae9e470 in do_handle_one_connection(CONNECT*, bool) /test/10.5_dbg_san/sql/sql_connect.cc:1407 #19 0x55ce4ae9f62f in handle_one_connection /test/10.5_dbg_san/sql/sql_connect.cc:1319 #20 0x151b44a9ca93 in start_thread nptl/pthread_create.c:447 #21 0x151b44b29c3b in clone3 ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78   0x5070000563c8 is located 56 bytes inside of 80-byte region [0x507000056390,0x5070000563e0) freed by thread T28 here: #0 0x55ce49d83617 in __interceptor_free (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x8564617) #1 0x55ce4e7f2a72 in my_free /test/10.5_dbg_san/mysys/my_malloc.c:213 #2 0x151b1a371ad4 in spider_free_mem(st_spider_transaction*, void*, unsigned long) /test/10.5_dbg_san/storage/spider/spd_malloc.cc:182 #3 0x151b1a497667 in spider_mbase_handler::~spider_mbase_handler() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7596 #4 0x151b1a498136 in spider_mysql_handler::~spider_mysql_handler() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7607 #5 0x151b1a4981ae in spider_mysql_handler::~spider_mysql_handler() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7607 #6 0x151b1a3bcd48 in ha_spider::close() /test/10.5_dbg_san/storage/spider/ha_spider.cc:563 #7 0x55ce4bafcacf in handler::ha_close() /test/10.5_dbg_san/sql/handler.cc:3173 #8 0x55ce4acb2674 in closefrm(TABLE*) /test/10.5_dbg_san/sql/table.cc:4424 #9 0x55ce4b53358d in intern_close_table /test/10.5_dbg_san/sql/table_cache.cc:220 #10 0x55ce4b537deb in tc_purge() /test/10.5_dbg_san/sql/table_cache.cc:314 #11 0x55ce4af32e4b in fix_table_open_cache /test/10.5_dbg_san/sql/sys_vars.cc:3879 #12 0x55ce49eaaf3a in sys_var::update(THD*, set_var*) /test/10.5_dbg_san/sql/set_var.cc:208 #13 0x55ce49eadadf in set_var::update(THD*) /test/10.5_dbg_san/sql/set_var.cc:859 #14 0x55ce49eb5a87 in sql_set_variables(THD*, List<set_var_base>*, bool) /test/10.5_dbg_san/sql/set_var.cc:746 #15 0x55ce4a5704f6 in mysql_execute_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:5172 #16 0x55ce4a588b4d in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:8237 #17 0x55ce4a5982da in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:1891 #18 0x55ce4a5a6b32 in do_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:1375 #19 0x55ce4ae9e470 in do_handle_one_connection(CONNECT*, bool) /test/10.5_dbg_san/sql/sql_connect.cc:1407 #20 0x55ce4ae9f62f in handle_one_connection /test/10.5_dbg_san/sql/sql_connect.cc:1319 #21 0x151b44a9ca93 in start_thread nptl/pthread_create.c:447   previously allocated by thread T28 here: #0 0x55ce49d83967 in __interceptor_malloc (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x8564967) #1 0x55ce4e7f2714 in my_malloc /test/10.5_dbg_san/mysys/my_malloc.c:91 #2 0x151b1a371f07 in spider_bulk_alloc_mem(st_spider_transaction*, unsigned int, char const*, char const*, unsigned long, unsigned long, ...) /test/10.5_dbg_san/storage/spider/spd_malloc.cc:230 #3 0x151b1a475dee in spider_mbase_handler::init() /test/10.5_dbg_san/storage/spider/spd_db_mysql.cc:7653 #4 0x151b1a3295d7 in spider_get_share(char const*, TABLE*, THD*, ha_spider*, int*) /test/10.5_dbg_san/storage/spider/spd_table.cc:5052 #5 0x151b1a40def8 in ha_spider::open(char const*, int, unsigned int) /test/10.5_dbg_san/storage/spider/ha_spider.cc:310 #6 0x55ce4bafd4f9 in handler::ha_open(TABLE*, char const*, int, unsigned int, st_mem_root*, List<String>*) /test/10.5_dbg_san/sql/handler.cc:3105 #7 0x55ce4acf233f in open_table_from_share(THD*, TABLE_SHARE*, st_mysql_const_lex_string const*, unsigned int, unsigned int, unsigned int, TABLE*, bool, List<String>*) /test/10.5_dbg_san/sql/table.cc:4320 #8 0x55ce4a14b71f in open_table(THD*, TABLE_LIST*, Open_table_context*) /test/10.5_dbg_san/sql/sql_base.cc:2024 #9 0x55ce4a15fd53 in open_and_process_table /test/10.5_dbg_san/sql/sql_base.cc:3819 #10 0x55ce4a15fd53 in open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) /test/10.5_dbg_san/sql/sql_base.cc:4303 #11 0x55ce4a1686b5 in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/10.5_dbg_san/sql/sql_base.cc:5250 #12 0x55ce4a5660ae in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/10.5_dbg_san/sql/sql_base.h:509 #13 0x55ce4a5660ae in mysql_execute_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:4742 #14 0x55ce4a588b4d in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:8237 #15 0x55ce4a5982da in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /test/10.5_dbg_san/sql/sql_parse.cc:1891 #16 0x55ce4a5a6b32 in do_command(THD*) /test/10.5_dbg_san/sql/sql_parse.cc:1375 #17 0x55ce4ae9e470 in do_handle_one_connection(CONNECT*, bool) /test/10.5_dbg_san/sql/sql_connect.cc:1407 #18 0x55ce4ae9f62f in handle_one_connection /test/10.5_dbg_san/sql/sql_connect.cc:1319 #19 0x151b44a9ca93 in start_thread nptl/pthread_create.c:447   Thread T28 created by T0 here: #0 0x55ce49d27815 in pthread_create (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x8508815) #1 0x55ce49dde350 in create_thread_to_handle_connection(CONNECT*) /test/10.5_dbg_san/sql/mysqld.cc:6116 #2 0x55ce49de9c8e in create_new_thread(CONNECT*) /test/10.5_dbg_san/sql/mysqld.cc:6175 #3 0x55ce49dea3b6 in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /test/10.5_dbg_san/sql/mysqld.cc:6240 #4 0x55ce49deb238 in handle_connections_sockets() /test/10.5_dbg_san/sql/mysqld.cc:6367 #5 0x55ce49df1f60 in run_main_loop /test/10.5_dbg_san/sql/mysqld.cc:5357 #6 0x55ce49df1f60 in mysqld_main(int, char**) /test/10.5_dbg_san/sql/mysqld.cc:5768 #7 0x55ce49dc950a in main /test/10.5_dbg_san/sql/main.cc:25 #8 0x151b44a2a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #9 0x151b44a2a28a in __libc_start_main_impl ../csu/libc-start.c:360 #10 0x55ce49cf45d4 in _start (/test/UBASAN_MD091124-mariadb-10.5.28-linux-x86_64-dbg/bin/mariadbd+0x84d55d4)   SUMMARY: AddressSanitizer: heap-use-after-free /test/10.5_dbg_san/storage/spider/spd_table.cc:415 in spider_link_get_key(st_spider_link_for_hash*, unsigned long*, char) Shadow bytes around the buggy address: 0x0a0e80002c20: 00 00 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 0x0a0e80002c30: 00 00 00 00 00 fa fa fa fa fa 00 00 00 00 00 00 0x0a0e80002c40: 00 00 00 fa fa fa fa fa fd fd fd fd fd fd fd fd 0x0a0e80002c50: fd fa fa fa fa fa fd fd fd fd fd fd fd fd fd fa 0x0a0e80002c60: fa fa fa fa fd fd fd fd fd fd fd fd fd fa fa fa =>0x0a0e80002c70: fa fa fd fd fd fd fd fd fd[fd]fd fd fa fa fa fa 0x0a0e80002c80: 00 00 00 00 00 00 00 00 00 fa fa fa fa fa 00 00 0x0a0e80002c90: 00 00 f7 00 f7 00 f7 fa fa fa fa fa 00 00 00 00 0x0a0e80002ca0: 00 00 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 0x0a0e80002cb0: 00 00 00 fa fa fa fa fa fd fd fd fd fd fd fd fd 0x0a0e80002cc0: fd fa fa fa fa fa fd fd fd fd fd fd fd fd fd fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==3034654==ABORTING 241109 13:08:14 [ERROR] mysqld got signal 6 ; FWIW, this stack was previously also seen in MDEV-31357 .

            People

              ycp Yuchen Pei
              Roel Roel Van de Paar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.