[MDEV-30981] Spider UBSAN: null pointer passed as argument 2, which is declared to never be null in spider_create_trx_alter_table on ALTER Created: 2023-03-31  Updated: 2023-10-26  Resolved: 2023-06-07

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - Spider
Affects Version/s: 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11, 11.0, 11.1
Fix Version/s: 10.5.22, 10.6.15, 10.9.8, 10.10.6, 10.11.5, 11.0.3, 11.1.2

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Yuchen Pei
Resolution: Fixed Votes: 0
Labels: UBSAN, affects-tests, not-10.4, regression-10.5

Issue Links:
Blocks
is blocked by MDEV-30435 Fix code duplication w.r.t. memcpy in... Closed
Relates

 Description    

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE TABLE t (c INT) ENGINE=Spider PARTITION BY LIST (c) (PARTITION p VALUES IN (1,2));
 
ALTER TABLE t ENGINE=InnoDB;

Leads to:

11.0.2 a79abb6517f2fa68b48e61aa3354a0631e3a63f7 (Debug)

 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:760:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:766:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:772:11: runtime error: null pointer passed as argument 2, which is declared to never be null

11.0.2 a79abb6517f2fa68b48e61aa3354a0631e3a63f7 (Debug)

 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:760:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
    #0 0x1535919f83f2 in spider_create_trx_alter_table(st_spider_transaction*, st_spider_share*, bool) /test/11.0_dbg_san/storage/spider/spd_trx.cc:760
 
    #1 0x153591ccf8f7 in ha_spider::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/storage/spider/ha_spider.cc:8804
 
    #2 0x557f7467b6ca in ha_partition::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/sql/ha_partition.cc:2378
 
    #3 0x557f72114579 in mysql_prepare_alter_table(THD*, TABLE*, Table_specification_st*, Alter_info*, Alter_table_ctx*) /test/11.0_dbg_san/sql/sql_table.cc:9014
 
    #4 0x557f72165f0f in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, TABLE_LIST*, Recreate_info*, Alter_info*, unsigned int, st_order*, bool, bool) /test/11.0_dbg_san/sql/sql_table.cc:10479
 
    #5 0x557f724e5595 in Sql_cmd_alter_table::execute(THD*) /test/11.0_dbg_san/sql/sql_alter.cc:558
 
    #6 0x557f71ac8cc7 in mysql_execute_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:6003
 
    #7 0x557f71ad25e6 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.0_dbg_san/sql/sql_parse.cc:7999
 
    #8 0x557f71ae237a in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1894
 
    #9 0x557f71af017f in do_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1407
 
    #10 0x557f724b4459 in do_handle_one_connection(CONNECT*, bool) /test/11.0_dbg_san/sql/sql_connect.cc:1416
 
    #11 0x557f724b5974 in handle_one_connection /test/11.0_dbg_san/sql/sql_connect.cc:1318
 
    #12 0x1535b5b42b42 in start_thread nptl/pthread_create.c:442
 
    #13 0x1535b5bd49ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff)
 
 
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:766:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
    #0 0x1535919f8685 in spider_create_trx_alter_table(st_spider_transaction*, st_spider_share*, bool) /test/11.0_dbg_san/storage/spider/spd_trx.cc:766
 
    #1 0x153591ccf8f7 in ha_spider::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/storage/spider/ha_spider.cc:8804
 
    #2 0x557f7467b6ca in ha_partition::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/sql/ha_partition.cc:2378
 
    #3 0x557f72114579 in mysql_prepare_alter_table(THD*, TABLE*, Table_specification_st*, Alter_info*, Alter_table_ctx*) /test/11.0_dbg_san/sql/sql_table.cc:9014
 
    #4 0x557f72165f0f in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, TABLE_LIST*, Recreate_info*, Alter_info*, unsigned int, st_order*, bool, bool) /test/11.0_dbg_san/sql/sql_table.cc:10479
 
    #5 0x557f724e5595 in Sql_cmd_alter_table::execute(THD*) /test/11.0_dbg_san/sql/sql_alter.cc:558
 
    #6 0x557f71ac8cc7 in mysql_execute_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:6003
 
    #7 0x557f71ad25e6 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.0_dbg_san/sql/sql_parse.cc:7999
 
    #8 0x557f71ae237a in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1894
 
    #9 0x557f71af017f in do_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1407
 
    #10 0x557f724b4459 in do_handle_one_connection(CONNECT*, bool) /test/11.0_dbg_san/sql/sql_connect.cc:1416
 
    #11 0x557f724b5974 in handle_one_connection /test/11.0_dbg_san/sql/sql_connect.cc:1318
 
    #12 0x1535b5b42b42 in start_thread nptl/pthread_create.c:442
 
    #13 0x1535b5bd49ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff)
 
 
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:772:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
    #0 0x1535919f8912 in spider_create_trx_alter_table(st_spider_transaction*, st_spider_share*, bool) /test/11.0_dbg_san/storage/spider/spd_trx.cc:772
 
    #1 0x153591ccf8f7 in ha_spider::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/storage/spider/ha_spider.cc:8804
 
    #2 0x557f7467b6ca in ha_partition::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/sql/ha_partition.cc:2378
 
    #3 0x557f72114579 in mysql_prepare_alter_table(THD*, TABLE*, Table_specification_st*, Alter_info*, Alter_table_ctx*) /test/11.0_dbg_san/sql/sql_table.cc:9014
 
    #4 0x557f72165f0f in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, TABLE_LIST*, Recreate_info*, Alter_info*, unsigned int, st_order*, bool, bool) /test/11.0_dbg_san/sql/sql_table.cc:10479
 
    #5 0x557f724e5595 in Sql_cmd_alter_table::execute(THD*) /test/11.0_dbg_san/sql/sql_alter.cc:558
 
    #6 0x557f71ac8cc7 in mysql_execute_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:6003
 
    #7 0x557f71ad25e6 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.0_dbg_san/sql/sql_parse.cc:7999
 
    #8 0x557f71ae237a in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1894
 
    #9 0x557f71af017f in do_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1407
 
    #10 0x557f724b4459 in do_handle_one_connection(CONNECT*, bool) /test/11.0_dbg_san/sql/sql_connect.cc:1416
 
    #11 0x557f724b5974 in handle_one_connection /test/11.0_dbg_san/sql/sql_connect.cc:1318
 
    #12 0x1535b5b42b42 in start_thread nptl/pthread_create.c:442
 
    #13 0x1535b5bd49ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff)

Setup:

Compiled with GCC >=7.5.0 (I use GCC 11.3.0) and:
 
    -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON -DWITH_RAPID=OFF -DWSREP_LIB_WITH_ASAN=ON
 
Set before execution:
 
    export ASAN_OPTIONS=quarantine_size_mb=512:atexit=0:detect_invalid_pointer_pairs=3:dump_instruction_bytes=1:abort_on_error=1:allocator_may_return_null=1

Bug confirmed present in:
MariaDB: 10.5.20 (dbg), 10.5.20 (opt), 10.6.13 (dbg), 10.6.13 (opt), 10.7.8 (dbg), 10.7.8 (opt), 10.8.8 (dbg), 10.8.8 (opt), 10.9.6 (dbg), 10.9.6 (opt), 10.10.4 (dbg), 10.10.4 (opt), 10.11.3 (dbg), 10.11.3 (opt), 11.0.2 (dbg), 11.0.2 (opt)

 Comments   
Comment by Roel Van de Paar [ 2023-05-19 ]

An additional simpler testcase which leads to a slightly different stack. This issue is seen many times during test runs.

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE TABLE t (c INT KEY) ENGINE=Spider;
 
ALTER TABLE t ADD d BIT FIRST;

Leads to:

11.0.2 368dd22a816f3b437bccd0b9ff28b9de9b1abf0a (Debug)

 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:760:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:766:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:772:11: runtime error: null pointer passed as argument 2, which is declared to never be null

11.0.2 368dd22a816f3b437bccd0b9ff28b9de9b1abf0a (Debug)

 
    #0 0x14fa0b67bf1a in spider_create_trx_alter_table(st_spider_transaction*, st_spider_share*, bool) /test/11.0_dbg_san/storage/spider/spd_trx.cc:760
 
    #1 0x14fa0b87f270 in ha_spider::create(char const*, TABLE*, HA_CREATE_INFO*) /test/11.0_dbg_san/storage/spider/ha_spider.cc:8700
 
    #2 0x55df6a2ee382 in handler::ha_create(char const*, TABLE*, HA_CREATE_INFO*) /test/11.0_dbg_san/sql/handler.cc:5660
 
    #3 0x55df6a2f3b84 in ha_create_table(THD*, char const*, char const*, char const*, HA_CREATE_INFO*, st_mysql_const_unsigned_lex_string*, bool) /test/11.0_dbg_san/sql/handler.cc:6129
 
    #4 0x55df6927a4a5 in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, TABLE_LIST*, Recreate_info*, Alter_info*, unsigned int, st_order*, bool, bool) /test/11.0_dbg_san/sql/sql_table.cc:10932
 
    #5 0x55df695f19df in Sql_cmd_alter_table::execute(THD*) /test/11.0_dbg_san/sql/sql_alter.cc:558
 
    #6 0x55df68bc4054 in mysql_execute_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:6015
 
    #7 0x55df68bcd973 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.0_dbg_san/sql/sql_parse.cc:8014
 
    #8 0x55df68bdd707 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1894
 
    #9 0x55df68beb542 in do_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1407
 
    #10 0x55df695c08b5 in do_handle_one_connection(CONNECT*, bool) /test/11.0_dbg_san/sql/sql_connect.cc:1416
 
    #11 0x55df695c1dd0 in handle_one_connection /test/11.0_dbg_san/sql/sql_connect.cc:1318
 
    #12 0x14fa30294b42 in start_thread nptl/pthread_create.c:442
 
    #13 0x14fa303269ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff)

Bug confirmed present in:
MariaDB: 10.5.21 (dbg), 10.5.21 (opt), 10.6.14 (dbg), 10.6.14 (opt), 10.9.7 (dbg), 10.9.7 (opt), 10.10.5 (dbg), 10.10.5 (opt), 10.11.4 (dbg), 10.11.4 (opt), 11.0.2 (dbg), 11.0.2 (opt), 11.1.0 (dbg), 11.1.0 (opt)

Comment by Roel Van de Paar [ 2023-05-19 ]

Another slightly different stack with this testcase:

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE TABLE t (c INT AUTO_INCREMENT,KEY(c)) ENGINE=Spider;
 
ALTER TABLE t ENGINE=Spider;

Leads to:

11.0.2 368dd22a816f3b437bccd0b9ff28b9de9b1abf0a (Debug)

 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:760:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:766:11: runtime error: null pointer passed as argument 2, which is declared to never be null
 
/test/11.0_dbg_san/storage/spider/spd_trx.cc:772:11: runtime error: null pointer passed as argument 2, which is declared to never be null

11.0.2 368dd22a816f3b437bccd0b9ff28b9de9b1abf0a (Debug)

 
    #0 0x148c2647bf1a in spider_create_trx_alter_table(st_spider_transaction*, st_spider_share*, bool) /test/11.0_dbg_san/storage/spider/spd_trx.cc:760
 
    #1 0x148c2674f471 in ha_spider::update_create_info(HA_CREATE_INFO*) /test/11.0_dbg_san/storage/spider/ha_spider.cc:8789
 
    #2 0x5557cf3f7bf9 in mysql_prepare_alter_table(THD*, TABLE*, Table_specification_st*, Alter_info*, Alter_table_ctx*) /test/11.0_dbg_san/sql/sql_table.cc:9014
 
    #3 0x5557cf449981 in mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, Table_specification_st*, TABLE_LIST*, Recreate_info*, Alter_info*, unsigned int, st_order*, bool, bool) /test/11.0_dbg_san/sql/sql_table.cc:10479
 
    #4 0x5557cf7cb9df in Sql_cmd_alter_table::execute(THD*) /test/11.0_dbg_san/sql/sql_alter.cc:558
 
    #5 0x5557ced9e054 in mysql_execute_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:6015
 
    #6 0x5557ceda7973 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/11.0_dbg_san/sql/sql_parse.cc:8014
 
    #7 0x5557cedb7707 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1894
 
    #8 0x5557cedc5542 in do_command(THD*, bool) /test/11.0_dbg_san/sql/sql_parse.cc:1407
 
    #9 0x5557cf79a8b5 in do_handle_one_connection(CONNECT*, bool) /test/11.0_dbg_san/sql/sql_connect.cc:1416
 
    #10 0x5557cf79bdd0 in handle_one_connection /test/11.0_dbg_san/sql/sql_connect.cc:1318
 
    #11 0x148c4c094b42 in start_thread nptl/pthread_create.c:442
 
    #12 0x148c4c1269ff  (/lib/x86_64-linux-gnu/libc.so.6+0x1269ff)

Bug confirmed present in:
MariaDB: 10.5.21 (dbg), 10.5.21 (opt), 10.6.14 (dbg), 10.6.14 (opt), 10.9.7 (dbg), 10.9.7 (opt), 10.10.5 (dbg), 10.10.5 (opt), 10.11.4 (dbg), 10.11.4 (opt), 11.0.2 (dbg), 11.0.2 (opt), 11.1.0 (dbg), 11.1.0 (opt)

Bug (or feature/syntax) confirmed not present in:
MariaDB: 10.4.30 (dbg), 10.4.30 (opt)

Comment by Roel Van de Paar [ 2023-05-20 ]

All testcases thus far involve ALTER TABLE. Yet another stack with:

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE TABLE t (c INT) PARTITION BY HASH (c) PARTITIONS 2;
 
ALTER TABLE t ENGINE=Spider;

Leads to UniqueID:

UBSAN|null pointer passed as argument 2, which is declared to never be null|storage/spider/spd_trx.cc|spider_create_trx_alter_table|ha_spider::create|handler::ha_create|ha_partition::create

Comment by Roel Van de Paar [ 2023-05-20 ]

Please also test with

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE SERVER srv FOREIGN DATA WRAPPER MYSQL OPTIONS (SOCKET '',DATABASE'',USER'',PASSWORD '');
 
CREATE TABLE t (c INT KEY,c1 BLOB,c2 TEXT) ENGINE=Spider COMMENT='WRAPPER "mysql",srv "srv",TABLE "tm"';
 
ALTER TABLE t ADD UNIQUE (c1);

Comment by Yuchen Pei [ 2023-05-23 ]

The offending part of the code is:

11.1 4e5b771e980edfdad5c5414aa62c81d409d585a4

 
    tmp_tgt_dsns[roop_count] = tmp_tgt_dsns_char;
 
    memcpy(tmp_tgt_dsns_char, share_alter->tmp_tgt_dsns[roop_count],
 
      sizeof(char) * share_alter->tmp_tgt_dsns_lengths[roop_count]);
 
    tmp_tgt_dsns_char +=
 
      share_alter->tmp_tgt_dsns_lengths[roop_count] + 1;
 
 
 
    tmp_tgt_filedsns[roop_count] = tmp_tgt_filedsns_char;
 
    memcpy(tmp_tgt_filedsns_char, share_alter->tmp_tgt_filedsns[roop_count],
 
      sizeof(char) * share_alter->tmp_tgt_filedsns_lengths[roop_count]);
 
    tmp_tgt_filedsns_char +=
 
      share_alter->tmp_tgt_filedsns_lengths[roop_count] + 1;
 
 
 
    tmp_tgt_drivers[roop_count] = tmp_tgt_drivers_char;
 
    memcpy(tmp_tgt_drivers_char, share_alter->tmp_tgt_drivers[roop_count],
 
      sizeof(char) * share_alter->tmp_tgt_drivers_lengths[roop_count]);
 
    tmp_tgt_drivers_char +=
 
      share_alter->tmp_tgt_drivers_lengths[roop_count] + 1;

This is similar to an issue pointed out by marko in MDEV-26541 at [1]. A proper fix would be MDEV-30435, so I'm marking that ticket as blocking the present one.

[1] https://jira.mariadb.org/browse/MDEV-26541?focusedCommentId=246967&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-246967

Comment by Roel Van de Paar [ 2023-06-01 ]

This issue, besides terminating / affecting quite a few tests is often seen in combination with errors like Got error 1429 when reading table and Got error 1100 when reading table. I am analyzing this further.

Note to self: 271168, 1782 1842

Comment by Roel Van de Paar [ 2023-06-02 ]

This comment shows how running ./mtr --debug ha_part (or ./mtr --debug ha) on an UBSAN/ASAN build will produce the same issue as reported in this bug.

Comment by Roel Van de Paar [ 2023-06-06 ]

Re: Got error 1100 see MDEV-31409. Seems to be unconnected ftm. Got error 1429 is confirmed unrelated.

Comment by Yuchen Pei [ 2023-06-07 ]

fixed by MDEV-30435

Comment by Roel Van de Paar [ 2023-10-26 ]

Observed in bb-11.3-mdev-28856-and-fixes:

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
DROP TABLE t1;
 
SET SESSION SPIDER_IGNORE_COMMENTS=1;
 
CREATE TABLE t1 (a INT, b VARCHAR(255), PRIMARY KEY(a)) ENGINE=Spider;
 
CREATE TABLE t1 (a INT) ENGINE=InnoDB;
 
ALTER TABLE t1 read_only=0;
 
SHUTDOWN;

Or

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
DROP TABLE t1;
 
CREATE TABLE t1 (a INT, b VARCHAR(255), PRIMARY KEY(a)) ENGINE=Spider;
 
CREATE TABLE t1 (a INT) ENGINE=InnoDB;
 
ALTER TABLE t1 COMMENT="read_only_mode '1'";
 
SHUTDOWN;

Causes:

11.3.0 cc08a83ef4225960dccb46bd68fc549160d21841 (Optimized, UBASAN)

 
UBSAN|null pointer passed as argument 2, which is declared to never be null|storage/spider/spd_trx.cc|spider_create_trx_alter_table|ha_spider::create|handler::ha_create|ha_create_table

Update: This is confirmed to be due to the missing MDEV-30981 patch.

Comment by Roel Van de Paar [ 2023-10-26 ]

Another similar issue:

INSTALL PLUGIN spider SONAME 'ha_spider.so';
 
CREATE server srv FOREIGN DATA wrapper mysql options (socket '../socket.sock',DATABASE 'test',USER 'spider',PASSWORD '');
 
CREATE TABLE t1 (a INT,b VARCHAR(1),PRIMARY KEY(a)) ENGINE=Spider PARTITION BY RANGE (a) (PARTITION p1 VALUES LESS THAN (1) REMOTE_SERVER='srv' REMOTE_TABLE='t1',PARTITION p2 VALUES LESS THAN MAXVALUE REMOTE_SERVER='srv' REMOTE_TABLE='t1');
 
ALTER TABLE t1 READ_ONLY=0;


UBSAN|null pointer passed as argument 2, which is declared to never be null|storage/spider/spd_trx.cc|spider_create_trx_alter_table|ha_spider::update_create_info|ha_partition::update_create_info|mysql_prepare_alter_table

Confirmed to be caused by this bug, not present in a patched preview-11.3-preview branch.

Generated at Thu Feb 08 10:20:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.