[MDEV-26814] UBSAN: runtime error: applying non-zero offset 18446744073709551584 to null pointer on JSON_ARRAY_INSERT, runtime error: pointer index expression with base 0x000000000001 overflowed to 0xffffffffffffffe1 Created: 2021-10-13  Updated: 2023-11-28

Status: Confirmed
Project: MariaDB Server
Component/s: JSON
Affects Version/s: 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11, 11.0, 11.1, 11.2
Fix Version/s: 10.4, 10.5, 10.6, 10.11, 11.0, 11.1, 11.2

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Rucha Deodhar
Resolution: Unresolved Votes: 0
Labels: UBSAN, affects-tests

Issue Links:
Relates
relates to MDEV-26272 The macro MASTER_INFO_VAR invokes und... Closed
relates to MDEV-25454 Make MariaDB server UBSAN safe Confirmed
relates to MDEV-28136 MariaDB ASAN Unknown Crash in Item_fu... Confirmed

 Description    

SELECT JSON_ARRAY_INSERT (0,NULL,1);

Leads to:

10.6.5 309209c51c211d0733a6c7a67fb70a523a1706f7 (Optimized)

 
/test/10.6_opt_san/sql/item_jsonfunc.cc:1815:26: runtime error: applying non-zero offset 18446744073709551584 to null pointer

10.6.5 309209c51c211d0733a6c7a67fb70a523a1706f7 (Optimized)

 
    #1 0x55ccffb5c3dd in Type_handler::Item_send_str(Item*, Protocol*, st_value*) const /test/10.6_opt_san/sql/sql_type.cc:7455
 
    #2 0x55ccfe85ef91 in Protocol::send_result_set_row(List<Item>*) /test/10.6_opt_san/sql/protocol.cc:1327
 
    #3 0x55ccfebc7119 in select_send::send_data(List<Item>&) /test/10.6_opt_san/sql/sql_class.cc:3073
 
    #4 0x55ccff2096c7 in JOIN::exec_inner() /test/10.6_opt_san/sql/sql_select.cc:4603
 
    #5 0x55ccff20d6e9 in JOIN::exec() /test/10.6_opt_san/sql/sql_select.cc:4515
 
    #6 0x55ccff1fd255 in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /test/10.6_opt_san/sql/sql_select.cc:4993
 
    #7 0x55ccff201103 in handle_select(THD*, LEX*, select_result*, unsigned long) /test/10.6_opt_san/sql/sql_select.cc:545
 
    #8 0x55ccfee9865f in execute_sqlcom_select /test/10.6_opt_san/sql/sql_parse.cc:6256
 
    #9 0x55ccfeed8043 in mysql_execute_command(THD*, bool) /test/10.6_opt_san/sql/sql_parse.cc:3946
 
    #10 0x55ccfee68428 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.6_opt_san/sql/sql_parse.cc:8030
 
    #11 0x55ccfeebdcf5 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.6_opt_san/sql/sql_parse.cc:1896
 
    #12 0x55ccfeec94f2 in do_command(THD*, bool) /test/10.6_opt_san/sql/sql_parse.cc:1404
 
    #13 0x55ccff71d7fd in do_handle_one_connection(CONNECT*, bool) /test/10.6_opt_san/sql/sql_connect.cc:1418
 
    #14 0x55ccff7202f4 in handle_one_connection /test/10.6_opt_san/sql/sql_connect.cc:1312
 
    #15 0x55cd01899ee1 in pfs_spawn_thread /test/10.6_opt_san/storage/perfschema/pfs.cc:2201
 
    #16 0x14dc15db6608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477
 
    #17 0x14dc1502c292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)

10.6.5 309209c51c211d0733a6c7a67fb70a523a1706f7 (Debug)

 
    #1 0x5587ca438c10 in Type_handler::Item_send_str(Item*, Protocol*, st_value*) const /test/10.6_dbg_san/sql/sql_type.cc:7455
 
    #2 0x5587c9e98128 in Type_handler_string_result::Item_send(Item*, Protocol*, st_value*) const /test/10.6_dbg_san/sql/sql_type.h:5440
 
    #3 0x5587c8d48e27 in Item::send(Protocol*, st_value*) /test/10.6_dbg_san/sql/item.h:1227
 
    #4 0x5587c8d2f4e7 in Protocol::send_result_set_row(List<Item>*) /test/10.6_dbg_san/sql/protocol.cc:1327
 
    #5 0x5587c917a4e1 in select_send::send_data(List<Item>&) /test/10.6_dbg_san/sql/sql_class.cc:3073
 
    #6 0x5587c98c4d92 in select_result_sink::send_data_with_check(List<Item>&, st_select_lex_unit*, unsigned long long) /test/10.6_dbg_san/sql/sql_class.h:5631
 
    #7 0x5587c98c4d92 in JOIN::exec_inner() /test/10.6_dbg_san/sql/sql_select.cc:4603
 
    #8 0x5587c98cc5a0 in JOIN::exec() /test/10.6_dbg_san/sql/sql_select.cc:4515
 
    #9 0x5587c98bcef2 in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /test/10.6_dbg_san/sql/sql_select.cc:4993
 
    #10 0x5587c98be87a in handle_select(THD*, LEX*, select_result*, unsigned long) /test/10.6_dbg_san/sql/sql_select.cc:545
 
    #11 0x5587c94c10ae in execute_sqlcom_select /test/10.6_dbg_san/sql/sql_parse.cc:6256
 
    #12 0x5587c95245d0 in mysql_execute_command(THD*, bool) /test/10.6_dbg_san/sql/sql_parse.cc:3946
 
    #13 0x5587c94894ba in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.6_dbg_san/sql/sql_parse.cc:8030
 
    #14 0x5587c94fe1c8 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.6_dbg_san/sql/sql_parse.cc:1896
 
    #15 0x5587c9514c10 in do_command(THD*, bool) /test/10.6_dbg_san/sql/sql_parse.cc:1404
 
    #16 0x5587c9f38f9e in do_handle_one_connection(CONNECT*, bool) /test/10.6_dbg_san/sql/sql_connect.cc:1418
 
    #17 0x5587c9f3be37 in handle_one_connection /test/10.6_dbg_san/sql/sql_connect.cc:1312
 
    #18 0x5587cc551344 in pfs_spawn_thread /test/10.6_dbg_san/storage/perfschema/pfs.cc:2201
 
    #19 0x14aa23ff8608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477
 
    #20 0x14aa2326e292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)

Setup:

Compiled with GCC >=7.5.0 (I use GCC 9.3.0) and:
 
    -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON -DWITH_RAPID=OFF -DWSREP_LIB_WITH_ASAN=ON
 
Set before execution:
 
    export UBSAN_OPTIONS=print_stacktrace=1

Bug confirmed present in:
MariaDB: 10.2.41 (dbg), 10.2.41 (opt), 10.3.32 (dbg), 10.3.32 (opt), 10.4.22 (dbg), 10.4.22 (opt), 10.5.13 (dbg), 10.5.13 (opt), 10.6.5 (dbg), 10.6.5 (opt), 10.7.1 (dbg), 10.7.1 (opt)

Different stacks across versions

UBSAN|applying non-zero offset X to null pointer|sql/item_jsonfunc.cc|Item_func_json_array_insert::val_str|Item::send|Protocol::send_result_set_row|select_send::send_data
 
UBSAN|applying non-zero offset X to null pointer|sql/item_jsonfunc.cc|Item_func_json_array_insert::val_str|Type_handler::Item_send_str|Protocol::send_result_set_row|select_send::send_data
 
UBSAN|applying non-zero offset X to null pointer|sql/item_jsonfunc.cc|Item_func_json_array_insert::val_str|Type_handler::Item_send_str|Type_handler_string_result::Item_send|Item::send



 Comments   
Comment by Marko Mäkelä [ 2021-10-13 ]

Perhaps more obvious diagnostic messages could be produced if Type_handler::Item_send_str() and similar functions were declared with _attribute_((nonnull)). Based on the output, the problem seems to be a null pointer, possibly buf.

Comment by Roel Van de Paar [ 2022-09-06 ]

Updated versions, additional testcase, and additional new stacks

DUMMY;
 
SELECT * FROM information_schema.table_privileges ORDER BY GRANTEE;
 
SELECT JSON_ARRAY_INSERT (0,NULL,0);

Leads to:

10.11.0 fe1f8f2c6b6f3b8e3383168225f9ae7853028947 (Debug)

 
/test/10.11_dbg_san/sql/item_jsonfunc.cc:2092:26: runtime error: pointer index expression with base 0x000000000001 overflowed to 0xffffffffffffffe1

10.11.0 fe1f8f2c6b6f3b8e3383168225f9ae7853028947 (Debug)

 
    #0 0x55f3abdf713b in Item_func_json_array_insert::val_str(String*) /test/10.11_dbg_san/sql/item_jsonfunc.cc:2092
 
    #1 0x55f3abfed09e in Type_handler::Item_send_str(Item*, Protocol*, st_value*) const /test/10.11_dbg_san/sql/sql_type.cc:7454
 
    #2 0x55f3aba0f18c in Type_handler_string_result::Item_send(Item*, Protocol*, st_value*) const /test/10.11_dbg_san/sql/sql_type.h:5446
 
    #3 0x55f3aa6b38c7 in Item::send(Protocol*, st_value*) /test/10.11_dbg_san/sql/item.h:1227
 
    #4 0x55f3aa8736ec in Protocol::send_result_set_row(List<Item>*) /test/10.11_dbg_san/sql/protocol.cc:1332
 
    #5 0x55f3aac8f618 in select_send::send_data(List<Item>&) /test/10.11_dbg_san/sql/sql_class.cc:3099
 
    #6 0x55f3ab466226 in select_result_sink::send_data_with_check(List<Item>&, st_select_lex_unit*, unsigned long long) /test/10.11_dbg_san/sql/sql_class.h:5679
 
    #7 0x55f3ab466226 in JOIN::exec_inner() /test/10.11_dbg_san/sql/sql_select.cc:4678
 
    #8 0x55f3ab46d1c6 in JOIN::exec() /test/10.11_dbg_san/sql/sql_select.cc:4590
 
    #9 0x55f3ab45abec in mysql_select(THD*, TABLE_LIST*, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /test/10.11_dbg_san/sql/sql_select.cc:5070
 
    #10 0x55f3ab45f3bd in handle_select(THD*, LEX*, select_result*, unsigned long) /test/10.11_dbg_san/sql/sql_select.cc:581
 
    #11 0x55f3aafe3b58 in execute_sqlcom_select /test/10.11_dbg_san/sql/sql_parse.cc:6261
 
    #12 0x55f3ab044352 in mysql_execute_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:3945
 
    #13 0x55f3aafb1c88 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/10.11_dbg_san/sql/sql_parse.cc:8035
 
    #14 0x55f3ab01e85f in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1894
 
    #15 0x55f3ab030a70 in do_command(THD*, bool) /test/10.11_dbg_san/sql/sql_parse.cc:1407
 
    #16 0x55f3abaa1b41 in do_handle_one_connection(CONNECT*, bool) /test/10.11_dbg_san/sql/sql_connect.cc:1418
 
    #17 0x55f3abaa432c in handle_one_connection /test/10.11_dbg_san/sql/sql_connect.cc:1312
 
    #18 0x145df1a2a608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
 
    #19 0x145df0c9f132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)

Setup:

Compiled with GCC >=7.5.0 (I use GCC 9.4.0) and:
 
    -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON -DWITH_RAPID=OFF -DWSREP_LIB_WITH_ASAN=ON
 
Set before execution:
 
    export UBSAN_OPTIONS=print_stacktrace=1

Bug confirmed present in:
MariaDB: 10.3.37 (opt), 10.4.27 (dbg), 10.4.27 (opt), 10.6.10 (dbg), 10.9.2 (opt), 10.10.2 (dbg), 10.10.2 (opt), 10.11.0 (dbg), 10.11.0 (opt)

Bug (or feature/syntax) confirmed not present in:
MariaDB: 10.3.37 (dbg), 10.5.18 (dbg), 10.5.18 (opt), 10.6.10 (opt), 10.7.6 (dbg), 10.7.6 (opt), 10.8.5 (dbg), 10.8.5 (opt), 10.9.2 (dbg)

New stacks seen (besides already existing ones, ref above)

UBSAN|pointer index expression with base X overflowed to Y|sql/item_jsonfunc.cc|Item_func_json_array_insert::val_str|Type_handler::Item_send_str|Protocol::send_result_set_row|select_send::send_data
 
UBSAN|pointer index expression with base X overflowed to Y|sql/item_jsonfunc.cc|Item_func_json_array_insert::val_str|Type_handler::Item_send_str|Type_handler_string_result::Item_send|Item::send

Comment by Roel Van de Paar [ 2023-08-09 ]

Regrettably, at various times there is very little UBSAN stack information available in the error log when the SELECT JSON_ARRAY_INSERT (0,NULL,1); testcase is used. This renders the already-maximum-resolution but still rather minimalistic UniqueID:

UBSAN|applying non-zero offset X to null pointer|sql/item_jsonfunc.cc

As such, other identical UBSAN issues in item_jsonfunc.cc may be masked (automatically filtered) till this bug is fixed.

Comment by Marko Mäkelä [ 2023-08-09 ]

I hope that the fix will not be a "fix" that causes undefined behavior on its own but is not caught by UBSAN. MDEV-26272 is an example of that, causing clang UBSAN builds to be unusable.

Generated at Thu Feb 08 09:48:08 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.