[MDEV-32585] ASAN: stack-buffer-overflow in get_defaults_options on SELECT Created: 2023-10-26  Updated: 2023-10-31

Status: Confirmed
Project: MariaDB Server
Component/s: Server, Storage Engine - Spider
Affects Version/s: 10.5, 10.6, 10.10
Fix Version/s: 10.5

Type: Bug Priority: Major
Reporter: Roel Van de Paar Assignee: Sergei Golubchik
Resolution: Unresolved Votes: 0
Labels: ASAN, memory_corruption, not-10.4

Issue Links:
Relates
relates to MDEV-28856 Spider: Implement more engine-defined... Closed

 Description    

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE TABLE t (c INT) ENGINE=Spider COMMENT='default_group "none"';
 
SELECT * FROM t;

Or, alternatively (new syntax):

INSTALL PLUGIN Spider SONAME 'ha_spider.so';
 
CREATE TABLE t (c INT) ENGINE=Spider DEFAULT_GROUP=none;
 
SELECT * FROM t;

Leads to:

bb-11.3-mdev-28856-and-fixes cc08a83ef4225960dccb46bd68fc549160d21841 (Optimized, UBASAN)

 
==1161763==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x14e66e7fdad8 at pc 0x55a3c37fdb72 bp 0x14e66e7fd730 sp 0x14e66e7fd720
 
READ of size 8 at 0x14e66e7fdad8 thread T37
 
    #0 0x55a3c37fdb71 in get_defaults_options /test/bb-11.3-mdev-28856-and-fixes_opt_san/mysys/my_default.c:293
 
    #1 0x55a3c37fde06 in my_load_defaults /test/bb-11.3-mdev-28856-and-fixes_opt_san/mysys/my_default.c:417
 
    #2 0x55a3c104fe66 in mysql_read_default_options /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql-common/client.c:912
 
    #3 0x55a3c10632cb in server_mysql_real_connect /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql-common/client.c:2730
 
    #4 0x14e66df52dab in spider_db_mbase::connect(char*, char*, char*, long, char*, char*, int, long long) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_db_mysql.cc:1956
 
    #5 0x14e66dc83401 in spider_db_connect(st_spider_share const*, st_spider_conn*, int) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_db_conn.cc:143
 
    #6 0x14e66dc8826a in spider_db_conn_queue_action(st_spider_conn*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_db_conn.cc:256
 
    #7 0x14e66dcba3b5 in spider_db_before_query(st_spider_conn*, int*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_db_conn.cc:579
 
    #8 0x14e66dcbb380 in spider_db_set_names_internal(st_spider_transaction*, st_spider_share*, st_spider_conn*, int, int*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_db_conn.cc:807
 
    #9 0x14e66e005829 in spider_mbase_handler::show_table_status(int, int, unsigned int) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_db_mysql.cc:13489
 
    #10 0x14e66ddafce8 in spider_get_sts(st_spider_share*, int, long, ha_spider*, double, int, int, int, unsigned int) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_table.cc:7091
 
    #11 0x14e66ddd85c6 in spider_share_get_sts_crd(THD*, ha_spider*, st_spider_share*, TABLE*, bool, bool, int*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_table.cc:5270
 
    #12 0x14e66dddcf0b in spider_init_share(char const*, TABLE*, THD*, ha_spider*, int*, st_spider_share*, TABLE_SHARE*, bool) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_table.cc:5429
 
    #13 0x14e66ddde35b in spider_get_share(char const*, TABLE*, THD*, ha_spider*, int*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/spd_table.cc:5520
 
    #14 0x14e66ded490c in ha_spider::open(char const*, int, unsigned int) /test/bb-11.3-mdev-28856-and-fixes_opt_san/storage/spider/ha_spider.cc:312
 
    #15 0x55a3c12190b2 in handler::ha_open(TABLE*, char const*, int, unsigned int, st_mem_root*, List<String>*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/handler.cc:3499
 
    #16 0x55a3c0408817 in open_table_from_share(THD*, TABLE_SHARE*, st_mysql_const_lex_string const*, unsigned int, unsigned int, unsigned int, TABLE*, bool, List<String>*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/table.cc:4567
 
    #17 0x55a3bf8119fe in open_table(THD*, TABLE_LIST*, Open_table_context*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_base.cc:2218
 
    #18 0x55a3bf828539 in open_and_process_table /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_base.cc:4148
 
    #19 0x55a3bf828539 in open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_base.cc:4633
 
    #20 0x55a3bf82c6bc in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_base.cc:5607
 
    #21 0x55a3bfbd0aad in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_base.h:525
 
    #22 0x55a3bfbd0aad in execute_sqlcom_select /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_parse.cc:5932
 
    #23 0x55a3bfc34f03 in mysql_execute_command(THD*, bool) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_parse.cc:3911
 
    #24 0x55a3bfc43a92 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_parse.cc:7734
 
    #25 0x55a3bfc4fafd in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_parse.cc:1893
 
    #26 0x55a3bfc5ad28 in do_command(THD*, bool) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_parse.cc:1406
 
    #27 0x55a3c058a83c in do_handle_one_connection(CONNECT*, bool) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_connect.cc:1445
 
    #28 0x55a3c058ce3c in handle_one_connection /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/sql_connect.cc:1347
 
    #29 0x14e693a94ac2 in start_thread nptl/pthread_create.c:442
 
    #30 0x14e693b26a3f  (/lib/x86_64-linux-gnu/libc.so.6+0x126a3f)
 
 
 
Address 0x14e66e7fdad8 is located in stack of thread T37 at offset 88 in frame
 
    #0 0x55a3c104fc3f in mysql_read_default_options /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql-common/client.c:895
 
 
 
  This frame has 6 object(s):
 
    [32, 36) 'argc' (line 896)
 
    [48, 56) 'argv' (line 897)
 
    [80, 88) 'argv_buff' (line 897) <== Memory access at offset 88 overflows this variable
 
    [112, 152) 'groups' (line 898)
 
    [192, 704) 'buff' (line 1063)
 
    [768, 1280) 'buff2' (line 1063)
 
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
 
      (longjmp and C++ exceptions *are* supported)
 
Thread T37 created by T0 here:
 
    #0 0x55a3bf337715 in __interceptor_pthread_create (/test/28856_P2_UBASAN_MD211023-mariadb-11.3.0-linux-x86_64-opt/bin/mariadbd+0x7d1d715)
 
    #1 0x55a3bf3ec33e in create_thread_to_handle_connection(CONNECT*) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/mysqld.cc:6150
 
    #2 0x55a3bf3ff10f in handle_accepted_socket(st_mysql_socket, st_mysql_socket) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/mysqld.cc:6274
 
    #3 0x55a3bf400067 in handle_connections_sockets() /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/mysqld.cc:6398
 
    #4 0x55a3bf40304d in mysqld_main(int, char**) /test/bb-11.3-mdev-28856-and-fixes_opt_san/sql/mysqld.cc:6045
 
    #5 0x14e693a29d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
 
 
 
SUMMARY: AddressSanitizer: stack-buffer-overflow /test/bb-11.3-mdev-28856-and-fixes_opt_san/mysys/my_default.c:293 in get_defaults_options
 
Shadow bytes around the buggy address:
 
  0x029d4dcf7b00: 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 f2 f2 f2
 
  0x029d4dcf7b10: f2 f2 00 00 00 00 f2 f2 f2 f2 00 00 00 00 00 00
 
  0x029d4dcf7b20: 00 f2 f2 f2 f2 f2 00 00 00 00 00 00 00 00 f2 f2
 
  0x029d4dcf7b30: f2 f2 00 00 f2 f2 00 00 f3 f3 00 00 00 00 00 00
 
  0x029d4dcf7b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
=>0x029d4dcf7b50: f1 f1 f1 f1 04 f2 00 f2 f2 f2 00[f2]f2 f2 00 00
 
  0x029d4dcf7b60: 00 00 00 f2 f2 f2 f2 f2 00 00 00 00 00 00 00 00
 
  0x029d4dcf7b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x029d4dcf7b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x029d4dcf7b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x029d4dcf7ba0: 00 00 00 00 00 00 00 00 f2 f2 f2 f2 f2 f2 f2 f2
 
Shadow byte legend (one shadow byte represents 8 application bytes):
 
  Addressable:           00
 
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
 
  Freed heap region:       fd
 
  Stack left redzone:      f1
 
  Stack mid redzone:       f2
 
  Stack right redzone:     f3
 
  Stack after return:      f5
 
  Stack use after scope:   f8
 
  Global redzone:          f9
 
  Global init order:       f6
 
  Poisoned by user:        f7
 
  Container overflow:      fc
 
  Array cookie:            ac
 
  Intra object redzone:    bb
 
  ASan internal:           fe
 
  Left alloca redzone:     ca
 
  Right alloca redzone:    cb
 
  Shadow gap:              cc
 
==1161763==ABORTING
 
231026 17:46:06 [ERROR] mysqld got signal 6 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
 
 
Server version: 11.3.0-MariaDB source revision: cc08a83ef4225960dccb46bd68fc549160d21841
 
key_buffer_size=134217728
 
read_buffer_size=131072
 
max_used_connections=1
 
max_threads=153
 
thread_count=21
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467990 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x62b00015e218
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x14e66e803760 thread_stack 0x5fc00
 
asan_interceptors.o:0(__interceptor_backtrace.part.0)[0x55a3bf324c9e]
 
mysys/stacktrace.c:216(my_print_stacktrace)[0x55a3c387aaa2]
 
sql/signal_handler.cc:241(handle_fatal_signal)[0x55a3c11e60e7]
 
libc_sigaction.c:0(__restore_rt)[0x14e693a42520]
 
nptl/pthread_kill.c:44(__pthread_kill_implementation)[0x14e693a969fc]
 
posix/raise.c:27(__GI_raise)[0x14e693a42476]
 
stdlib/abort.c:81(__GI_abort)[0x14e693a287f3]
 
/test/28856_P2_UBASAN_MD211023-mariadb-11.3.0-linux-x86_64-opt/bin/mariadbd(+0x7d97782)[0x55a3bf3b1782]
 
/test/28856_P2_UBASAN_MD211023-mariadb-11.3.0-linux-x86_64-opt/bin/mariadbd(+0x7da333c)[0x55a3bf3bd33c]
 
/test/28856_P2_UBASAN_MD211023-mariadb-11.3.0-linux-x86_64-opt/bin/mariadbd(+0x7d827ec)[0x55a3bf39c7ec]
 
/test/28856_P2_UBASAN_MD211023-mariadb-11.3.0-linux-x86_64-opt/bin/mariadbd(+0x7d82085)[0x55a3bf39c085]
 
/test/28856_P2_UBASAN_MD211023-mariadb-11.3.0-linux-x86_64-opt/bin/mariadbd(__asan_report_load8+0x3b)[0x55a3bf39cecb]
 
mysys/my_default.c:293(get_defaults_options)[0x55a3c37fdb72]
 
mysys/my_default.c:417(my_load_defaults)[0x55a3c37fde07]
 
sql-common/client.c:913(mysql_read_default_options)[0x55a3c104fe67]
 
sql-common/client.c:2734(server_mysql_real_connect)[0x55a3c10632cc]
 
spider/spd_db_mysql.cc:1955(spider_db_mbase::connect(char*, char*, char*, long, char*, char*, int, long long))[0x14e66df52dac]
 
spider/spd_db_conn.cc:143(spider_db_connect(st_spider_share const*, st_spider_conn*, int))[0x14e66dc83402]
 
spider/spd_db_conn.cc:256(spider_db_conn_queue_action(st_spider_conn*))[0x14e66dc8826b]
 
spider/spd_db_conn.cc:579(spider_db_before_query(st_spider_conn*, int*))[0x14e66dcba3b6]
 
spider/spd_db_conn.cc:807(spider_db_set_names_internal(st_spider_transaction*, st_spider_share*, st_spider_conn*, int, int*))[0x14e66dcbb381]
 
spider/spd_db_mysql.cc:13489(spider_mbase_handler::show_table_status(int, int, unsigned int))[0x14e66e00582a]
 
spider/spd_table.cc:7093(spider_get_sts(st_spider_share*, int, long, ha_spider*, double, int, int, int, unsigned int))[0x14e66ddafce9]
 
spider/spd_table.cc:5270(spider_share_get_sts_crd(THD*, ha_spider*, st_spider_share*, TABLE*, bool, bool, int*))[0x14e66ddd85c7]
 
spider/spd_table.cc:5429(spider_init_share(char const*, TABLE*, THD*, ha_spider*, int*, st_spider_share*, TABLE_SHARE*, bool))[0x14e66dddcf0c]
 
spider/spd_table.cc:5520(spider_get_share(char const*, TABLE*, THD*, ha_spider*, int*))[0x14e66ddde35c]
 
spider/ha_spider.cc:312(ha_spider::open(char const*, int, unsigned int))[0x14e66ded490d]
 
sql/handler.cc:3499(handler::ha_open(TABLE*, char const*, int, unsigned int, st_mem_root*, List<String>*))[0x55a3c12190b3]
 
sql/table.cc:4570(open_table_from_share(THD*, TABLE_SHARE*, st_mysql_const_lex_string const*, unsigned int, unsigned int, unsigned int, TABLE*, bool, List<String>*))[0x55a3c0408818]
 
sql/sql_base.cc:2226(open_table(THD*, TABLE_LIST*, Open_table_context*))[0x55a3bf8119ff]
 
sql/sql_base.cc:3942(open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*))[0x55a3bf82853a]
 
sql/sql_base.cc:5607(open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*))[0x55a3bf82c6bd]
 
sql/sql_base.h:525(open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int))[0x55a3bfbd0aae]
 
sql/sql_parse.cc:3911(mysql_execute_command(THD*, bool))[0x55a3bfc34f04]
 
sql/sql_parse.cc:7751(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x55a3bfc43a93]
 
sql/sql_parse.cc:1895(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool))[0x55a3bfc4fafe]
 
sql/sql_parse.cc:1408(do_command(THD*, bool))[0x55a3bfc5ad29]
 
sql/sql_connect.cc:1445(do_handle_one_connection(CONNECT*, bool))[0x55a3c058a83d]
 
sql/sql_connect.cc:1353(handle_one_connection)[0x55a3c058ce3d]
 
nptl/pthread_create.c:442(start_thread)[0x14e693a94ac3]
 
x86_64/clone3.S:83(__clone3)[0x14e693b26a40]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x629000244238): SELECT * FROM t



 Comments   
Comment by Roel Van de Paar [ 2023-10-26 ]

While a debug preview-11.3-preview at rev 465f9beea1c43a1dad74330aa2dc30927bc224f5 does fail in the same way, only the optimized build of bb-11.3-mdev-28856-and-fixes fails.

Comment by Yuchen Pei [ 2023-10-26 ]

Here's how it happens, at 11.3 465f9beea1c43a1dad74330aa2dc30927bc224f5:

  • In server_mysql_real_connect(), if the options contain either a
    default file or a default group, it will call
    mysql_read_default_options() (A)

MYSQL * STDCALL
 
CLI_MYSQL_REAL_CONNECT(MYSQL *mysql,const char *host, const char *user,
 
		       const char *passwd, const char *db,
 
		       uint port, const char *unix_socket,ulong client_flag)
 
{
 
//  [... 31 lines elided]
 
  /* use default options */
 
  if (mysql->options.my_cnf_file || mysql->options.my_cnf_group)
 
  {
 
    mysql_read_default_options(&mysql->options,
 
			       (mysql->options.my_cnf_file ?
 
				mysql->options.my_cnf_file : "my"),
 
			       mysql->options.my_cnf_group); // (A)
 
//  [... 5 lines elided]
 
  }
 
//  [... 481 lines elided]
 
}

  • In mysql_read_default_options(), it declares an singleton array of
    char* argv_buff (B), and assign it to a char** argv (C), before
    calling my_load_defaults() (D)

void mysql_read_default_options(struct st_mysql_options *options,
 
				const char *filename,const char *group)
 
{
 
  int argc;
 
  char *argv_buff[1],**argv;    // (B)
 
  const char *groups[5];
 
  DBUG_ENTER("mysql_read_default_options");
 
  DBUG_PRINT("enter",("file: %s  group: %s",filename,group ? group :"NULL"));
 
 
 
  compile_time_assert(OPT_keep_this_one_last ==
 
                      array_elements(default_options));
 
 
 
  argc=1; argv=argv_buff; argv_buff[0]= (char*) "client"; // (C)
 
  groups[0]= (char*) "client";
 
  groups[1]= (char*) "client-server";
 
  groups[2]= (char*) "client-mariadb";
 
  groups[3]= (char*) group;
 
  groups[4]=0;
 
 
 
  my_load_defaults(filename, groups, &argc, &argv, NULL); // (D)
 
//  [... 176 lines elided]
 
}

  • In my_load_defaults(), it calls get_defaults_options() with argv (E)

int my_load_defaults(const char *conf_file, const char **groups, int *argc,
 
                     char ***argv, const char ***default_directories)
 
{
 
  DYNAMIC_ARRAY args;
 
  int args_used= 0;
 
  int error= 0;
 
  MEM_ROOT alloc;
 
  char *ptr,**res;
 
  const char **dirs;
 
  DBUG_ENTER("my_load_defaults");
 
 
 
  init_alloc_root(key_memory_defaults, &alloc, 512, 0, MYF(0));
 
  if ((dirs= init_default_directories(&alloc)) == NULL)
 
    goto err;
 
 
 
  args_used= get_defaults_options(*argv); // (E)
 
//  [... 75 lines elided]
 
}

  • In get_defaults_options(), it increments argv (F), before dereferencing
    it (G), but remember it was assigned a singleton array, so we have the
    overflow (boom)

int get_defaults_options(char **argv)
 
{
 
  static char file_buffer[FN_REFLEN];
 
  static char extra_file_buffer[FN_REFLEN];
 
  char **orig_argv= argv;
 
 
 
  argv++; /* Skip program name */ // (F)
 
 
 
  my_defaults_file= my_defaults_group_suffix= my_defaults_extra_file= 0;
 
  my_no_defaults= my_print_defaults= FALSE;
 
 
 
  if (*argv && !strcmp(*argv, "--no-defaults")) // (G)
 
//  [... 42 lines elided]
 
}

So it does not look like a spider bug.

Why has this not been discovered before?

Comment by Yuchen Pei [ 2023-10-27 ]

It is reproducible on

  • 10.10 d2a867cdf08850563a39381aee09b4528442feb8
  • 10.6 ec2574fd8fd46da765fd60b0ca9650b729073401
  • 10.5 b5e43a1d3539c7254c298dc9f63a261281345d59

but not reproducible on

  • 10.4 c9f87b881315af80e73da3b0b74b4ca36f1cab18
Comment by Yuchen Pei [ 2023-10-31 ]

It seems to me the following commit that adds the line
args_used= get_defaults_options(*argv); to my_load_defaults()
is the bad commit. Indeed, building at this commit with ASAN (but not
with UBSAN which could cause a leaksanitizer failure that masks the
overflow failure) results in the same overflow failure, but the
failure is not reproducible at its parent commit.

3e56972712395d371f82cda564b039dcbda3100a
 
Commit:     Sergei Golubchik <serg@mariadb.org>
 
CommitDate: Mon Oct 14 10:29:30 2019 +0200
 
cleanup: unify --defaults* option handling
 
 
 
process all --defaults* options uniformly,
 
get rid of special case for --no-defaults and --print-defaults
 
use realpath instead of blindly concatenating pwd and relative path.

Generated at Thu Feb 08 10:32:28 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.