[MDEV-15005] ASAN: stack-buffer-overflow in my_strnncollsp_simple Created: 2018-01-19  Updated: 2018-03-30  Resolved: 2018-03-19

Status: Closed
Project: MariaDB Server
Component/s: Data types
Affects Version/s: 10.1, 10.2, 10.3
Fix Version/s: 10.1.32, 10.2.14, 10.3.6

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Alexander Barkov
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-15725 Another memory leak in cmp_item_sort_... Closed
Sprint: 10.2.14, 10.1.32

 Description   

set names latin1;
SELECT CONVERT(1, CHAR) IN ('100', 10, '101');

10.1 ASAN 4794e5b091c

==15851==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f58790bc8d0 at pc 0x559a1ea28641 bp 0x7f58790bc840 sp 0x7f58790bc838
READ of size 1 at 0x7f58790bc8d0 thread T6
    #0 0x559a1ea28640 in my_strnncollsp_simple /data/src/10.1/strings/ctype-simple.c:166
    #1 0x559a1d834758 in sortcmp(String const*, String const*, charset_info_st const*) /data/src/10.1/sql/sql_string.cc:762
    #2 0x559a1dc7ffdd in cmp_item_sort_string::cmp(Item*) /data/src/10.1/sql/item_cmpfunc.h:1321
    #3 0x559a1dc6d319 in Item_func_in::val_int() /data/src/10.1/sql/item_cmpfunc.cc:4444
    #4 0x559a1dbe2cd7 in Item::send(Protocol*, String*) /data/src/10.1/sql/item.cc:6589
    #5 0x559a1d4fd894 in Protocol::send_result_set_row(List<Item>*) /data/src/10.1/sql/protocol.cc:914
    #6 0x559a1d606af5 in select_send::send_data(List<Item>&) /data/src/10.1/sql/sql_class.cc:2844
    #7 0x559a1d7b1ae7 in JOIN::exec_inner() /data/src/10.1/sql/sql_select.cc:2591
    #8 0x559a1d7bae34 in JOIN::exec() /data/src/10.1/sql/sql_select.cc:2512
    #9 0x559a1d7af68d in mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /data/src/10.1/sql/sql_select.cc:3449
    #10 0x559a1d7afe7b in handle_select(THD*, LEX*, select_result*, unsigned long) /data/src/10.1/sql/sql_select.cc:384
    #11 0x559a1d6826a8 in execute_sqlcom_select /data/src/10.1/sql/sql_parse.cc:5926
    #12 0x559a1d69bfdb in mysql_execute_command(THD*) /data/src/10.1/sql/sql_parse.cc:2976
    #13 0x559a1d6b3548 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /data/src/10.1/sql/sql_parse.cc:7347
    #14 0x559a1d6b9d49 in dispatch_command(enum_server_command, THD*, char*, unsigned int) /data/src/10.1/sql/sql_parse.cc:1477
    #15 0x559a1d6c02fc in do_command(THD*) /data/src/10.1/sql/sql_parse.cc:1106
    #16 0x559a1d962844 in do_handle_one_connection(THD*) /data/src/10.1/sql/sql_connect.cc:1330
    #17 0x559a1d962d55 in handle_one_connection /data/src/10.1/sql/sql_connect.cc:1242
    #18 0x559a1e2278f6 in pfs_spawn_thread /data/src/10.1/storage/perfschema/pfs.cc:1861
    #19 0x7f5885808493 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7493)
    #20 0x7f5883bc193e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe893e)
 
Address 0x7f58790bc8d0 is located in stack of thread T6 at offset 64 in frame
    #0 0x559a1dc7fdfd in cmp_item_sort_string::cmp(Item*) /data/src/10.1/sql/item_cmpfunc.h:1314
 
  This frame has 2 object(s):
    [32, 64) 'tmp' <== Memory access at offset 64 overflows this variable
    [96, 176) 'buff'
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
Thread T6 created by T0 here:
    #0 0x7f5885a41bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
    #1 0x559a1e23314d in spawn_thread_v1 /data/src/10.1/storage/perfschema/pfs.cc:1911
 
SUMMARY: AddressSanitizer: stack-buffer-overflow /data/src/10.1/strings/ctype-simple.c:166 my_strnncollsp_simple
Shadow bytes around the buggy address:
  0x0feb8f20f8c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feb8f20f8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feb8f20f8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feb8f20f8f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feb8f20f900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0feb8f20f910: 00 00 f1 f1 f1 f1 00 00 00 00[f2]f2 f2 f2 00 00
  0x0feb8f20f920: 00 00 00 00 00 00 00 00 f4 f4 00 00 00 00 00 00
  0x0feb8f20f930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0feb8f20f940: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00
  0x0feb8f20f950: 00 f4 f4 f4 f3 f3 f3 f3 00 00 00 00 00 00 00 00
  0x0feb8f20f960: 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00 f2 f2
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Contiguous container OOB:fc
  ASan internal:           fe
==15851==ABORTING



 Comments   
Comment by Alexander Barkov [ 2018-03-19 ]

The problem is also repeatable without ASAN. Instead of the crash, it returns a wrong result:

SET NAMES latin1;
SELECT CONVERT(1, CHAR) IN ('100', 10, '1');

+--------------------------------------+
| CONVERT(1, CHAR) IN ('100', 10, '1') |
+--------------------------------------+
|                                    0 |
+--------------------------------------+

The expected result is to return 1.

If I change the number 10 to a string '10', it returns a correct result:

SET NAMES latin1;
SELECT CONVERT(1, CHAR) IN ('100', '10', '1');

+----------------------------------------+
| CONVERT(1, CHAR) IN ('100', '10', '1') |
+----------------------------------------+
|                                      1 |
+----------------------------------------+

Note, if I change the session character set to utf8, it returns a good result again, even with the number 10:

SET NAMES utf8;
SELECT CONVERT(1, CHAR) IN ('100', 10, '1');

+--------------------------------------+
| CONVERT(1, CHAR) IN ('100', 10, '1') |
+--------------------------------------+
|                                    1 |
+--------------------------------------+

Generated at Thu Feb 08 08:17:58 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.