[MDEV-17589] Stack-buffer-overflow with indexed varchar (utf8) field Created: 2018-11-01  Updated: 2018-12-19  Resolved: 2018-12-19

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 10.2, 10.3
Fix Version/s: 10.1.38, 10.0.38, 10.2.20, 10.3.12

Type: Bug Priority: Major
Reporter: Alice Sherepa Assignee: Varun Gupta (Inactive)
Resolution: Fixed Votes: 0
Labels: None


 Description    

--source include/have_innodb.inc
 
 
 
CREATE TABLE t1 (v1 varchar(1024) CHARACTER SET utf8, KEY v1 (v1)) ENGINE=InnoDB;
 
INSERT INTO t1 VALUES ('king'), ('bad');
 
 
 
SELECT MIN(x.v1) FROM (SELECT t1.* FROM t1 WHERE t1.v1 >= 'p') x;

10.2 abcd09c95a49d02bf136a21d62d38a2, built with -DWITH_ASAN=ON -DCMAKE_BUILD_TYPE=Debug

=================================================================
 
==4403==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f3a57c21060 at pc 0x7f3a70602bec bp 0x7f3a57c1fe30 sp 0x7f3a57c1f5d8
 
WRITE of size 3071 at 0x7f3a57c21060 thread T27
 
    #0 0x7f3a70602beb in __asan_memset (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x8cbeb)
 
    #1 0x564bf59ffda9 in Field_varstring::get_key_image(unsigned char*, unsigned int, Field::imagetype) /git/10.2/sql/field.cc:7812
 
    #2 0x564bf5daaa2c in matching_cond /git/10.2/sql/opt_sum.cc:828
 
    #3 0x564bf5dab6cb in find_key_for_maxmin /git/10.2/sql/opt_sum.cc:947
 
    #4 0x564bf5da805b in opt_sum_query(THD*, List<TABLE_LIST>&, List<Item>&, Item*) /git/10.2/sql/opt_sum.cc:395
 
    #5 0x564bf5521512 in JOIN::optimize_inner() /git/10.2/sql/sql_select.cc:1495
 
    #6 0x564bf551d94e in JOIN::optimize() /git/10.2/sql/sql_select.cc:1115
 
    #7 0x564bf5536c24 in mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /git/10.2/sql/sql_select.cc:3790
 
    #8 0x564bf55167e6 in handle_select(THD*, LEX*, select_result*, unsigned long) /git/10.2/sql/sql_select.cc:376
 
    #9 0x564bf549d5a1 in execute_sqlcom_select /git/10.2/sql/sql_parse.cc:6477
 
    #10 0x564bf548a5b1 in mysql_execute_command(THD*) /git/10.2/sql/sql_parse.cc:3484
 
    #11 0x564bf54a5d06 in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /git/10.2/sql/sql_parse.cc:8011
 
    #12 0x564bf5481255 in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /git/10.2/sql/sql_parse.cc:1824
 
    #13 0x564bf547e409 in do_command(THD*) /git/10.2/sql/sql_parse.cc:1378
 
    #14 0x564bf57a3671 in do_handle_one_connection(CONNECT*) /git/10.2/sql/sql_connect.cc:1335
 
    #15 0x564bf57a3079 in handle_one_connection /git/10.2/sql/sql_connect.cc:1241
 
    #16 0x564bf694835d in pfs_spawn_thread /git/10.2/storage/perfschema/pfs.cc:1862
 
    #17 0x7f3a6f20a6b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
 
    #18 0x7f3a6e69f41c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10741c)
 
 
 
Address 0x7f3a57c21060 is located in stack of thread T27 at offset 3584 in frame
 
    #0 0x564bf5da7050 in opt_sum_query(THD*, List<TABLE_LIST>&, List<Item>&, Item*) /git/10.2/sql/opt_sum.cc:243
 
 
 
  This frame has 7 object(s):
 
    [32, 36) 'range_fl'
 
    [96, 100) 'prefix_len'
 
    [160, 192) 'it'
 
    [224, 256) 'ti'
 
    [288, 320) '_db_stack_frame_'
 
    [352, 464) 'ref'
 
    [512, 3584) 'key_buff' <== Memory access at offset 3584 overflows this variable
 
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
 
      (longjmp and C++ exceptions *are* supported)
 
Thread T27 created by T0 here:
 
    #0 0x7f3a705ac253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
 
    #1 0x564bf694874a in spawn_thread_v1 /git/10.2/storage/perfschema/pfs.cc:1912
 
    #2 0x564bf528ddc6 in inline_mysql_thread_create /git/10.2/include/mysql/psi/mysql_thread.h:1239
 
    #3 0x564bf52a1fec in create_thread_to_handle_connection(CONNECT*) /git/10.2/sql/mysqld.cc:6454
 
    #4 0x564bf52a26ec in create_new_thread /git/10.2/sql/mysqld.cc:6524
 
    #5 0x564bf52a372f in handle_connections_sockets() /git/10.2/sql/mysqld.cc:6799
 
    #6 0x564bf52a153c in mysqld_main(int, char**) /git/10.2/sql/mysqld.cc:6073
 
    #7 0x564bf528c75f in main /git/10.2/sql/main.cc:25
 
    #8 0x7f3a6e5b882f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
 
 
 
SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 __asan_memset
 
Shadow bytes around the buggy address:
 
  0x0fe7caf7c1b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x0fe7caf7c1c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x0fe7caf7c1d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x0fe7caf7c1e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x0fe7caf7c1f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
=>0x0fe7caf7c200: 00 00 00 00 00 00 00 00 00 00 00 00[f3]f3 f3 f3
 
  0x0fe7caf7c210: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x0fe7caf7c220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 
  0x0fe7caf7c230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1
 
  0x0fe7caf7c240: f1 f1 01 f4 f4 f4 f2 f2 f2 f2 00 00 00 00 f2 f2
 
  0x0fe7caf7c250: f2 f2 00 00 00 00 f3 f3 f3 f3 00 00 00 00 00 00
 
Shadow byte legend (one shadow byte represents 8 application bytes):
 
  Addressable:           00
 
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
 
  Heap right redzone:      fb
 
  Freed heap region:       fd
 
  Stack left redzone:      f1
 
  Stack mid redzone:       f2
 
  Stack right redzone:     f3
 
  Stack partial redzone:   f4
 
  Stack after return:      f5
 
  Stack use after scope:   f8
 
  Global redzone:          f9
 
  Global init order:       f6
 
  Poisoned by user:        f7
 
  Container overflow:      fc
 
  Array cookie:            ac
 
  Intra object redzone:    bb
 
  ASan internal:           fe
 
==4403==ABORTING
 
----------SERVER LOG END-------------



 Comments   
Comment by Varun Gupta (Inactive) [ 2018-11-05 ]

#define MAX_KEY_LENGTH 3072 /* max possible key */

For field v1 (charset utf8)
1024*3= 3072
add 2 bytes for value length-bytes
add 1 byte for NULL-indicator byte

So the total length for the lookup key would be 3075 keys and
hence we would always encounter an overflow as the buffer
we allocate to store the key is like 

uchar key[MAX_KEY_LENGTH]

We have a check in mysql_prepare_create_table where we check 

key_part_length > file->max_key_part_length()

but the key_part_length does not include the length and NULL bytes

Comment by Varun Gupta (Inactive) [ 2018-11-05 ]

Possible solutions

Solution #1: is to count NULL-bytes and length bytes in the key_part_length in mysql_prepare_create_table().

It will cause a problem that some table definitions that were created earlier will no longer be accepted.

Solution #2: is to keep the limits on key size the same, but increase key buffer space.

Introduce a new constant as MAX_KEY_LENGTH_FOR_LOOKUP which would take into account the LENGTH and NULL BYTES.
the estimated size would be:
MAX_KEY_PARTS * (HA_KEY_BLOB_LENGTH + NULL_BYTE) + MAX_KEY_LENGTH

Comment by Sergei Petrunia [ 2018-11-05 ]

For Solution #2, maybe, increase MAX_KEY_LENGTH (as there are many places that use "char buffer[MAX_KEY_LENGTH]"), and change the mysql_prepare_create_table() to use some different constant which will still be 3072.

Comment by Sergei Golubchik [ 2018-11-05 ]

Yeah, I'd also suggest to increase MAX_KEY_LENGTH and fix the mysql_prepare_create_table() check.

Comment by Varun Gupta (Inactive) [ 2018-11-05 ]

patch
http://lists.askmonty.org/pipermail/commits/2018-November/013057.html

Comment by Sergei Petrunia [ 2018-11-08 ]

Review input http://lists.askmonty.org/pipermail/commits/2018-November/013068.html

Comment by Sergei Petrunia [ 2018-11-08 ]

varun also please change MDEV_ to MDEV in the commit message so that grepping for commit can find it.

Comment by Elena Stepanova [ 2018-11-10 ]

For the record, on Windows it looks like this:

Error:Run-Time Check Failure #2 - Stack around the variable 'key_buff' was corrupted. At d:\qa-win-debug\build\sql\opt_sum.cc:491
 
181108  2:32:55 [ERROR] mysqld got exception 0x80000003 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.2.19-MariaDB-debug-log
 
key_buffer_size=1048576
 
read_buffer_size=131072
 
max_used_connections=7
 
max_threads=65537
 
thread_count=12
 
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4233 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0xf661adda98
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
mysqld.exe!handle_rtc_failure()[my_init.c:281]
 
mysqld.exe!failwithmessage()
 
mysqld.exe!_RTC_StackFailure()
 
mysqld.exe!_RTC_CheckStackVars()
 
mysqld.exe!opt_sum_query()[opt_sum.cc:491]
 
mysqld.exe!JOIN::optimize_inner()[sql_select.cc:1495]
 
mysqld.exe!JOIN::optimize()[sql_select.cc:1115]
 
mysqld.exe!mysql_select()[sql_select.cc:3790]
 
mysqld.exe!handle_select()[sql_select.cc:364]
 
mysqld.exe!execute_sqlcom_select()[sql_parse.cc:6478]
 
mysqld.exe!mysql_execute_command()[sql_parse.cc:3484]
 
mysqld.exe!mysql_parse()[sql_parse.cc:8012]
 
mysqld.exe!dispatch_command()[sql_parse.cc:1826]
 
mysqld.exe!do_command()[sql_parse.cc:1377]
 
mysqld.exe!threadpool_process_request()[threadpool_common.cc:366]
 
mysqld.exe!tp_callback()[threadpool_common.cc:192]
 
mysqld.exe!tp_callback()[threadpool_win.cc:378]
 
mysqld.exe!work_callback()[threadpool_win.cc:452]
 
ntdll.dll!RtlFreeUnicodeString()
 
ntdll.dll!RtlFreeUnicodeString()
 
KERNEL32.DLL!BaseThreadInitThunk()
 
ntdll.dll!RtlUserThreadStart()

Comment by Varun Gupta (Inactive) [ 2018-11-12 ]

Patch
http://lists.askmonty.org/pipermail/commits/2018-November/013082.html

Comment by Sergei Petrunia [ 2018-11-14 ]

The patch is ok, see the question about the version in the email

Comment by Varun Gupta (Inactive) [ 2018-12-18 ]

Backported MDEV-10360 and MDEV-11196 to 10.0

Generated at Thu Feb 08 08:37:36 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.