[MDEV-24262] Server crashes in st_join_table::cleanup upon erroneous GROUP_CONCAT Created: 2020-11-20  Updated: 2022-04-07  Resolved: 2022-04-07

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 10.2, 10.3, 10.4, 10.5, 10.6
Fix Version/s: 10.2.44, 10.3.35, 10.4.25, 10.5.16, 10.6.8, 10.7.4, 10.8.3

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Oleg Smirnov
Resolution: Duplicate Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-24560 SIGSEGV in st_join_table::cleanup + s... Closed

 Description   

CREATE TABLE t1 (a INT, b INT);
INSERT INTO t1 VALUES (1,10),(2,20);
--error ER_BAD_FIELD_ERROR
SELECT b, GROUP_CONCAT(b ORDER BY 2), MIN(a) AS f FROM t1 GROUP BY b ORDER BY f;
 
# Cleanup
DROP TABLE t1;

10.2 1248c654

#3  <signal handler called>
#4  0x0000561a6f0e469b in st_join_table::cleanup (this=0x7f144c015680) at /data/src/10.2/sql/sql_select.cc:11957
#5  0x0000561a6f0e5608 in JOIN::cleanup (this=0x7f144c013ca8, full=true) at /data/src/10.2/sql/sql_select.cc:12370
#6  0x0000561a6f0ce49d in JOIN::destroy (this=0x7f144c013ca8) at /data/src/10.2/sql/sql_select.cc:3667
#7  0x0000561a6f1758c5 in st_select_lex::cleanup (this=0x7f144c0050c8) at /data/src/10.2/sql/sql_union.cc:1567
#8  0x0000561a6f0cea71 in mysql_select (thd=0x7f144c000d90, tables=0x7f144c0132d0, wild_num=0, fields=..., conds=0x0, og_num=2, order=0x7f144c013b68, group=0x7f144c013a08, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7f144c013c88, unit=0x7f144c004988, select_lex=0x7f144c0050c8) at /data/src/10.2/sql/sql_select.cc:3848
#9  0x0000561a6f0c2b5e in handle_select (thd=0x7f144c000d90, lex=0x7f144c0048c8, result=0x7f144c013c88, setup_tables_done_option=0) at /data/src/10.2/sql/sql_select.cc:361
#10 0x0000561a6f08d4c5 in execute_sqlcom_select (thd=0x7f144c000d90, all_tables=0x7f144c0132d0) at /data/src/10.2/sql/sql_parse.cc:6249
#11 0x0000561a6f083e14 in mysql_execute_command (thd=0x7f144c000d90) at /data/src/10.2/sql/sql_parse.cc:3558
#12 0x0000561a6f091253 in mysql_parse (thd=0x7f144c000d90, rawbuf=0x7f144c0126f8 "SELECT b, GROUP_CONCAT(b ORDER BY 2), MIN(a) AS f FROM t1 GROUP BY b ORDER BY f", length=79, parser_state=0x7f145e23d5f0, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:7761
#13 0x0000561a6f07f4e8 in dispatch_command (command=COM_QUERY, thd=0x7f144c000d90, packet=0x7f144c008b51 "SELECT b, GROUP_CONCAT(b ORDER BY 2), MIN(a) AS f FROM t1 GROUP BY b ORDER BY f", packet_length=79, is_com_multi=false, is_next_command=false) at /data/src/10.2/sql/sql_parse.cc:1827
#14 0x0000561a6f07dfe3 in do_command (thd=0x7f144c000d90) at /data/src/10.2/sql/sql_parse.cc:1381
#15 0x0000561a6f1d8309 in do_handle_one_connection (connect=0x561a72298980) at /data/src/10.2/sql/sql_connect.cc:1336
#16 0x0000561a6f1d806e in handle_one_connection (arg=0x561a72298980) at /data/src/10.2/sql/sql_connect.cc:1241
#17 0x0000561a6f9ffa76 in pfs_spawn_thread (arg=0x561a7227bd40) at /data/src/10.2/storage/perfschema/pfs.cc:1869
#18 0x00007f14645f3609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#19 0x00007f14641cf293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Non-debug build on my machine produces double free or corruption (out) error and hangs upon crash reporting. The stack trace from the not-completely-dead process:

10.2 non-debug 1248c654

#0  __lll_lock_wait_private (futex=futex@entry=0x7fe18e328b80 <main_arena>) at ./lowlevellock.c:35
#1  0x00007fe18e1da049 in __malloc_fork_lock_parent () at arena.c:156
#2  0x00007fe18e2231aa in __libc_fork () at ../sysdeps/nptl/fork.c:73
#3  0x0000560911919244 in start_addr2line_fork (binary_path=0x7ffc59dd325e "/data/bld/10.2-rel-nightly/bin/mysqld") at /data/src/10.2/mysys/my_addr_resolve.c:192
#4  start_addr2line_fork (binary_path=0x7ffc59dd325e "/data/bld/10.2-rel-nightly/bin/mysqld") at /data/src/10.2/mysys/my_addr_resolve.c:175
#5  0x0000560911919378 in my_addr_resolve (ptr=0x5609118fe532 <my_print_stacktrace+50>, loc=loc@entry=0x7fe1882af9c0) at /data/src/10.2/mysys/my_addr_resolve.c:241
#6  0x00005609118fe58b in print_with_addr_resolve (n=<optimized out>, addrs=0x7fe1882af9e0) at /data/src/10.2/mysys/stacktrace.c:158
#7  my_print_stacktrace (stack_bottom=<optimized out>, thread_stack=299008) at /data/src/10.2/mysys/stacktrace.c:176
#8  0x00005609113a4b4b in handle_fatal_signal (sig=6) at /data/src/10.2/sql/signal_handler.cc:206
#9  <signal handler called>
#10 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#11 0x00007fe18e162859 in __GI_abort () at abort.c:79
#12 0x00007fe18e1cd3ee in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fe18e2f7285 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#13 0x00007fe18e1d547c in malloc_printerr (str=str@entry=0x7fe18e2f9670 "double free or corruption (out)") at malloc.c:5347
#14 0x00007fe18e1d7120 in _int_free (av=0x7fe18e328b80 <main_arena>, p=0x7fe178080620, have_lock=<optimized out>) at malloc.c:4314
#15 0x00005609118f28cd in free_root (root=root@entry=0x7fe1882b0860, MyFlags=MyFlags@entry=0) at /data/src/10.2/mysys/my_alloc.c:395
#16 0x000056091123691e in free_tmp_table (thd=0x7fe178000c48, entry=0x7fe1781768b0) at /data/src/10.2/sql/sql_select.cc:18190
#17 0x0000560911254aef in JOIN::cleanup (this=this@entry=0x7fe178010900, full=full@entry=true) at /data/src/10.2/sql/sql_select.cc:12383
#18 0x0000560911254e19 in JOIN::destroy (this=0x7fe178010900) at /data/src/10.2/sql/sql_select.cc:3667
#19 0x00005609112997fd in st_select_lex::cleanup (this=this@entry=0x7fe178004dc0) at /data/src/10.2/sql/sql_union.cc:1567
#20 0x000056091125616a in mysql_select (thd=0x7fe178000c48, tables=0x7fe17800ff28, wild_num=0, fields=..., conds=0x0, og_num=2, order=0x7fe1780107c0, group=0x7fe178010660, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7fe1780108e0, unit=0x7fe178004680, select_lex=0x7fe178004dc0) at /data/src/10.2/sql/sql_select.cc:3848
#21 0x0000560911256a17 in handle_select (thd=thd@entry=0x7fe178000c48, lex=lex@entry=0x7fe1780045c0, result=result@entry=0x7fe1780108e0, setup_tables_done_option=setup_tables_done_option@entry=0) at /data/src/10.2/sql/sql_select.cc:361
#22 0x00005609111ec131 in execute_sqlcom_select (thd=0x7fe178000c48, all_tables=0x7fe17800ff28) at /data/src/10.2/sql/sql_parse.cc:6249
#23 0x00005609111f9782 in mysql_execute_command (thd=0x7fe178000c48) at /data/src/10.2/sql/sql_parse.cc:3558
#24 0x00005609111fc6db in mysql_parse (thd=thd@entry=0x7fe178000c48, rawbuf=<optimized out>, length=<optimized out>, parser_state=parser_state@entry=0x7fe1882b25b0, is_com_multi=is_com_multi@entry=false, is_next_command=is_next_command@entry=false) at /data/src/10.2/sql/sql_parse.cc:7761
#25 0x00005609111ff96d in dispatch_command (command=COM_QUERY, thd=0x7fe178000c48, packet=<optimized out>, packet_length=<optimized out>, is_com_multi=<optimized out>, is_next_command=<optimized out>) at /data/src/10.2/sql/sql_class.h:1095
#26 0x0000560911200af7 in do_command (thd=0x7fe178000c48) at /data/src/10.2/sql/sql_parse.cc:1381
#27 0x00005609112d9286 in do_handle_one_connection (connect=connect@entry=0x5609133ea338) at /data/src/10.2/sql/sql_connect.cc:1336
#28 0x00005609112d93ff in handle_one_connection (arg=arg@entry=0x5609133ea338) at /data/src/10.2/sql/sql_connect.cc:1241
#29 0x0000560911873b86 in pfs_spawn_thread (arg=0x5609133fb1b8) at /data/src/10.2/storage/perfschema/pfs.cc:1869
#30 0x00007fe18e66a609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#31 0x00007fe18e25f293 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

ASAN build produces heap-use-after-free:

10.2 1248c654 ASAN

==3580398==ERROR: AddressSanitizer: heap-use-after-free on address 0x623000016d90 at pc 0x5604e83ea536 bp 0x7f7596333c30 sp 0x7f7596333c20
READ of size 8 at 0x623000016d90 thread T5
    #0 0x5604e83ea535 in st_join_table::cleanup() /data/src/10.2/sql/sql_select.cc:11957
    #1 0x5604e83ed440 in JOIN::cleanup(bool) /data/src/10.2/sql/sql_select.cc:12370
    #2 0x5604e83b002b in JOIN::destroy() /data/src/10.2/sql/sql_select.cc:3667
    #3 0x5604e856ed3f in st_select_lex::cleanup() /data/src/10.2/sql/sql_union.cc:1567
    #4 0x5604e83b10ee in mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /data/src/10.2/sql/sql_select.cc:3848
    #5 0x5604e838d9dd in handle_select(THD*, LEX*, select_result*, unsigned long) /data/src/10.2/sql/sql_select.cc:361
    #6 0x5604e8304dc6 in execute_sqlcom_select /data/src/10.2/sql/sql_parse.cc:6249
    #7 0x5604e82f1d0b in mysql_execute_command(THD*) /data/src/10.2/sql/sql_parse.cc:3558
    #8 0x5604e830e2fb in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /data/src/10.2/sql/sql_parse.cc:7761
    #9 0x5604e82e73fe in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /data/src/10.2/sql/sql_parse.cc:1827
    #10 0x5604e82e41bd in do_command(THD*) /data/src/10.2/sql/sql_parse.cc:1381
    #11 0x5604e866b73b in do_handle_one_connection(CONNECT*) /data/src/10.2/sql/sql_connect.cc:1336
    #12 0x5604e866affe in handle_one_connection /data/src/10.2/sql/sql_connect.cc:1241
    #13 0x5604e9a05c49 in pfs_spawn_thread /data/src/10.2/storage/perfschema/pfs.cc:1869
    #14 0x7f75a1082608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477
    #15 0x7f75a0c5c292 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)
 
0x623000016d90 is located 144 bytes inside of 6092-byte region [0x623000016d00,0x6230000184cc)
freed by thread T5 here:
    #0 0x7f75a15e97cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
    #1 0x5604e9b23a40 in free_memory /data/src/10.2/mysys/safemalloc.c:279
    #2 0x5604e9b22f8c in sf_free /data/src/10.2/mysys/safemalloc.c:197
    #3 0x5604e9aef2d4 in my_free /data/src/10.2/mysys/my_malloc.c:218
    #4 0x5604e9acd09d in free_root /data/src/10.2/mysys/my_alloc.c:395
    #5 0x5604e84144a8 in free_tmp_table(THD*, TABLE*) /data/src/10.2/sql/sql_select.cc:18190
    #6 0x5604e83aa021 in JOIN::create_postjoin_aggr_table(st_join_table*, List<Item>*, st_order*, bool, bool, bool) /data/src/10.2/sql/sql_select.cc:3045
    #7 0x5604e83a4f19 in JOIN::make_aggr_tables_info() /data/src/10.2/sql/sql_select.cc:2578
    #8 0x5604e83a0e81 in JOIN::optimize_inner() /data/src/10.2/sql/sql_select.cc:2249
    #9 0x5604e83956c1 in JOIN::optimize() /data/src/10.2/sql/sql_select.cc:1117
    #10 0x5604e83b0d4d in mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /data/src/10.2/sql/sql_select.cc:3822
    #11 0x5604e838d9dd in handle_select(THD*, LEX*, select_result*, unsigned long) /data/src/10.2/sql/sql_select.cc:361
    #12 0x5604e8304dc6 in execute_sqlcom_select /data/src/10.2/sql/sql_parse.cc:6249
    #13 0x5604e82f1d0b in mysql_execute_command(THD*) /data/src/10.2/sql/sql_parse.cc:3558
    #14 0x5604e830e2fb in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /data/src/10.2/sql/sql_parse.cc:7761
    #15 0x5604e82e73fe in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /data/src/10.2/sql/sql_parse.cc:1827
    #16 0x5604e82e41bd in do_command(THD*) /data/src/10.2/sql/sql_parse.cc:1381
    #17 0x5604e866b73b in do_handle_one_connection(CONNECT*) /data/src/10.2/sql/sql_connect.cc:1336
    #18 0x5604e866affe in handle_one_connection /data/src/10.2/sql/sql_connect.cc:1241
    #19 0x5604e9a05c49 in pfs_spawn_thread /data/src/10.2/storage/perfschema/pfs.cc:1869
    #20 0x7f75a1082608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477
 
previously allocated by thread T5 here:
    #0 0x7f75a15e9bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
    #1 0x5604e9b228fe in sf_malloc /data/src/10.2/mysys/safemalloc.c:118
    #2 0x5604e9aee863 in my_malloc /data/src/10.2/mysys/my_malloc.c:101
    #3 0x5604e9acbed5 in alloc_root /data/src/10.2/mysys/my_alloc.c:243
    #4 0x5604e9acc74a in multi_alloc_root /data/src/10.2/mysys/my_alloc.c:312
    #5 0x5604e8406b10 in create_tmp_table(THD*, TMP_TABLE_PARAM*, List<Item>&, st_order*, bool, bool, unsigned long long, unsigned long long, char const*, bool, bool) /data/src/10.2/sql/sql_select.cc:16800
    #6 0x5604e83a902a in JOIN::create_postjoin_aggr_table(st_join_table*, List<Item>*, st_order*, bool, bool, bool) /data/src/10.2/sql/sql_select.cc:2973
    #7 0x5604e83a4f19 in JOIN::make_aggr_tables_info() /data/src/10.2/sql/sql_select.cc:2578
    #8 0x5604e83a0e81 in JOIN::optimize_inner() /data/src/10.2/sql/sql_select.cc:2249
    #9 0x5604e83956c1 in JOIN::optimize() /data/src/10.2/sql/sql_select.cc:1117
    #10 0x5604e83b0d4d in mysql_select(THD*, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /data/src/10.2/sql/sql_select.cc:3822
    #11 0x5604e838d9dd in handle_select(THD*, LEX*, select_result*, unsigned long) /data/src/10.2/sql/sql_select.cc:361
    #12 0x5604e8304dc6 in execute_sqlcom_select /data/src/10.2/sql/sql_parse.cc:6249
    #13 0x5604e82f1d0b in mysql_execute_command(THD*) /data/src/10.2/sql/sql_parse.cc:3558
    #14 0x5604e830e2fb in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) /data/src/10.2/sql/sql_parse.cc:7761
    #15 0x5604e82e73fe in dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool) /data/src/10.2/sql/sql_parse.cc:1827
    #16 0x5604e82e41bd in do_command(THD*) /data/src/10.2/sql/sql_parse.cc:1381
    #17 0x5604e866b73b in do_handle_one_connection(CONNECT*) /data/src/10.2/sql/sql_connect.cc:1336
    #18 0x5604e866affe in handle_one_connection /data/src/10.2/sql/sql_connect.cc:1241
    #19 0x5604e9a05c49 in pfs_spawn_thread /data/src/10.2/storage/perfschema/pfs.cc:1869
    #20 0x7f75a1082608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477
 
Thread T5 created by T0 here:
    #0 0x7f75a1516805 in pthread_create (/lib/x86_64-linux-gnu/libasan.so.5+0x3a805)
    #1 0x5604e9a0603a in spawn_thread_v1 /data/src/10.2/storage/perfschema/pfs.cc:1919
    #2 0x5604e8089083 in inline_mysql_thread_create /data/src/10.2/include/mysql/psi/mysql_thread.h:1246
    #3 0x5604e80a0c2b in create_thread_to_handle_connection(CONNECT*) /data/src/10.2/sql/mysqld.cc:6567
    #4 0x5604e80a13c6 in create_new_thread /data/src/10.2/sql/mysqld.cc:6637
    #5 0x5604e80a2558 in handle_connections_sockets() /data/src/10.2/sql/mysqld.cc:6895
    #6 0x5604e809ff7c in mysqld_main(int, char**) /data/src/10.2/sql/mysqld.cc:6186
    #7 0x5604e808793c in main /data/src/10.2/sql/main.cc:25
    #8 0x7f75a0b610b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
 
SUMMARY: AddressSanitizer: heap-use-after-free /data/src/10.2/sql/sql_select.cc:11957 in st_join_table::cleanup()
Shadow bytes around the buggy address:
  0x0c467fffad60: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa
  0x0c467fffad70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fffad80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fffad90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c467fffada0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c467fffadb0: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fffadc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fffadd0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fffade0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fffadf0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c467fffae00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==3580398==ABORTING

Reproducible on 10.2+ with at least InnoDB and MyISAM. The failure appeared in the very early 10.2 branch after this merge:

commit b532be9f8ce4872be4a3f1ef2fa296a1de347859 (HEAD)
Merge: a4c81986aad 0f62eee2f87
Author: Sergei Petrunia
Date:   Tue Apr 12 00:00:53 2016 +0200
 
    Merge ../10.2-window-funcs-r12 into 10.2



 Comments   
Comment by Oleg Smirnov [ 2022-04-02 ]

The same patch as for MDEV-24560 is fixing this issue. The testcase from this task is added to bb-10.9-MDEV-24560.

Generated at Thu Feb 08 09:28:39 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.