[MDEV-6736] [10.0] Valgrind warnings 'Invalid read' in subselect_engine::calc_const_tables with SQ in WHERE and HAVING, ORDER BY, materialization+semijoin Created: 2014-09-12  Updated: 2018-11-05  Resolved: 2018-03-17

Status: Closed
Project: MariaDB Server
Component/s: Optimizer
Affects Version/s: 10.0.13, 10.0
Fix Version/s: 10.0.35

Type: Bug Priority: Minor
Reporter: Elena Stepanova Assignee: Varun Gupta (Inactive)
Resolution: Fixed Votes: 1
Labels: optimizer, valgrind

Issue Links:
Relates
relates to MDEV-17616 ASAN heap-use-after-free in subselec... Closed
Sprint: 10.1.32

 Description    

CREATE TABLE t1 (a INT) ENGINE=MyISAM;
 
INSERT INTO t1 VALUES (3),(8);
 
 
 
CREATE TABLE t2 (b INT) ENGINE=MyISAM;
 
INSERT INTO t2 VALUES (2),(1);
 
 
 
SELECT a FROM t1
 
WHERE 9 IN ( SELECT MIN( a ) FROM t1 )  
HAVING a <> ( SELECT COUNT(*) FROM t2 ) 
ORDER BY a;


==30848== Invalid read of size 1
 
==30848==    at 0x90E68F: subselect_engine::calc_const_tables(List<TABLE_LIST>&) (item_subselect.cc:4166)
 
==30848==    by 0x90E708: subselect_single_select_engine::upper_select_const_tables() (item_subselect.cc:4175)
 
==30848==    by 0x904F8B: Item_subselect::update_used_tables() (item_subselect.cc:888)
 
==30848==    by 0x8C917E: Item_func::update_used_tables() (item_func.cc:445)
 
==30848==    by 0x6B6F0A: JOIN::exec_inner() (sql_select.cc:2927)
 
==30848==    by 0x6B4CDB: JOIN::exec() (sql_select.cc:2369)
 
==30848==    by 0x6B8046: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3307)
 
==30848==    by 0x6AE640: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:372)
 
==30848==    by 0x683345: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:5269)
 
==30848==    by 0x67B646: mysql_execute_command(THD*) (sql_parse.cc:2552)
 
==30848==    by 0x685AD0: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6415)
 
==30848==    by 0x6788E7: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1307)
 
==30848==    by 0x677C8C: do_command(THD*) (sql_parse.cc:1004)
 
==30848==    by 0x7941C9: do_handle_one_connection(THD*) (sql_connect.cc:1379)
 
==30848==    by 0x793F1C: handle_one_connection (sql_connect.cc:1293)
 
==30848==    by 0xCC9D29: pfs_spawn_thread (pfs.cc:1860)
 
==30848==  Address 0xdbe21dd is 2,269 bytes inside a block of size 4,772 free'd
 
==30848==    at 0x4C27BF4: free (vg_replace_malloc.c:469)
 
==30848==    by 0xE52CEE: free_memory (safemalloc.c:276)
 
==30848==    by 0xE529B2: sf_free (safemalloc.c:194)
 
==30848==    by 0xE41D2D: my_free (my_malloc.c:216)
 
==30848==    by 0xE362D9: free_root (my_alloc.c:385)
 
==30848==    by 0x6D96E1: free_tmp_table(THD*, TABLE*) (sql_select.cc:17081)
 
==30848==    by 0x91013E: subselect_hash_sj_engine::cleanup() (item_subselect.cc:5002)
 
==30848==    by 0x9034AB: Item_subselect::cleanup() (item_subselect.cc:152)
 
==30848==    by 0x90364A: Item_in_subselect::cleanup() (item_subselect.cc:189)
 
==30848==    by 0x6CB94E: st_join_table::cleanup() (sql_select.cc:11224)
 
==30848==    by 0x6CC748: JOIN::cleanup(bool) (sql_select.cc:11620)
 
==30848==    by 0x6CC35B: JOIN::join_free() (sql_select.cc:11507)
 
==30848==    by 0x6B68B7: JOIN::exec_inner() (sql_select.cc:2844)
 
==30848==    by 0x6B4CDB: JOIN::exec() (sql_select.cc:2369)
 
==30848==    by 0x6B8046: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3307)
 
==30848==    by 0x6AE640: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:372)
 
==30848== Invalid read of size 8
 
==30848==    at 0x90E69E: subselect_engine::calc_const_tables(List<TABLE_LIST>&) (item_subselect.cc:4167)
 
==30848==    by 0x90E708: subselect_single_select_engine::upper_select_const_tables() (item_subselect.cc:4175)
 
==30848==    by 0x904F8B: Item_subselect::update_used_tables() (item_subselect.cc:888)
 
==30848==    by 0x8C917E: Item_func::update_used_tables() (item_func.cc:445)
 
==30848==    by 0x6B6F0A: JOIN::exec_inner() (sql_select.cc:2927)
 
==30848==    by 0x6B4CDB: JOIN::exec() (sql_select.cc:2369)
 
==30848==    by 0x6B8046: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3307)
 
==30848==    by 0x6AE640: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:372)
 
==30848==    by 0x683345: execute_sqlcom_select(THD*, TABLE_LIST*) (sql_parse.cc:5269)
 
==30848==    by 0x67B646: mysql_execute_command(THD*) (sql_parse.cc:2552)
 
==30848==    by 0x685AD0: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6415)
 
==30848==    by 0x6788E7: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1307)
 
==30848==    by 0x677C8C: do_command(THD*) (sql_parse.cc:1004)
 
==30848==    by 0x7941C9: do_handle_one_connection(THD*) (sql_connect.cc:1379)
 
==30848==    by 0x793F1C: handle_one_connection (sql_connect.cc:1293)
 
==30848==    by 0xCC9D29: pfs_spawn_thread (pfs.cc:1860)
 
==30848==  Address 0xdbe21a0 is 2,208 bytes inside a block of size 4,772 free'd
 
==30848==    at 0x4C27BF4: free (vg_replace_malloc.c:469)
 
==30848==    by 0xE52CEE: free_memory (safemalloc.c:276)
 
==30848==    by 0xE529B2: sf_free (safemalloc.c:194)
 
==30848==    by 0xE41D2D: my_free (my_malloc.c:216)
 
==30848==    by 0xE362D9: free_root (my_alloc.c:385)
 
==30848==    by 0x6D96E1: free_tmp_table(THD*, TABLE*) (sql_select.cc:17081)
 
==30848==    by 0x91013E: subselect_hash_sj_engine::cleanup() (item_subselect.cc:5002)
 
==30848==    by 0x9034AB: Item_subselect::cleanup() (item_subselect.cc:152)
 
==30848==    by 0x90364A: Item_in_subselect::cleanup() (item_subselect.cc:189)
 
==30848==    by 0x6CB94E: st_join_table::cleanup() (sql_select.cc:11224)
 
==30848==    by 0x6CC748: JOIN::cleanup(bool) (sql_select.cc:11620)
 
==30848==    by 0x6CC35B: JOIN::join_free() (sql_select.cc:11507)
 
==30848==    by 0x6B68B7: JOIN::exec_inner() (sql_select.cc:2844)
 
==30848==    by 0x6B4CDB: JOIN::exec() (sql_select.cc:2369)
 
==30848==    by 0x6B8046: mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:3307)
 
==30848==    by 0x6AE640: handle_select(THD*, LEX*, select_result*, unsigned long) (sql_select.cc:372)

Stack trace from:

revision-id: elenst@montyprogram.com-20140911232154-8kfkykep1qtsn8fi
 
revno: 4393
 
branch-nick: 10.0

EXPLAIN:

id	select_type	table	type	possible_keys	key	key_len	ref	rows	filtered	Extra
 
1	PRIMARY	<subquery2>	const	distinct_key	distinct_key	4	const	1	100.00	Using temporary; Using filesort
 
1	PRIMARY	t1	ALL	NULL	NULL	NULL	NULL	2	100.00	Using join buffer (flat, BNL join)
 
3	SUBQUERY	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	Select tables optimized away
 
2	MATERIALIZED	t1	ALL	NULL	NULL	NULL	NULL	2	100.00	
Warnings:
 
Note	1003	select `test`.`t1`.`a` AS `a` from  <materialize> (select min(`test`.`t1`.`a`) from `test`.`t1`) join `test`.`t1` where (`<subquery2>`.`MIN( a )` = 9) having (`test`.`t1`.`a` <> (select count(0) from `test`.`t2`)) order by `test`.`t1`.`a`

Minimal optimizer_switch: materialization=on,semijoin=on

 Comments   
Comment by Elena Stepanova [ 2018-01-19 ]

Still reproducible on 10.0, but not on 10.1/10.2.

10.0 4b8cd4536a2c ASAN

 
==25219==ERROR: AddressSanitizer: heap-use-after-free on address 0x6210000671dd at pc 0xd3eb47 bp 0x7f2d15797c60 sp 0x7f2d15797c58
 
READ of size 1 at 0x6210000671dd thread T5
 
    #0 0xd3eb46 in subselect_engine::calc_const_tables(List<TABLE_LIST>&) /data/src/10.0/sql/item_subselect.cc:4242
 
    #1 0xd3ec22 in subselect_single_select_engine::upper_select_const_tables() /data/src/10.0/sql/item_subselect.cc:4251
 
    #2 0xd2009a in Item_subselect::update_used_tables() /data/src/10.0/sql/item_subselect.cc:902
 
    #3 0xc71a4d in Item_func::update_used_tables() /data/src/10.0/sql/item_func.cc:446
 
    #4 0x7dad09 in JOIN::exec_inner() /data/src/10.0/sql/sql_select.cc:2937
 
    #5 0x7dd343 in JOIN::exec() /data/src/10.0/sql/sql_select.cc:2379
 
    #6 0x7d24a6 in mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) /data/src/10.0/sql/sql_select.cc:3318
 
    #7 0x7d2ad6 in handle_select(THD*, LEX*, select_result*, unsigned long) /data/src/10.0/sql/sql_select.cc:373
 
    #8 0x6b566c in execute_sqlcom_select /data/src/10.0/sql/sql_parse.cc:5293
 
    #9 0x6ccea8 in mysql_execute_command(THD*) /data/src/10.0/sql/sql_parse.cc:2553
 
    #10 0x6e19f7 in mysql_parse(THD*, char*, unsigned int, Parser_state*) /data/src/10.0/sql/sql_parse.cc:6569
 
    #11 0x6e5616 in dispatch_command(enum_server_command, THD*, char*, unsigned int) /data/src/10.0/sql/sql_parse.cc:1296
 
    #12 0x6e9f0f in do_command(THD*) /data/src/10.0/sql/sql_parse.cc:999
 
    #13 0x975c3b in do_handle_one_connection(THD*) /data/src/10.0/sql/sql_connect.cc:1377
 
    #14 0x975ea2 in handle_one_connection /data/src/10.0/sql/sql_connect.cc:1292
 
    #15 0x11a95d4 in pfs_spawn_thread /data/src/10.0/storage/perfschema/pfs.cc:1861
 
    #16 0x7f2d1fdb2493 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7493)
 
    #17 0x7f2d1e16b93e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe893e)
 
 
 
0x6210000671dd is located 2269 bytes inside of 4772-byte region [0x621000066900,0x621000067ba4)
 
freed by thread T5 here:
 
    #0 0x7f2d2001c527 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x54527)
 
    #1 0x18e68b3 in free_memory /data/src/10.0/mysys/safemalloc.c:276
 
 
 
previously allocated by thread T5 here:
 
    #0 0x7f2d2001c73f in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x5473f)
 
    #1 0x18e6986 in sf_malloc /data/src/10.0/mysys/safemalloc.c:115
 
    #2 0x19c964a (/data/bld/10.0-asan/bin/mysqld+0x19c964a)
 
 
 
Thread T5 created by T0 here:
 
    #0 0x7f2d1ffebbba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
 
    #1 0x11b4e2b in spawn_thread_v1 /data/src/10.0/storage/perfschema/pfs.cc:1911
 
 
 
SUMMARY: AddressSanitizer: heap-use-after-free /data/src/10.0/sql/item_subselect.cc:4242 subselect_engine::calc_const_tables(List<TABLE_LIST>&)
 
Shadow bytes around the buggy address:
 
  0x0c4280004de0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004df0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
=>0x0c4280004e30: fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd fd
 
  0x0c4280004e40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
  0x0c4280004e80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
 
Shadow byte legend (one shadow byte represents 8 application bytes):
 
  Addressable:           00
 
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
 
  Heap right redzone:      fb
 
  Freed heap region:       fd
 
  Stack left redzone:      f1
 
  Stack mid redzone:       f2
 
  Stack right redzone:     f3
 
  Stack partial redzone:   f4
 
  Stack after return:      f5
 
  Stack use after scope:   f8
 
  Global redzone:          f9
 
  Global init order:       f6
 
  Poisoned by user:        f7
 
  Contiguous container OOB:fc
 
  ASan internal:           fe
 
==25219==ABORTING

Comment by Varun Gupta (Inactive) [ 2018-03-12 ] 

explain extended 
SELECT a FROM t1
 
WHERE 9 IN ( SELECT MIN( a ) FROM t1 )  
HAVING a <> ( SELECT COUNT(*) FROM t2 ) 
ORDER BY a;
 
id	select_type	table	type	possible_keys	key	key_len	ref	rows	filtered	Extra
 
1	PRIMARY	<subquery2>	const	distinct_key	distinct_key	4	const	1	100.00	Using temporary; Using filesort
 
1	PRIMARY	t1	ALL	NULL	NULL	NULL	NULL	2	100.00	Using join buffer (flat, BNL join)
 
3	SUBQUERY	NULL	NULL	NULL	NULL	NULL	NULL	NULL	NULL	Select tables optimized away
 
2	MATERIALIZED	t1	ALL	NULL	NULL	NULL	NULL	2	100.00	
Warnings:
 
Note	1003	select `test`.`t1`.`a` AS `a` from  <materialize> (select min(`test`.`t1`.`a`) from `test`.`t1`) join `test`.`t1` where (`<subquery2>`.`MIN( a )` = 9) having (`test`.`t1`.`a` <> (select count(0) from `test`.`t2`)) order by `test`.`t1`.`a`

Comment by Varun Gupta (Inactive) [ 2018-03-13 ]

In the function JOIN::exec_inner , I see

    /*
 
      curr_join->join_free() will call JOIN::cleanup(full=TRUE). It will not
 
      be safe to call update_used_tables() after that.
 
    */
 
    if (curr_join->tmp_having)
 
        curr_join->tmp_having->update_used_tables();

But later in this function we have

 
 
            table_map used_tables= (curr_join->const_table_map |
 
                                    curr_table->table->map);
 
            curr_join->tmp_having->update_used_tables();

So this is where we kind of start accessing tables(temporary table) which were freed with the call to JOIN::join_free()

Comment by Sergei Petrunia [ 2018-03-15 ]

The patch is here: http://lists.askmonty.org/pipermail/commits/2018-March/012084.html (it has an incorrect comment, it says MDEV-6731 instead)

Comment by Sergei Petrunia [ 2018-03-16 ]

Looking at the patch...

table_map subselect_engine::calc_const_tables(List<TABLE_LIST> &list)
 
{
 
  table_map map= 0;
 
  List_iterator<TABLE_LIST> ti(list);
 
  TABLE_LIST *table;
 
  //for (; table; table= table->next_leaf)
 
  while ((table= ti++))
 
  {
 
    TABLE *tbl= table->table;
 
    if (tbl && tbl->const_table)   //(*)
 
      map|= tbl->map;

and this is invoked from

table_map subselect_single_select_engine::upper_select_const_tables()

For JTBM materialization nests, we have tbl==NULL and skip them in the computation.

There is no much point in computing a bitmap of const_tables in the upper select after JOIN::cleanup has been called for the upper select.

OTOH, this is what 10.1 does, and it seems to be harmless.

Comment by Sergei Petrunia [ 2018-03-16 ]

So, let's have 10.0 do what 10.1 does, here.

Generated at Thu Feb 08 07:14:12 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.