[MDEV-7696] Server goes into an endless loop on 2nd execution of PS with ONLY_FULL_GROUP_BY, subqueries Created: 2015-03-10  Updated: 2015-12-02  Resolved: 2015-12-02

Status: Closed
Project: MariaDB Server
Component/s: Optimizer, Prepared Statements
Affects Version/s: 5.5, 10.0, 10.1
Fix Version/s: 5.5.45, 10.0.21

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Michael Widenius
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
Relates
relates to MDEV-7688 Server crashes in TABLE_LIST::is_view... Closed
relates to MDEV-7689 Server crashes in Item_subselect::rec... Closed
relates to MDEV-7690 Server crashes in Item_field::used_ta... Closed
relates to MDEV-7751 Server crashes in TABLE_LIST::reset_c... Closed
relates to MDEV-7688 Server crashes in TABLE_LIST::is_view... Closed
relates to MDEV-7689 Server crashes in Item_subselect::rec... Closed
relates to MDEV-7690 Server crashes in Item_field::used_ta... Closed
relates to MDEV-7751 Server crashes in TABLE_LIST::reset_c... Closed

 Description   

Attention: I am only getting the problem on a VALGRIND build
cmake . -DCMAKE_BUILD_TYPE=Debug -DWITH_VALGRIND=YES

See also MDEV-7688, MDEV-7689, MDEV-7690, MDEV-7751 - they all are somewhat similar, probably there are duplicates among them; but effects are different everywhere.

The problem appeared in 5.5 tree with the following revision:

revno: 4255
revision-id: sanja@askmonty.org-20140731071110-4uyuza0ykesfxe1b
parent: monty@mariadb.org-20140730102752-3ilbpvmbhr2r9a4x
committer: sanja@askmonty.org
branch nick: work-maria-5.5-MDEV-6441
timestamp: Thu 2014-07-31 10:11:10 +0300
message:
  MDEV-6441: memory leak
  
  mysql_derived_prepare() was executed on the statement memory.
  Now it is executed on the runtime memory.
  All bugs induced by this were fixed.

Test case

SET sql_mode = ONLY_FULL_GROUP_BY;
 
CREATE TABLE t1 (a INT) ENGINE=MyISAM;
INSERT INTO t1 VALUES (4),(6);
 
CREATE TABLE t2 (b INT) ENGINE=MyISAM;
INSERT INTO t2 VALUES (1),(8);
 
CREATE TABLE t3 (c INT) ENGINE=MyISAM;
INSERT INTO t3 VALUES (8),(3);
 
PREPARE stmt FROM "
SELECT *
FROM ( SELECT t2.* FROM t2, t3 ) AS from_sq
WHERE 0 IN ( 
  SELECT a FROM t1 
  WHERE a IN ( 
    SELECT t1.a FROM t1, t1 t1_1
    WHERE b = t1.a
  )
)
"; 
 
EXECUTE stmt;
EXECUTE stmt;

Execution stops on the 2nd EXECUTE, the process consumes a lot of CPU, process list shows

+----+------+-----------------+------+---------+------+-----------+------------------------------------------------------------------------------------------------------+----------+
| Id | User | Host            | db   | Command | Time | State     | Info                                                                                                 | Progress |
+----+------+-----------------+------+---------+------+-----------+------------------------------------------------------------------------------------------------------+----------+
|  1 | root | localhost       | test | Query   |  325 | preparing | SELECT *
FROM ( SELECT t2.* FROM t2, t3 ) AS from_sq
WHERE 0 IN ( 
  SELECT a FROM t1 
  WHERE a IN  |    0.000 |
|  2 | root | localhost:55329 | test | Query   |    0 | NULL      | show processlist                                                                                     |    0.000 |
+----+------+-----------------+------+---------+------+-----------+------------------------------------------------------------------------------------------------------+----------+
2 rows in set (0.00 sec)

Some stack traces from the running process (5.5 commit 34f37aa0c0aa87cfb6908500e937516ff37ea6f0):

Thread 2 (Thread 0x7fa34cfb6700 (LWP 20676)):
#0  0x0000000000683914 in compare_fields_by_table_order (field1=0x7fa34c446b98, field2=0x7fa34c445b58, table_join_idx=0x7fa34c4dd098) at sql/sql_select.cc:12225
#1  0x0000000000836e2d in bubble_sort<Item> (list_to_sort=0x7fa34c4c42d8, sort_func=0x68387a <compare_fields_by_table_order(Item*, Item*, void*)>, arg=0x7fa34c4dd098) at sql/sql_list.h:604
#2  0x0000000000833712 in Item_equal::sort (this=0x7fa34c4c4218, compare=0x68387a <compare_fields_by_table_order(Item*, Item*, void*)>, arg=0x7fa34c4dd098) at sql/item_cmpfunc.cc:5901
#3  0x000000000068468f in substitute_for_best_equal_field (context_tab=0x1, cond=0x7fa34c445c98, cond_equal=0x7fa34c445d88, table_join_idx=0x7fa34c4dd098) at sql/sql_select.cc:12635
#4  0x0000000000667141 in JOIN::optimize (this=0x7fa34c44d818) at sql/sql_select.cc:1317
#5  0x000000000066d551 in mysql_select (thd=0x7fa34d118000, rref_pointer_array=0x7fa34c4d6a88, tables=0x7fa34c44c018, wild_num=0, fields=..., conds=0x7fa34c445c98, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2416184064, result=0x7fa34c444098, unit=0x7fa34c4d60c8, select_lex=0x7fa34c4d67c8) at sql/sql_select.cc:3080
#6  0x0000000000663e63 in handle_select (thd=0x7fa34d118000, lex=0x7fa34c4d6018, result=0x7fa34c444098, setup_tables_done_option=0) at sql/sql_select.cc:319
#7  0x000000000063c293 in execute_sqlcom_select (thd=0x7fa34d118000, all_tables=0x7fa34c44c018) at sql/sql_parse.cc:4689
#8  0x000000000063546c in mysql_execute_command (thd=0x7fa34d118000) at sql/sql_parse.cc:2234
#9  0x0000000000656b1b in Prepared_statement::execute (this=0x7fa34c4cc280, expanded_query=0x7fa34cfb4c50, open_cursor=false) at sql/sql_prepare.cc:3928
#10 0x0000000000655c32 in Prepared_statement::execute_loop (this=0x7fa34c4cc280, expanded_query=0x7fa34cfb4c50, open_cursor=false, packet=0x0, packet_end=0x0) at sql/sql_prepare.cc:3587
#11 0x0000000000653d56 in mysql_sql_stmt_execute (thd=0x7fa34d118000) at sql/sql_prepare.cc:2737
#12 0x000000000063549a in mysql_execute_command (thd=0x7fa34d118000) at sql/sql_parse.cc:2244
#13 0x000000000063ed98 in mysql_parse (thd=0x7fa34d118000, rawbuf=0x7fa34c41c0d8 "EXECUTE stmt", length=12, parser_state=0x7fa34cfb5610) at sql/sql_parse.cc:5909
#14 0x00000000006329b3 in dispatch_command (command=COM_QUERY, thd=0x7fa34d118000, packet=0x7fa34d18c001 "EXECUTE stmt", packet_length=12) at sql/sql_parse.cc:1079
#15 0x0000000000631b3f in do_command (thd=0x7fa34d118000) at sql/sql_parse.cc:793
#16 0x0000000000739b14 in do_handle_one_connection (thd_arg=0x7fa34d118000) at sql/sql_connect.cc:1266
#17 0x00000000007395d3 in handle_one_connection (arg=0x7fa34d118000) at sql/sql_connect.cc:1181
#18 0x0000000000b836e3 in pfs_spawn_thread (arg=0x7fa3513ed760) at storage/perfschema/pfs.cc:1015
#19 0x00007fa3532bab50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#20 0x00007fa351bc870d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

Thread 2 (Thread 0x7fa34cfb6700 (LWP 20676)):
#0  0x00000000005a1b3a in base_list_iterator::rewind (this=0x7fa34cfb3940) at sql/sql_list.h:459
#1  0x00000000006a6bbe in List_iterator<Item>::rewind (this=0x7fa34cfb3940) at sql/sql_list.h:549
#2  0x0000000000836eb6 in bubble_sort<Item> (list_to_sort=0x7fa34c4c42d8, sort_func=0x68387a <compare_fields_by_table_order(Item*, Item*, void*)>, arg=0x7fa34c4dd098) at sql/sql_list.h:614
#3  0x0000000000833712 in Item_equal::sort (this=0x7fa34c4c4218, compare=0x68387a <compare_fields_by_table_order(Item*, Item*, void*)>, arg=0x7fa34c4dd098) at sql/item_cmpfunc.cc:5901
#4  0x000000000068468f in substitute_for_best_equal_field (context_tab=0x1, cond=0x7fa34c445c98, cond_equal=0x7fa34c445d88, table_join_idx=0x7fa34c4dd098) at sql/sql_select.cc:12635
#5  0x0000000000667141 in JOIN::optimize (this=0x7fa34c44d818) at sql/sql_select.cc:1317
#6  0x000000000066d551 in mysql_select (thd=0x7fa34d118000, rref_pointer_array=0x7fa34c4d6a88, tables=0x7fa34c44c018, wild_num=0, fields=..., conds=0x7fa34c445c98, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2416184064, result=0x7fa34c444098, unit=0x7fa34c4d60c8, select_lex=0x7fa34c4d67c8) at sql/sql_select.cc:3080
#7  0x0000000000663e63 in handle_select (thd=0x7fa34d118000, lex=0x7fa34c4d6018, result=0x7fa34c444098, setup_tables_done_option=0) at sql/sql_select.cc:319
#8  0x000000000063c293 in execute_sqlcom_select (thd=0x7fa34d118000, all_tables=0x7fa34c44c018) at sql/sql_parse.cc:4689
#9  0x000000000063546c in mysql_execute_command (thd=0x7fa34d118000) at sql/sql_parse.cc:2234
#10 0x0000000000656b1b in Prepared_statement::execute (this=0x7fa34c4cc280, expanded_query=0x7fa34cfb4c50, open_cursor=false) at sql/sql_prepare.cc:3928
#11 0x0000000000655c32 in Prepared_statement::execute_loop (this=0x7fa34c4cc280, expanded_query=0x7fa34cfb4c50, open_cursor=false, packet=0x0, packet_end=0x0) at sql/sql_prepare.cc:3587
#12 0x0000000000653d56 in mysql_sql_stmt_execute (thd=0x7fa34d118000) at sql/sql_prepare.cc:2737
#13 0x000000000063549a in mysql_execute_command (thd=0x7fa34d118000) at sql/sql_parse.cc:2244
#14 0x000000000063ed98 in mysql_parse (thd=0x7fa34d118000, rawbuf=0x7fa34c41c0d8 "EXECUTE stmt", length=12, parser_state=0x7fa34cfb5610) at sql/sql_parse.cc:5909
#15 0x00000000006329b3 in dispatch_command (command=COM_QUERY, thd=0x7fa34d118000, packet=0x7fa34d18c001 "EXECUTE stmt", packet_length=12) at sql/sql_parse.cc:1079
#16 0x0000000000631b3f in do_command (thd=0x7fa34d118000) at sql/sql_parse.cc:793
#17 0x0000000000739b14 in do_handle_one_connection (thd_arg=0x7fa34d118000) at sql/sql_connect.cc:1266
#18 0x00000000007395d3 in handle_one_connection (arg=0x7fa34d118000) at sql/sql_connect.cc:1181
#19 0x0000000000b836e3 in pfs_spawn_thread (arg=0x7fa3513ed760) at storage/perfschema/pfs.cc:1015
#20 0x00007fa3532bab50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#21 0x00007fa351bc870d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

Thread 2 (Thread 0x7fa34cfb6700 (LWP 20676)):
#0  0x000000000058eace in base_list_iterator::next_fast (this=0x7fa34cfb3a10) at sql/sql_list.h:453
#1  0x00000000006a7373 in List_iterator_fast<Item_equal>::operator++ (this=0x7fa34cfb3a10) at sql/sql_list.h:569
#2  0x00000000006846a0 in substitute_for_best_equal_field (context_tab=0x1, cond=0x7fa34c445c98, cond_equal=0x7fa34c445d88, table_join_idx=0x7fa34c4dd098) at sql/sql_select.cc:12633
#3  0x0000000000667141 in JOIN::optimize (this=0x7fa34c44d818) at sql/sql_select.cc:1317
#4  0x000000000066d551 in mysql_select (thd=0x7fa34d118000, rref_pointer_array=0x7fa34c4d6a88, tables=0x7fa34c44c018, wild_num=0, fields=..., conds=0x7fa34c445c98, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2416184064, result=0x7fa34c444098, unit=0x7fa34c4d60c8, select_lex=0x7fa34c4d67c8) at sql/sql_select.cc:3080
#5  0x0000000000663e63 in handle_select (thd=0x7fa34d118000, lex=0x7fa34c4d6018, result=0x7fa34c444098, setup_tables_done_option=0) at sql/sql_select.cc:319
#6  0x000000000063c293 in execute_sqlcom_select (thd=0x7fa34d118000, all_tables=0x7fa34c44c018) at sql/sql_parse.cc:4689
#7  0x000000000063546c in mysql_execute_command (thd=0x7fa34d118000) at sql/sql_parse.cc:2234
#8  0x0000000000656b1b in Prepared_statement::execute (this=0x7fa34c4cc280, expanded_query=0x7fa34cfb4c50, open_cursor=false) at sql/sql_prepare.cc:3928
#9  0x0000000000655c32 in Prepared_statement::execute_loop (this=0x7fa34c4cc280, expanded_query=0x7fa34cfb4c50, open_cursor=false, packet=0x0, packet_end=0x0) at sql/sql_prepare.cc:3587
#10 0x0000000000653d56 in mysql_sql_stmt_execute (thd=0x7fa34d118000) at sql/sql_prepare.cc:2737
#11 0x000000000063549a in mysql_execute_command (thd=0x7fa34d118000) at sql/sql_parse.cc:2244
#12 0x000000000063ed98 in mysql_parse (thd=0x7fa34d118000, rawbuf=0x7fa34c41c0d8 "EXECUTE stmt", length=12, parser_state=0x7fa34cfb5610) at sql/sql_parse.cc:5909
#13 0x00000000006329b3 in dispatch_command (command=COM_QUERY, thd=0x7fa34d118000, packet=0x7fa34d18c001 "EXECUTE stmt", packet_length=12) at sql/sql_parse.cc:1079
#14 0x0000000000631b3f in do_command (thd=0x7fa34d118000) at sql/sql_parse.cc:793
#15 0x0000000000739b14 in do_handle_one_connection (thd_arg=0x7fa34d118000) at sql/sql_connect.cc:1266
#16 0x00000000007395d3 in handle_one_connection (arg=0x7fa34d118000) at sql/sql_connect.cc:1181
#17 0x0000000000b836e3 in pfs_spawn_thread (arg=0x7fa3513ed760) at storage/perfschema/pfs.cc:1015
#18 0x00007fa3532bab50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#19 0x00007fa351bc870d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112



 Comments   
Comment by Elena Stepanova [ 2015-12-02 ]

Fixed in 5.5.45 / 10.0.21 by the following commit:

commit 2e941fe9fce7f1667993916ff3f238a283286d3f
Author: Monty <monty@mariadb.org>
Date:   Thu Jun 25 23:18:48 2015 +0300
 
    Fixed crashing bug when using ONLY_FULL_GROUP_BY in a stored procedure/trigger that is repeatedly executed.
    This is MDEV-7601, including it's sub tasks MDEV-7594, MDEV-7555, MDEV-7590, MDEV-7581, MDEV-7589
    
    The problem was that select_lex->non_agg_fields was not properly reset for re-execution and this caused an overwrite of a random memory position.
    The fix was move non_agg_fields from select_lext to JOIN, which is properly reset.

Generated at Thu Feb 08 07:21:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.