[MDEV-6441] memory leak Created: 2014-07-13  Updated: 2014-08-05  Resolved: 2014-07-31

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 5.5.38, 10.0.12
Fix Version/s: 5.5.39, 10.0.13, 10.1.1

Type: Bug Priority: Critical
Reporter: DJV-COM Assignee: Oleksandr Byelkin
Resolution: Fixed Votes: 0
Labels: None

Attachments: Text File bugTest.txt    

 Description   

memory leak



 Comments   
Comment by Elena Stepanova [ 2014-07-14 ]

Thanks for the report and test case!

This crude MTR test demonstrates the problem (not to be included into the regression test suite):

 
 
--echo # Memory at the beginning of the test
 
exec top -b -n 1 | grep mysqld;
 
 
 
--disable_query_log
 
--disable_result_log
 
 
 
 
 
CREATE TABLE `tt00` (
 
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
 
  `f0` int(11) unsigned NOT NULL DEFAULT '0',
 
  `f1` int(11) unsigned NOT NULL DEFAULT '0',
 
  PRIMARY KEY (`id`),
 
  UNIQUE KEY `id` (`id`)
 
);
 
 
 
CREATE TABLE `tt01` (
 
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
 
  `f02` bigint(20) unsigned NOT NULL DEFAULT '0',
 
  `f03` int(11) unsigned NOT NULL DEFAULT '0',
 
  PRIMARY KEY (`id`),
 
  UNIQUE KEY `id` (`id`)
 
);
 
 
 
CREATE ALGORITHM=UNDEFINED SQL SECURITY DEFINER VIEW `tt0000` AS 
  SELECT 
    `tt00`.`f0` AS `f0`,
 
    `tt00`.`f1` AS `f1`,
 
    `tt01`.`f02` AS `f02`,
 
    `tt01`.`f03` AS `f03` 
  FROM 
    (`tt00` LEFT JOIN `tt01` ON((`tt00`.`id` = `tt01`.`f02`)));
 
 
 
--delimiter |
 
CREATE FUNCTION `BugTest`(
 
        p0 BIGINT(20) UNSIGNED
 
    )
 
    RETURNS bigint(20) unsigned
 
    DETERMINISTIC
 
    CONTAINS SQL
 
    SQL SECURITY DEFINER
 
    COMMENT ''
 
BEGIN
 
 
 
DECLARE k0 INTEGER UNSIGNED DEFAULT 0;
 
DECLARE lResult INTEGER UNSIGNED DEFAULT 0;
 
 
 
 SET k0 = 0;
 
 WHILE k0 < 1000 DO
 
    SELECT COUNT(*) as `f00` INTO lResult  FROM `tt0000` WHERE `tt0000`.`f0` = p0; -- BUG
 
   SET k0 = k0 + 1;
 
 END WHILE;
 
 
 
  RETURN(k0);
 
END|
 
--delimiter ;
 
 
 
--connect (con1,localhost,root,,)
 
 
 
--let $i = 1
 
--let $count = 1000
 
 
 
while($count)
 
{
 
 eval SELECT `BugTest`($i);
 
 --inc $i
 
 --dec $count
 
}
 
 
 
--enable_query_log
 
--enable_result_log
 
 
 
--echo # Memory after running the flow, before disconnecting
 
exec top -b -n 1 | grep mysqld;
 
 
 
--disconnect con1
 
 
 
--connection default
 
 
 
--echo # Memory after disconnecting the connection
 
exec top -b -n 1 | grep mysqld;
 
 
 
DROP VIEW tt0000;
 
DROP TABLE tt00, tt01;
 
DROP FUNCTION BugTest;
 
 
 
FLUSH TABLES;
 
FLUSH STATUS;
 
 
 
--enable_query_log
 
--enable_result_log
 
 
 
--echo # Memory after dropping/flushing 
exec top -b -n 1 | grep mysqld;

Here is the output on current 5.5:

# Memory at the beginning of the test
 
32306 elenst    20   0  124m  67m 7908 S   0.0  0.8   0:00.24 mysqld
 
# Memory after running the flow, before disconnecting
 
32306 elenst    20   0 2968m 2.0g 8284 S   0.0 26.1   8:03.09 mysqld
 
# Memory after disconnecting the connection
 
32306 elenst    20   0 2968m 1.3g 8284 S  91.3 16.5   8:03.24 mysqld
 
...
 
# Memory after dropping/flushing 
32306 elenst    20   0 2968m 551m 8284 S 104.3  6.9   8:03.44 mysqld

Compare with MySQL 5.6, for example:

# Memory at the beginning of the test
 
32459 elenst    20   0  176m  51m 7180 S   0.0  0.6   0:00.27 mysqld
 
# Memory after running the flow, before disconnecting
 
32459 elenst    20   0  176m  51m 7180 S   0.0  0.6   4:15.52 mysqld
 
# Memory after disconnecting the connection
 
32459 elenst    20   0  176m  51m 7288 S   0.0  0.6   4:15.52 mysqld
 
...
 
# Memory after dropping/flushing 
32459 elenst    20   0  176m  51m 7500 S   0.0  0.6   4:15.52 mysqld

Comment by Oleksandr Byelkin [ 2014-07-17 ]

As I suspected it is creating a temporary table problem.

4  0x0000000000ce56b6 in alloc_root (mem_root=0x7ffff1828838, length=520) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/mysys/my_alloc.c:159
 
(gdb) frame 5
 
#5  0x000000000067691a in Sql_alloc::operator new[] (size=520, mem_root=0x7ffff1828838) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_list.h:41
 
(gdb) frame 6
 
#6  0x000000000065e69b in create_tmp_table (thd=0x7ffff2262000, param=0x7ffff1b81a78, fields=..., group=0x0, distinct=false, save_sum_fields=true, select_options=2416188160, rows_limit=18446744073709551615, table_alias=0x7ffff181e5b8 "tt0000", do_not_open=true, keep_row_order=false) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:14919
 
(gdb) p thd->query()
 
warning: can't find linker symbol for virtual table for `THD' value
 
warning: can't find linker symbol for virtual table for `THD' value
 
warning: can't find linker symbol for virtual table for `THD' value
 
warning: can't find linker symbol for virtual table for `Statement' value
 
warning: can't find linker symbol for virtual table for `Statement' value
 
$1 = 0x7ffff1892b18 "SELECT COUNT(*) as `f00` INTO lResult  FROM `tt0000` WHERE `tt0000`.`f0` =  NAME_CONST('p0',1)"
 
(gdb) frame 7
 
#7  0x00000000006bf4ae in select_union::create_result_table (this=0x7ffff1b81a58, thd_arg=0x7ffff2262000, column_types=0x7ffff1963740, is_union_distinct=false, options=2416188160, alias=0x7ffff181e5b8 "tt0000", bit_fields_as_long=false, create_table=false, keep_row_order=false) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_union.cc:158
 
(gdb) where
 
#0  0x00007ffff6bccf79 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
 
#1  0x00007ffff6bd0388 in __GI_abort () at abort.c:89
 
#2  0x00007ffff6bc5e36 in __assert_fail_base (fmt=0x7ffff6d17718 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0xef2410 "mem_root->memused < 2*1024*1024", file=file@entry=0xef2380 "/home/bell/maria/bzr/work-maria-5.5-MDEV-6441/mysys/my_alloc.c", line=line@entry=159, function=function@entry=0xef24c4 <__PRETTY_FUNCTION__.9956> "alloc_root") at assert.c:92
 
#3  0x00007ffff6bc5ee2 in __GI___assert_fail (assertion=0xef2410 "mem_root->memused < 2*1024*1024", file=0xef2380 "/home/bell/maria/bzr/work-maria-5.5-MDEV-6441/mysys/my_alloc.c", line=159, function=0xef24c4 <__PRETTY_FUNCTION__.9956> "alloc_root") at assert.c:101
 
#4  0x0000000000ce56b6 in alloc_root (mem_root=0x7ffff1828838, length=520) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/mysys/my_alloc.c:159
 
#5  0x000000000067691a in Sql_alloc::operator new[] (size=520, mem_root=0x7ffff1828838) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_list.h:41
 
#6  0x000000000065e69b in create_tmp_table (thd=0x7ffff2262000, param=0x7ffff1b81a78, fields=..., group=0x0, distinct=false, save_sum_fields=true, select_options=2416188160, rows_limit=18446744073709551615, table_alias=0x7ffff181e5b8 "tt0000", do_not_open=true, keep_row_order=false) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:14919
 
#7  0x00000000006bf4ae in select_union::create_result_table (this=0x7ffff1b81a58, thd_arg=0x7ffff2262000, column_types=0x7ffff1963740, is_union_distinct=false, options=2416188160, alias=0x7ffff181e5b8 "tt0000", bit_fields_as_long=false, create_table=false, keep_row_order=false) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_union.cc:158
 
#8  0x00000000005e0635 in mysql_derived_prepare (thd=0x7ffff2262000, lex=0x7ffff1904018, derived=0x7ffff18e9418) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_derived.cc:681
 
#9  0x00000000005df9a9 in mysql_handle_single_derived (lex=0x7ffff1904018, derived=0x7ffff18e9418, phases=2) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_derived.cc:192
 
#10 0x00000000006e25c6 in TABLE_LIST::handle_derived (this=0x7ffff18e9418, lex=0x7ffff1904018, phases=2) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/table.cc:6566
 
#11 0x00000000005fc848 in st_select_lex::handle_derived (this=0x7ffff19047c0, lex=0x7ffff1904018, phases=2) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_lex.cc:3553
 
#12 0x0000000000637738 in JOIN::prepare (this=0x7ffff18ebe18, rref_pointer_array=0x7ffff1904a80, tables_init=0x7ffff18e9418, wild_num=0, conds_init=0x7ffff1845558, og_num=0, order_init=0x0, skip_order_by=false, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0x7ffff19047c0, unit_arg=0x7ffff19040c8) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:636
 
#13 0x000000000064090b in mysql_select (thd=0x7ffff2262000, rref_pointer_array=0x7ffff1904a80, tables=0x7ffff18e9418, wild_num=0, fields=..., conds=0x7ffff1845558, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7ffff1814218, unit=0x7ffff19040c8, select_lex=0x7ffff19047c0) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:3070
 
#14 0x0000000000636e29 in handle_select (thd=0x7ffff2262000, lex=0x7ffff1904018, result=0x7ffff1814218, setup_tables_done_option=0) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:319
 
#15 0x000000000060e4e4 in execute_sqlcom_select (thd=0x7ffff2262000, all_tables=0x7ffff18e9418) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:4688
 
#16 0x0000000000607098 in mysql_execute_command (thd=0x7ffff2262000) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:2233
 
#17 0x000000000090f27c in sp_instr_stmt::exec_core (this=0x7ffff181c3f8, thd=0x7ffff2262000, nextp=0x7ffff7f82860) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sp_head.cc:3220
 
#18 0x000000000090ea5f in sp_lex_keeper::reset_lex_and_exec_core (this=0x7ffff181c440, thd=0x7ffff2262000, nextp=0x7ffff7f82860, open_tables=false, instr=0x7ffff181c3f8) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sp_head.cc:2998
 
#19 0x000000000090efde in sp_instr_stmt::execute (this=0x7ffff181c3f8, thd=0x7ffff2262000, nextp=0x7ffff7f82860) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sp_head.cc:3144
 
#20 0x000000000090ab24 in sp_head::execute (this=0x7ffff1828818, thd=0x7ffff2262000, merge_da_on_success=true) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sp_head.cc:1431
 
#21 0x000000000090be80 in sp_head::execute_function (this=0x7ffff1828818, thd=0x7ffff2262000, argp=0x7ffff187a468, argcount=1, return_value_fld=0x7ffff1821a58) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sp_head.cc:1953
 
#22 0x0000000000845345 in Item_func_sp::execute_impl (this=0x7ffff187a3d8, thd=0x7ffff2262000) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/item_func.cc:6872
 
#23 0x00000000008450d6 in Item_func_sp::execute (this=0x7ffff187a3d8) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/item_func.cc:6801
 
#24 0x0000000000847e8a in Item_func_sp::val_int (this=0x7ffff187a3d8) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/item_func.h:2007
 
#25 0x00000000007f250c in Item::send (this=0x7ffff187a3d8, protocol=0x7ffff22625c8, buffer=0x7ffff7f83080) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/item.cc:6502
 
#26 0x0000000000564854 in Protocol::send_result_set_row (this=0x7ffff22625c8, row_items=0x7ffff2265b48) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/protocol.cc:903
 
#27 0x00000000005cd092 in select_send::send_data (this=0x7ffff1844518, items=...) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_class.cc:2370
 
#28 0x000000000063da24 in JOIN::exec (this=0x7ffff18eac18) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:2278
 
#29 0x00000000006409c1 in mysql_select (thd=0x7ffff2262000, rref_pointer_array=0x7ffff2265ce8, tables=0x0, wild_num=0, fields=..., conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147748608, result=0x7ffff1844518, unit=0x7ffff2265330, select_lex=0x7ffff2265a28) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:3090
 
#30 0x0000000000636e29 in handle_select (thd=0x7ffff2262000, lex=0x7ffff2265280, result=0x7ffff1844518, setup_tables_done_option=0) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_select.cc:319
 
#31 0x000000000060e4e4 in execute_sqlcom_select (thd=0x7ffff2262000, all_tables=0x0) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:4688
 
#32 0x0000000000607098 in mysql_execute_command (thd=0x7ffff2262000) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:2233
 
#33 0x0000000000610cb5 in mysql_parse (thd=0x7ffff2262000, rawbuf=0x7ffff181c038 "SELECT `BugTest`(1)", length=19, parser_state=0x7ffff7f845f0) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:5799
 
#34 0x0000000000604552 in dispatch_command (command=COM_QUERY, thd=0x7ffff2262000, packet=0x7ffff2358001 "", packet_length=19) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:1079
 
#35 0x00000000006036a2 in do_command (thd=0x7ffff2262000) at /home/bell/maria/bzr/work-maria-5.5-MDEV-6441/sql/sql_parse.cc:793

Comment by Oleksandr Byelkin [ 2014-07-17 ]

It looks like whole mysql_derived_prepare should be run on execution arena but it is not.

Comment by Oleksandr Byelkin [ 2014-07-22 ]

The problem is that mysql_derived_prepare() set statemnt arena at the beginning.
Probably some part of it need the arena but not all.

Comment by Oleksandr Byelkin [ 2014-07-24 ]

Above (executing mysql_derived_prepare() on stetement memory which lead to memory leak) is definetly a hack to hide real problems.
1. double saveing and restoring TABLE_LIST::prep_on_expr
2. TABLE_LIST::prep_on_expr planed as persistent but in same cases used to temporary store values then roll it back
3. there is problem of TABLE_LIST::on_expr pointing freed memory

Comment by Oleksandr Byelkin [ 2014-07-28 ]

Could you review it, please?

Generated at Thu Feb 08 07:11:55 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.