[MDEV-31127] Possible memory leak on running insert in PS mode against a table with a trigger that fires creation of a new partiton - Jira

XML

Word

Printable

Details

Type: Bug
Status: Open (View Workflow)
Priority: Minor
Resolution: Unresolved
Affects Version/s: 11.0(EOL)
Fix Version/s: 11.0(EOL)
Component/s: Prepared Statements, Versioned Tables
Labels:
None

Description

Consider the following test case

-- source include/have_partition.inc

-- source suite/versioning/common.inc

-- source suite/versioning/engines.inc

-- source include/have_sequence.inc

set timestamp= unix_timestamp('2000-01-01 00:00:00');

create or replace table t1 (x int) with system versioning

partition by system_time interval 1 hour auto

partitions 3;

create table t2 (x int);

create trigger tr after insert on t2 for each row update t1 set x= x + 11;

prepare ps2 from 'insert into t2 values (7)';

set timestamp= unix_timestamp('2000-01-01 06:00:00');

Execute ps2;

eXecute ps2;

set timestamp= unix_timestamp('2000-01-01 15:00:00');

exEcute ps2; # <=== Memory leak happens here

exeCute ps2;

# Cleanup

drop tables t1, t2;

deallocate prepare ps2

On running PREPARE ps2 a new entry for the trigger 'tr' is added into the hash table of used routines (this hash table is contained in lex).

  * frame #0: 0x00000001003c0a12 mariadbd`sp_add_used_routine(prelocking_ctx=0x00000001182b80b0, arena=0x000000010796d488, key=0x0000700001e7a228, handler=0x0000000101991770, belong_to_view=0x0000000000000000) at sp.cc:2335:30

    frame #1: 0x000000010068ec55 mariadbd`Table_triggers_list::add_tables_and_routines_for_triggers(this=0x000000010480c2a0, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0) at sql_trigger.cc:2540:15

    frame #2: 0x0000000100448722 mariadbd`DML_prelocking_strategy::handle_table(this=0x0000700001e7a968, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0, need_prelocking=0x0000700001e7a4e6) at sql_base.cc:5061:11

    frame #3: 0x0000000100445504 mariadbd`extend_table_list(thd=0x000000010480d088, tables=0x00000001182b9de0, prelocking_strategy=0x0000700001e7a968, has_prelocking_list=false) at sql_base.cc:3862:33

    frame #4: 0x0000000100447814 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182b9de0, counter=0x0000700001e7a964, flags=512, prelocking_strategy=0x0000700001e7a968, has_prelocking_list=false, ot_ctx=0x0000700001e7a828) at sql_base.cc:4162:10

    frame #5: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e7a978, counter=0x0000700001e7a964, flags=512, prelocking_strategy=0x0000700001e7a968) at sql_base.cc:4595:14

    frame #6: 0x000000010044abaa mariadbd`open_tables(thd=0x000000010480d088, tables=0x0000700001e7a978, counter=0x0000700001e7a964, flags=512, prelocking_strategy=0x0000700001e7a968) at sql_base.h:266:10

    frame #7: 0x000000010044a8fe mariadbd`open_normal_and_derived_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, flags=512, dt_phases=1) at sql_base.cc:5634:7

    frame #8: 0x0000000100578d8a mariadbd`mysql_test_insert_common(stmt=0x000000010796d488, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false) at sql_prepare.cc:1315:7

    frame #9: 0x0000000100578c95 mariadbd`mysql_test_insert(stmt=0x000000010796d488, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false) at sql_prepare.cc:1402:10

    frame #10: 0x0000000100571bc7 mariadbd`check_prepared_statement(stmt=0x000000010796d488) at sql_prepare.cc:2478:10

    frame #11: 0x000000010056b05b mariadbd`Prepared_statement::prepare(this=0x000000010796d488, packet="insert into t2 values (7)", packet_len=25) at sql_prepare.cc:4439:12

    frame #12: 0x000000010056bcf8 mariadbd`mysql_sql_stmt_prepare(thd=0x000000010480d088) at sql_prepare.cc:3030:19

    frame #13: 0x000000010051e917 mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=false) at sql_parse.cc:3960:5

    frame #14: 0x0000000100514af4 mariadbd`mysql_parse(thd=0x000000010480d088, rawbuf="prepare ps2 from 'insert into t2 values (7)'", length=44, parser_state=0x0000700001e7cc30) at sql_parse.cc:7999:18

(lldb) p key

(const MDL_key *) $0 = 0x0000700001e7a228

(lldb) p key->db_name()

(const char *) $1 = 0x0000700001e7a231 "test"

(lldb) p key->name()

(const char *) $2 = 0x0000700001e7a236 "tr"

On first running of 'EXECUTE ps2' the table t2 is opened and
a new partition is created as part of opening the table

    frame #1: 0x000000010043f7d8 mariadbd`TABLE::vers_switch_partition(this=0x0000000105827c88, thd=0x000000010480d088, table_list=0x00000001182bb4e8, ot_ctx=0x0000700001e796d8) at sql_base.cc:1791:13

    frame #2: 0x0000000100440c93 mariadbd`open_table(thd=0x000000010480d088, table_list=0x00000001182bb4e8, ot_ctx=0x0000700001e796d8) at sql_base.cc:2215:14

    frame #3: 0x0000000100447446 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182bb4e8, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=true, ot_ctx=0x0000700001e796d8) at sql_base.cc:4070:14

    frame #4: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14

    frame #5: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7

    frame #6: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, derived=true, flags=0) at sql_base.h:510:10

    frame #7: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9

    frame #8: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10

    frame #9: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14

It is resulted in returning an error by the function open_table() invoked from the function open_and_process_table

  * frame #0: 0x0000000100481f9c mariadbd`Open_table_context::can_recover_from_failed_open(this=0x0000700001e796d8) const at sql_base.h:546:21

    frame #1: 0x00000001004476b2 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182bb4e8, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=true, ot_ctx=0x0000700001e796d8) at sql_base.cc:4113:19

    frame #2: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14

    frame #3: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7

    frame #4: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, derived=true, flags=0) at sql_base.h:510:10

    frame #5: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9

    frame #6: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10

    frame #7: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14

but since the data member Open_table_context::m_action == OT_ADD_HISTORY_PARTITION
the tables opened when the PREPARE FROM statement was run are closed and trigger 'tr'
is removed from the hash table of used routines as part of this process and then the table
that caused opening failure is successfully re-opened in
Open_table_context::recover_from_failed_open and the trigger tr added again into the hash
table of used routines when the table being opened is extended to open triggers referenced
by the table.

  * frame #0: 0x00000001003c0a12 mariadbd`sp_add_used_routine(prelocking_ctx=0x00000001182b80b0, arena=0x000000010796d488, key=0x0000700001e790d8, handler=0x0000000101991770, belong_to_view=0x0000000000000000) at sp.cc:2335:30

    frame #1: 0x000000010068ec55 mariadbd`Table_triggers_list::add_tables_and_routines_for_triggers(this=0x000000010480c2a0, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0) at sql_trigger.cc:2540:15

    frame #2: 0x0000000100448722 mariadbd`DML_prelocking_strategy::handle_table(this=0x0000700001e79830, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0, need_prelocking=0x0000700001e79396) at sql_base.cc:5061:11

    frame #3: 0x0000000100445504 mariadbd`extend_table_list(thd=0x000000010480d088, tables=0x00000001182b9de0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false) at sql_base.cc:3862:33

    frame #4: 0x0000000100447814 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182b9de0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false, ot_ctx=0x0000700001e796d8) at sql_base.cc:4162:10

    frame #5: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14

    frame #6: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7

    frame #7: 0x00000001006f66de mariadbd`open_and_lock_tables0

The method Reprepare_observer::report_error is called later after the table successfully opened.

  * frame #0: 0x000000010056f808 mariadbd`Reprepare_observer::report_error(this=0x0000700001e7b2c8, thd=0x000000010480d088) at sql_prepare.cc:4030:3

    frame #1: 0x0000000100441b82 mariadbd`check_and_update_table_version(thd=0x000000010480d088, tables=0x00000001182bbdf0, table_share=0x00000001079866a0) at sql_base.cc:3021:36

    frame #2: 0x00000001004478b6 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182bbdf0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false, ot_ctx=0x0000700001e796d8) at sql_base.cc:4170:10

    frame #3: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14

    frame #4: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7

    frame #5: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, derived=true, flags=0) at sql_base.h:510:10

    frame #6: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9

    frame #7: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10

    frame #8: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14

On the second execution everything happens without any surprises, the statement 'execute ps2' is run successfully.
On the third execution, a new partition is created on opening the table t1 that results in
a failure after that close_tables_for_reopen() is called and an entry for trigger is removed
from the hash table of opened routines.

Then the table t1 is re-opened

  * frame #0: 0x0000000100443791 mariadbd`Open_table_context::recover_from_failed_open(this=0x0000700001e796d8) at sql_base.cc:3471:37

    frame #1: 0x0000000100445d68 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182c96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4625:22

    frame #2: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182c96e8, tables=0x00000001182c9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7

    frame #3: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182c9de0, derived=true, flags=0) at sql_base.h:510:10

    frame #4: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182c9de0, fields=0x00000001182c90a0, values_list=0x00000001182c90e8, update_fields=0x00000001182c90d0, update_values=0x00000001182c90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9

    frame #5: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10

    frame #6: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14

and after that server hit assert since we try to allocate a new memory chunk and memory root of prepared statement has been already marked as read only.

  * frame #0: 0x00000001003c0a12 mariadbd`sp_add_used_routine(prelocking_ctx=0x00000001182c80b0, arena=0x000000010796d488, key=0x0000700001e790d8, handler=0x0000000101991770, belong_to_view=0x0000000000000000) at sp.cc:2335:30

    frame #1: 0x000000010068ec55 mariadbd`Table_triggers_list::add_tables_and_routines_for_triggers(this=0x000000010480c2a0, thd=0x000000010480d088, prelocking_ctx=0x00000001182c80b0, table_list=0x00000001182c9de0) at sql_trigger.cc:2540:15

    frame #2: 0x0000000100448722 mariadbd`DML_prelocking_strategy::handle_table(this=0x0000700001e79830, thd=0x000000010480d088, prelocking_ctx=0x00000001182c80b0, table_list=0x00000001182c9de0, need_prelocking=0x0000700001e79396) at sql_base.cc:5061:11

    frame #3: 0x0000000100445504 mariadbd`extend_table_list(thd=0x000000010480d088, tables=0x00000001182c9de0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false) at sql_base.cc:3862:33

    frame #4: 0x0000000100447814 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182c9de0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false, ot_ctx=0x0000700001e796d8) at sql_base.cc:4162:10

    frame #5: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182c96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14

    frame #6: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182c96e8, tables=0x00000001182c9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7

    frame #7: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182c9de0, derived=true, flags=0) at sql_base.h:510:10

So, in short without too much details:
on third run of
EXECUTE ps2
a new partition is created on opening a table t1 that causes all opened tables to be closed
and every routine that any of theses tables depending on are evicted from the cache of
stored routines. Then all tables used (directly and indirectly) by the INSERT statement are
opened and on attempt to insert an entry for trigger 'tr' into the cache of stored routines
a new memory allocated. Next time a new partition should be created as part of firing
the trigger a new memory will be allocated for an entry in the cache of stored routine,
and it will happen every time a new partition be created by trigger invocation in context
of the same prepared statement. As soon as the prepared statement be deallocated all
this memory be released but until that the memory occupied by the prepared statement
will be continually increase.

Attachments

Issue Links

relates to

MDEV-14959 Control over memory allocated for SP/PS

Closed

MDEV-30889 Memory leak issues with MariaDB 10.6.12 and OOM Crashes

Confirmed

MDEV-35359 Assertion `(mem_root->flags & 4) == 0' failed in sp_add_used_routine / Table_triggers_list::add_tables_and_routines_for_triggers

Open

Activity

People

Assignee:: Aleksey Midenkov

Reporter:: Dmitry Shulga

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 2023-04-25 16:52

Updated:: 2024-11-07 19:30

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.