Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-31127

Possible memory leak on running insert in PS mode against a table with a trigger that fires creation of a new partiton

    XMLWordPrintable

Details

    Description

      Consider the following test case

      -- source include/have_partition.inc
      -- source suite/versioning/common.inc
      -- source suite/versioning/engines.inc
      -- source include/have_sequence.inc
       
      set timestamp= unix_timestamp('2000-01-01 00:00:00');
      create or replace table t1 (x int) with system versioning
      partition by system_time interval 1 hour auto
      partitions 3;
       
      create table t2 (x int);
      create trigger tr after insert on t2 for each row update t1 set x= x + 11;
       
      prepare ps2 from 'insert into t2 values (7)';
       
      set timestamp= unix_timestamp('2000-01-01 06:00:00');
      Execute ps2;
      eXecute ps2;
       
      set timestamp= unix_timestamp('2000-01-01 15:00:00');
      exEcute ps2; # <=== Memory leak happens here
      exeCute ps2;
       
      # Cleanup
      drop tables t1, t2;
      deallocate prepare ps2
      

      On running PREPARE ps2 a new entry for the trigger 'tr' is added into the hash table of used routines (this hash table is contained in lex).

        * frame #0: 0x00000001003c0a12 mariadbd`sp_add_used_routine(prelocking_ctx=0x00000001182b80b0, arena=0x000000010796d488, key=0x0000700001e7a228, handler=0x0000000101991770, belong_to_view=0x0000000000000000) at sp.cc:2335:30
          frame #1: 0x000000010068ec55 mariadbd`Table_triggers_list::add_tables_and_routines_for_triggers(this=0x000000010480c2a0, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0) at sql_trigger.cc:2540:15
          frame #2: 0x0000000100448722 mariadbd`DML_prelocking_strategy::handle_table(this=0x0000700001e7a968, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0, need_prelocking=0x0000700001e7a4e6) at sql_base.cc:5061:11
          frame #3: 0x0000000100445504 mariadbd`extend_table_list(thd=0x000000010480d088, tables=0x00000001182b9de0, prelocking_strategy=0x0000700001e7a968, has_prelocking_list=false) at sql_base.cc:3862:33
          frame #4: 0x0000000100447814 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182b9de0, counter=0x0000700001e7a964, flags=512, prelocking_strategy=0x0000700001e7a968, has_prelocking_list=false, ot_ctx=0x0000700001e7a828) at sql_base.cc:4162:10
          frame #5: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e7a978, counter=0x0000700001e7a964, flags=512, prelocking_strategy=0x0000700001e7a968) at sql_base.cc:4595:14
          frame #6: 0x000000010044abaa mariadbd`open_tables(thd=0x000000010480d088, tables=0x0000700001e7a978, counter=0x0000700001e7a964, flags=512, prelocking_strategy=0x0000700001e7a968) at sql_base.h:266:10
          frame #7: 0x000000010044a8fe mariadbd`open_normal_and_derived_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, flags=512, dt_phases=1) at sql_base.cc:5634:7
          frame #8: 0x0000000100578d8a mariadbd`mysql_test_insert_common(stmt=0x000000010796d488, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false) at sql_prepare.cc:1315:7
          frame #9: 0x0000000100578c95 mariadbd`mysql_test_insert(stmt=0x000000010796d488, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false) at sql_prepare.cc:1402:10
          frame #10: 0x0000000100571bc7 mariadbd`check_prepared_statement(stmt=0x000000010796d488) at sql_prepare.cc:2478:10
          frame #11: 0x000000010056b05b mariadbd`Prepared_statement::prepare(this=0x000000010796d488, packet="insert into t2 values (7)", packet_len=25) at sql_prepare.cc:4439:12
          frame #12: 0x000000010056bcf8 mariadbd`mysql_sql_stmt_prepare(thd=0x000000010480d088) at sql_prepare.cc:3030:19
          frame #13: 0x000000010051e917 mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=false) at sql_parse.cc:3960:5
          frame #14: 0x0000000100514af4 mariadbd`mysql_parse(thd=0x000000010480d088, rawbuf="prepare ps2 from 'insert into t2 values (7)'", length=44, parser_state=0x0000700001e7cc30) at sql_parse.cc:7999:18
       
      (lldb) p key
      (const MDL_key *) $0 = 0x0000700001e7a228
      (lldb) p key->db_name()
      (const char *) $1 = 0x0000700001e7a231 "test"
      (lldb) p key->name()
      (const char *) $2 = 0x0000700001e7a236 "tr"
      

      On first running of 'EXECUTE ps2' the table t2 is opened and
      a new partition is created as part of opening the table

          frame #1: 0x000000010043f7d8 mariadbd`TABLE::vers_switch_partition(this=0x0000000105827c88, thd=0x000000010480d088, table_list=0x00000001182bb4e8, ot_ctx=0x0000700001e796d8) at sql_base.cc:1791:13
          frame #2: 0x0000000100440c93 mariadbd`open_table(thd=0x000000010480d088, table_list=0x00000001182bb4e8, ot_ctx=0x0000700001e796d8) at sql_base.cc:2215:14
          frame #3: 0x0000000100447446 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182bb4e8, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=true, ot_ctx=0x0000700001e796d8) at sql_base.cc:4070:14
          frame #4: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14
          frame #5: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7
          frame #6: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, derived=true, flags=0) at sql_base.h:510:10
          frame #7: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9
          frame #8: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10
          frame #9: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14
      

      It is resulted in returning an error by the function open_table() invoked from the function open_and_process_table

        * frame #0: 0x0000000100481f9c mariadbd`Open_table_context::can_recover_from_failed_open(this=0x0000700001e796d8) const at sql_base.h:546:21
          frame #1: 0x00000001004476b2 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182bb4e8, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=true, ot_ctx=0x0000700001e796d8) at sql_base.cc:4113:19
          frame #2: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14
          frame #3: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7
          frame #4: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, derived=true, flags=0) at sql_base.h:510:10
          frame #5: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9
          frame #6: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10
          frame #7: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14
      

      but since the data member Open_table_context::m_action == OT_ADD_HISTORY_PARTITION
      the tables opened when the PREPARE FROM statement was run are closed and trigger 'tr'
      is removed from the hash table of used routines as part of this process and then the table
      that caused opening failure is successfully re-opened in
      Open_table_context::recover_from_failed_open and the trigger tr added again into the hash
      table of used routines when the table being opened is extended to open triggers referenced
      by the table.

        * frame #0: 0x00000001003c0a12 mariadbd`sp_add_used_routine(prelocking_ctx=0x00000001182b80b0, arena=0x000000010796d488, key=0x0000700001e790d8, handler=0x0000000101991770, belong_to_view=0x0000000000000000) at sp.cc:2335:30
          frame #1: 0x000000010068ec55 mariadbd`Table_triggers_list::add_tables_and_routines_for_triggers(this=0x000000010480c2a0, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0) at sql_trigger.cc:2540:15
          frame #2: 0x0000000100448722 mariadbd`DML_prelocking_strategy::handle_table(this=0x0000700001e79830, thd=0x000000010480d088, prelocking_ctx=0x00000001182b80b0, table_list=0x00000001182b9de0, need_prelocking=0x0000700001e79396) at sql_base.cc:5061:11
          frame #3: 0x0000000100445504 mariadbd`extend_table_list(thd=0x000000010480d088, tables=0x00000001182b9de0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false) at sql_base.cc:3862:33
          frame #4: 0x0000000100447814 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182b9de0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false, ot_ctx=0x0000700001e796d8) at sql_base.cc:4162:10
          frame #5: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14
          frame #6: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7
          frame #7: 0x00000001006f66de mariadbd`open_and_lock_tables0
      

      The method Reprepare_observer::report_error is called later after the table successfully opened.

        * frame #0: 0x000000010056f808 mariadbd`Reprepare_observer::report_error(this=0x0000700001e7b2c8, thd=0x000000010480d088) at sql_prepare.cc:4030:3
          frame #1: 0x0000000100441b82 mariadbd`check_and_update_table_version(thd=0x000000010480d088, tables=0x00000001182bbdf0, table_share=0x00000001079866a0) at sql_base.cc:3021:36
          frame #2: 0x00000001004478b6 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182bbdf0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false, ot_ctx=0x0000700001e796d8) at sql_base.cc:4170:10
          frame #3: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182b96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14
          frame #4: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182b96e8, tables=0x00000001182b9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7
          frame #5: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182b9de0, derived=true, flags=0) at sql_base.h:510:10
          frame #6: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182b9de0, fields=0x00000001182b90a0, values_list=0x00000001182b90e8, update_fields=0x00000001182b90d0, update_values=0x00000001182b90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9
          frame #7: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10
          frame #8: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14
      

      On the second execution everything happens without any surprises, the statement 'execute ps2' is run successfully.
      On the third execution, a new partition is created on opening the table t1 that results in
      a failure after that close_tables_for_reopen() is called and an entry for trigger is removed
      from the hash table of opened routines.

      Then the table t1 is re-opened

        * frame #0: 0x0000000100443791 mariadbd`Open_table_context::recover_from_failed_open(this=0x0000700001e796d8) at sql_base.cc:3471:37
          frame #1: 0x0000000100445d68 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182c96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4625:22
          frame #2: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182c96e8, tables=0x00000001182c9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7
          frame #3: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182c9de0, derived=true, flags=0) at sql_base.h:510:10
          frame #4: 0x00000001004bbb96 mariadbd`mysql_insert(thd=0x000000010480d088, table_list=0x00000001182c9de0, fields=0x00000001182c90a0, values_list=0x00000001182c90e8, update_fields=0x00000001182c90d0, update_values=0x00000001182c90b8, duplic=DUP_ERROR, ignore=false, result=0x0000000000000000) at sql_insert.cc:767:9
          frame #5: 0x000000010052174b mariadbd`mysql_execute_command(thd=0x000000010480d088, is_called_from_prepared_stmt=true) at sql_parse.cc:4569:10
          frame #6: 0x0000000100573621 mariadbd`Prepared_statement::execute(this=0x000000010796d488, expanded_query=0x0000700001e7b3b0, open_cursor=false) at sql_prepare.cc:5253:14
      

      and after that server hit assert since we try to allocate a new memory chunk and memory root of prepared statement has been already marked as read only.

        * frame #0: 0x00000001003c0a12 mariadbd`sp_add_used_routine(prelocking_ctx=0x00000001182c80b0, arena=0x000000010796d488, key=0x0000700001e790d8, handler=0x0000000101991770, belong_to_view=0x0000000000000000) at sp.cc:2335:30
          frame #1: 0x000000010068ec55 mariadbd`Table_triggers_list::add_tables_and_routines_for_triggers(this=0x000000010480c2a0, thd=0x000000010480d088, prelocking_ctx=0x00000001182c80b0, table_list=0x00000001182c9de0) at sql_trigger.cc:2540:15
          frame #2: 0x0000000100448722 mariadbd`DML_prelocking_strategy::handle_table(this=0x0000700001e79830, thd=0x000000010480d088, prelocking_ctx=0x00000001182c80b0, table_list=0x00000001182c9de0, need_prelocking=0x0000700001e79396) at sql_base.cc:5061:11
          frame #3: 0x0000000100445504 mariadbd`extend_table_list(thd=0x000000010480d088, tables=0x00000001182c9de0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false) at sql_base.cc:3862:33
          frame #4: 0x0000000100447814 mariadbd`open_and_process_table(thd=0x000000010480d088, tables=0x00000001182c9de0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830, has_prelocking_list=false, ot_ctx=0x0000700001e796d8) at sql_base.cc:4162:10
          frame #5: 0x0000000100445cb8 mariadbd`open_tables(thd=0x000000010480d088, options=0x00000001182c96e8, start=0x0000700001e797e0, counter=0x0000700001e797cc, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:4595:14
          frame #6: 0x000000010044a62f mariadbd`open_and_lock_tables(thd=0x000000010480d088, options=0x00000001182c96e8, tables=0x00000001182c9de0, derived=true, flags=0, prelocking_strategy=0x0000700001e79830) at sql_base.cc:5570:7
          frame #7: 0x00000001006f66de mariadbd`open_and_lock_tables(thd=0x000000010480d088, tables=0x00000001182c9de0, derived=true, flags=0) at sql_base.h:510:10
      

      So, in short without too much details:
      on third run of
      EXECUTE ps2
      a new partition is created on opening a table t1 that causes all opened tables to be closed
      and every routine that any of theses tables depending on are evicted from the cache of
      stored routines. Then all tables used (directly and indirectly) by the INSERT statement are
      opened and on attempt to insert an entry for trigger 'tr' into the cache of stored routines
      a new memory allocated. Next time a new partition should be created as part of firing
      the trigger a new memory will be allocated for an entry in the cache of stored routine,
      and it will happen every time a new partition be created by trigger invocation in context
      of the same prepared statement. As soon as the prepared statement be deallocated all
      this memory be released but until that the memory occupied by the prepared statement
      will be continually increase.

      Attachments

        Issue Links

          Activity

            People

              midenok Aleksey Midenkov
              shulga Dmitry Shulga
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.