Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-11811

dual master with parallel replication memory leak in write master

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Fixed
    • 10.0(EOL), 10.1(EOL), 10.2(EOL)
    • 10.0.30, 10.1.21
    • Replication
    • MariaDB 10.1.8 or MariaDB 10.1.18 on debina 8

    Description

      dual master with parallel replication memory leak in write master
      (slave_parallel_threads=8)

      master(active)<----->master(standby)

      if write in master(active) frequently, master(active) use memory more and more and then physical memory give out,2G physical memory per day,and then swap give up,the OS OOM Killer killed mysqld process.

      if you stop slave on master(active) before OOM Killer killed,the OS reclaim a lot of physical memory

      Attachments

        Activity

          ruochen wy added a comment -

          https://github.com/MariaDB/server/blob/74d52ded721d8d39bfb25d18a0b0b8ea8a7af884/sql/log_event.cc

          Gtid_list_log_event::do_apply_event function not use free_root(thd->mem_root, MYF(MY_KEEP_PREALLOC));

          i don't ensure.

          ruochen wy added a comment - https://github.com/MariaDB/server/blob/74d52ded721d8d39bfb25d18a0b0b8ea8a7af884/sql/log_event.cc Gtid_list_log_event::do_apply_event function not use free_root(thd->mem_root, MYF(MY_KEEP_PREALLOC)); i don't ensure.

          knielsen, would you want to handle this?

          elenst Elena Stepanova added a comment - knielsen , would you want to handle this?

          Hm, that's a good point actually.

          Gtid_list_log_event::do_apply_event() is one of the few functions that run
          on the active master in circular replication. And it modifies tables, so
          could be using thd->mem_root. And on standby slave,
          Query_log_event::do_apply_event() will run all the time to free the
          thd->mem_root - but not on the active master.

          I'll try to come up with a patch.

          knielsen Kristian Nielsen added a comment - Hm, that's a good point actually. Gtid_list_log_event::do_apply_event() is one of the few functions that run on the active master in circular replication. And it modifies tables, so could be using thd->mem_root. And on standby slave, Query_log_event::do_apply_event() will run all the time to free the thd->mem_root - but not on the active master. I'll try to come up with a patch.

          You are absolutely right, Gtid_list_log_event::do_apply_event() is missing free_root(), and this is causing a memory leak.

          Very nice catch, thanks!

          I was able to reproduce the leak with the attached (crude) mysql-test-run.pl script. It is fixed by this patch, which just adds the missing free_root():

          http://lists.askmonty.org/pipermail/commits/2017-January/010445.html

          knielsen Kristian Nielsen added a comment - You are absolutely right, Gtid_list_log_event::do_apply_event() is missing free_root(), and this is causing a memory leak. Very nice catch, thanks! I was able to reproduce the leak with the attached (crude) mysql-test-run.pl script. It is fixed by this patch, which just adds the missing free_root(): http://lists.askmonty.org/pipermail/commits/2017-January/010445.html

          I asked Monty to review the patch

          knielsen Kristian Nielsen added a comment - I asked Monty to review the patch

          Pushed to 10.0

          knielsen Kristian Nielsen added a comment - Pushed to 10.0

          People

            knielsen Kristian Nielsen
            ruochen wy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.