Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-773

100% CPU on idle MaxScale with MaxInfo

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 1.4.1, 1.4.3
    • 2.0.1
    • maxinfo
    • None
    • Ubuntu 14.04.3, AWS EC2 c4.4xlarge (16core, 30GB)
    • 2016-17

    Description

      I have 3 MaxScale instances with the same configuration. 1 is always active (any one of the 3), the other 2 are standby. Every 10 seconds or so, information is checked through the MaxInfo JSON listener. I've noticed that only idle instances exhibit what appears to be an memory leak issue with MaxInfo sockets not closing. The active instance, which is also running MaxInfo and having its JSON listener queried, doesn't have this problem. If a new instance is made active and the former active instance becomes idle, the former instance starts to have this problem eventually.

      We have a monit daemon on each MaxScale instance which is checking the availability of the MaxInfo's /status URI, which is why in the attached image you'll see the spike drop; that's monit restarting the MaxScale service. We also have a curl request that runs periodically to gather status information. If monit is off the and MaxScale reaches this high memory state the curl requests just hang forever until MaxScale is restarted.

      I have reason to believe this is a problem specific to our heavy use of MaxInfo, simply because when we started using MaxInfo this started to happen, but it could also be a coincidence. I have not tried to reproduce this with a newer version of MaxScale yet but I do plan to. I've found this behavior to exist on both versions 1.4.1 and 1.4.3

      Attachments

        1. maxscale_idle_cpu.png
          269 kB
          Mathew Hornbeek
        2. maxscale_leak.png
          50 kB
          Mathew Hornbeek

        Issue Links

          Activity

            People

              markus makela markus makela
              mat_h Mathew Hornbeek
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.