Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-20698

Master slowly running out of memory and gets killed by oom-killer

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Critical
    • Resolution: Unresolved
    • 10.3.17
    • None
    • Server
    • CentOS 7 (3.10.0-1062.1.1.el7.x86_64)

    Description

      Hi guys,

      we experience a serious problem since we upgraded from MariaDB 10.1.25 to MariaDB 10.3.17 about two weeks ago.

      The memory consumption of our master server slowly increases over time for no obvious reason and then the service gets killed by the oom-killer. This timespan is only about one day! Like for example today at 9:00 AM, the memory consumption of mysqld was 1,952 MB, while about two hours later (at 11:13 AM), it hit 3,484 MB (that's about 60% of available RAM). Even though the innodb_buffer_pool_size is limited to 1 GB.
      To prevent the master from getting killed by oom-killer, I manually restart the MariaDB service every evening.

      What's weird about this is, that before the upgrade the cluster consisting of one master and one slave worked like a charm. Even though we have a cluster controlled by MaxScale, which performs automatic failover if the master gets killed, this is not very pleasing if it happens regularily while people are working.

      Does anybody have an idea where this could suddenly come from or how to find out what's going on? Is the database corrupt?

      If you need more information or details, I will provide you with as much info as I can.

      I really hope that you guys can help me, as I absolutely ran out of ideas.

      Thank you in advance!

      Regards,
      matze

      Attachments

        1. mariadb_tcmalloc.service
          1 kB
          Matthias
        2. mysqld.prof.svg
          75 kB
          Matthias

        Issue Links

          Activity

            People

              Unassigned Unassigned
              matze Matthias
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.