Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Won't Fix
    • 5.5.37-galera, 5.5.39-galera
    • N/A
    • Galera
    • Ubuntu 12.04.4 LTS (GNU/Linux 2.6.32-28-server x86_64)

    Description

      package versions:
      mariadb-galera-server-5.5-5.5.37+maria-1~precise
      galera-25.3.5-wheezy

      3 or 4 servers have a similar memory leak profile to the memory-month-g1 image attached. Variables and settings attached are from the g1 server. The dates that an increase in memory usage occurs is the same on all server. The 4th server, memory-month-repl1 is only used as read only and has significantly lower traffic and lower memory usage overall.

      The restart at the beginning of the graph was an upgrade from an upgrade from 5.5.32. Looking at the year graphs it appears there was a leak there too that I hadn't noticed before.

      Client has another regional database on 5.5.37-MariaDB-1~precise-log (non-galera) with similar structured databases and code usage. If this non-galera instance has a leak is much slower.

      top - 09:14:36 up 172 days,  7:56,  2 users,  load average: 0.07, 0.02, 0.00
      Tasks: 128 total,   1 running, 127 sleeping,   0 stopped,   0 zombie
      Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
      Mem:   7970076k total,  7873132k used,    96944k free,   126140k buffers
      Swap:  1073144k total,     6408k used,  1066736k free,  2927628k cached
       
        PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                           
       7637 mysql     20   0 12.7g 5.5g 1.3g S    0 72.0 505:21.12 mysqld                                                                            
      13405 syslog    20   0  254m 4540 1040 S    0  0.1   3:04.63 rsyslogd                                                                          
      25822 openquer  20   0 17292 1428 1072 R    0  0.0   0:00.03 top                                                                               
          1 root      20   0 24356 1556  696 S    0  0.0   0:00.56 init          

      I've restarted repl with no query cache and lower usage settings as it was overallocated.

      There is a UAT environment with this that I can test out patches with.

      Would trying the patch from MDEV-4974 be of use?

      Attachments

        1. memory-month-g1.png
          memory-month-g1.png
          39 kB
        2. memory-month-repl.png
          memory-month-repl.png
          42 kB
        3. status.txt
          51 kB
        4. variables.txt
          48 kB
        5. valgrind.mysqld.7706
          392 kB

        Activity

          danblack Daniel Black added a comment -

          i'm not sure that valgrind report is going to help.

          I did attempt to get a valgrind on a UAT instance. Unfortunately there was insufficient RAM On the UAT instance to successfully run valgrind even with innodb_buffer_size set abnormally low.

          This week the UAT instance RAM has been upgraded and I'm running valgrind on the instance as I type. Apologies for the delay and lack of communication.

          danblack Daniel Black added a comment - i'm not sure that valgrind report is going to help. I did attempt to get a valgrind on a UAT instance. Unfortunately there was insufficient RAM On the UAT instance to successfully run valgrind even with innodb_buffer_size set abnormally low. This week the UAT instance RAM has been upgraded and I'm running valgrind on the instance as I type. Apologies for the delay and lack of communication.

          Besides some seemingly acceptable valgrind errors, I am still not able to reproduce this. Also tried to populate tables from the uploaded script with some sample data but no luck so far.

          nirbhay_c Nirbhay Choubey (Inactive) added a comment - Besides some seemingly acceptable valgrind errors, I am still not able to reproduce this. Also tried to populate tables from the uploaded script with some sample data but no luck so far.

          Could you try with 5.5.54-galera and see if this repeats?

          jplindst Jan Lindström (Inactive) added a comment - Could you try with 5.5.54-galera and see if this repeats?
          danblack Daniel Black added a comment -

          Sorry I've no access to this former employeer's client any more. arjen can say if its still a problem.

          MDEV-12205 has a prototype for debugging memory leaks that shouldn't be too invasive.

          danblack Daniel Black added a comment - Sorry I've no access to this former employeer's client any more. arjen can say if its still a problem. MDEV-12205 has a prototype for debugging memory leaks that shouldn't be too invasive.
          ralf.gebhardt Ralf Gebhardt added a comment -

          Unsupported version of Galera Cluster

          ralf.gebhardt Ralf Gebhardt added a comment - Unsupported version of Galera Cluster

          People

            seppo Seppo Jaakola
            danblack Daniel Black
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.