Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-34863

RAM Usage Changed Significantly Between 10.11 Releases

Details

    Description

      We run a pretty large database (about 5.5TB currently), and we were running into issues with the growing history list length that was well known under 10.11.6.

      We upgraded to 10.11.9 (10.11.8 first on some servers), and we immediately noticed that the memory usage was wildly different. 10.11.6 seemed very good at freeing memory, and it was great to see an accurate representation of how many resources the servers are actually using.

      With 10.11.9, you can see that the graph does a reverse hockey stick. It maximizes allocated memory based on innodb_buffer_pool_size, and then stays occupied, hardly fluctuating.

      We had seen this same kind of usage in MariaDB 10.3 and 10.6, so it was really nice when earlier 10.11 had what looked like much more robust memory handling. We wouldn't have a big problem with it except that one of our replicas unexpectedly OOMed over the weekend.

      So, what changed from 10.11.6 to 10.11.8? Is there a way to set more aggressive memory management?

      Attachments

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. MDEV-36197 Implement Buffer Pool Auto-Scaling Based on RAM Availability

          • Major - Major loss of function.
          • Open
          is part of

          Task - A task that needs to be done. MDEV-29445 reorganise innodb buffer pool (and remove buffer pool chunks)

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-24670 avoid OOM by linux kernel co-operative memory management

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-29445 reorganise innodb buffer pool (and remove buffer pool chunks)

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34753 memory pressure - erroneous termination condition

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-35424 False alarm/crash: innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Task - A task that needs to be done. MDEV-29432 innodb huge pages reclaim

          • Major - Major loss of function.
          • Needs Feedback

          Bug - A problem which impairs or prevents the functions of the product. MDEV-31953 madvise(frame, srv_page_size, MADV_FREE) is causing a performance regression

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            View 12 older comments
            marko Marko Mäkelä added a comment -

            This has been implemented in the same branch as MDEV-29445.

            marko Marko Mäkelä added a comment - This has been implemented in the same branch as MDEV-29445 .
            serg Sergei Golubchik added a comment -

            Will be fixed together with MDEV-29445, so closing.

            serg Sergei Golubchik added a comment - Will be fixed together with MDEV-29445 , so closing.
            marko Marko Mäkelä added a comment -

            While this fix has been tested and reviewed in the same development branch with MDEV-29445, it is a separate change on its own.

            marko Marko Mäkelä added a comment - While this fix has been tested and reviewed in the same development branch with MDEV-29445 , it is a separate change on its own.
            marko Marko Mäkelä added a comment -

            The memory pressure stall interface is now disabled by default. To enable it:

            SET GLOBAL innodb_buffer_pool_size_auto_min=0;

            or set it to a minimum value that you allow the buffer pool to be shrunk to, in response to running out of memory. Each event will attempt to reduce innodb_buffer_pool_size to (innodb_buffer_pool_size+innodb_buffer_pool_size_auto_min)/2.

            marko Marko Mäkelä added a comment - The memory pressure stall interface is now disabled by default. To enable it: SET GLOBAL innodb_buffer_pool_size_auto_min=0; or set it to a minimum value that you allow the buffer pool to be shrunk to, in response to running out of memory. Each event will attempt to reduce innodb_buffer_pool_size to (innodb_buffer_pool_size+innodb_buffer_pool_size_auto_min)/2 .
            suwalski Pat Suwalski added a comment -

            Awesome, gentlemen. I am looking forward to trying it.

            suwalski Pat Suwalski added a comment - Awesome, gentlemen. I am looking forward to trying it.

            People

              marko Marko Mäkelä
              suwalski Pat Suwalski
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.