Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-36197

Implement Buffer Pool Auto-Scaling Based on RAM Availability

Details

    Description

      Description
      Enable the buffer pool to adjust innodb_buffer_pool_size based on available RAM, targeting 75-80% by default, with configurable scaling factors and limits for stability. See https://jira.mariadb.org/browse/MDEV-36194 for more information on use case.

      Tasks

      1. Implement periodic RAM detection (e.g., via syscalls or OS notifications).
        Add system variables:
      2. buffer_pool_auto_scale_method= {OFF, RAM_percentage}

        (default: OFF).

      3. buffer_pool_auto_scale_factor=N (default: 0.75, range: 0.1–1.0).
      4. buffer_pool_min_size (default: 128MB), buffer_pool_max_size (default: unlimited).
      5. Set innodb_buffer_pool_size = RAM * buffer_pool_auto_scale_factor, respecting limits.
      6. Reject manual SET GLOBAL innodb_buffer_pool_size when auto-scaling is active.
      7. Make polling frequency configurable (e.g., once per second, per minute), defaulting to once per minute.

      Acceptance Criteria

      1. buffer_pool_auto_scale_method=RAM_percentage activates RAM-based scaling.
      2. Example: 16GB RAM, buffer_pool_auto_scale_factor=0.75, innodb_buffer_pool_size=12GB.
      3. buffer_pool_auto_scale_factor configurable between 0.1–1.0.
      4. Respects buffer_pool_min_size and buffer_pool_max_size.
      5. Manual SET GLOBAL innodb_buffer_pool_size fails with an error.
      6. Ensure the system variables are manually configurable.

      Attachments

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. MDEV-36194 Automatic Scaling for Dynamic Environments

          • Major - Major loss of function.
          • Open
          is blocked by

          Task - A task that needs to be done. MDEV-29445 reorganise innodb buffer pool (and remove buffer pool chunks)

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MDEV-34863 RAM Usage Changed Significantly Between 10.11 Releases

          • Major - Major loss of function.
          • Closed
          relates to

          Task - A task that needs to be done. MDEV-24670 avoid OOM by linux kernel co-operative memory management

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            adamluciano Adam Luciano created issue -
            adamluciano Adam Luciano made changes -
            Field Original Value New Value
            wlad Vladislav Vaintroub made changes -
            Assignee Vladislav Vaintroub [ wlad ] Marko Mäkelä [ marko ]
            wlad Vladislav Vaintroub made changes -
            Labels server innodb
            wlad Vladislav Vaintroub made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            wlad Vladislav Vaintroub made changes -
            Labels innodb
            adamluciano Adam Luciano made changes -
            Description *Description*
            Enable the buffer pool to adjust innodb_buffer_pool_size based on available RAM, targeting 75-80% by default, with configurable scaling factors and limits for stability.

            *Tasks*
            # Implement periodic RAM detection (e.g., via syscalls or OS notifications).
            Add system variables:
            # buffer_pool_auto_scale_method={OFF, RAM_percentage} (default: OFF).
            # buffer_pool_auto_scale_factor=N (default: 0.75, range: 0.1–1.0).
            # buffer_pool_min_size (default: 128MB), buffer_pool_max_size (default: unlimited).
            # Set innodb_buffer_pool_size = RAM * buffer_pool_auto_scale_factor, respecting limits.
            # Reject manual SET GLOBAL innodb_buffer_pool_size when auto-scaling is active.
            # Make polling frequency configurable (e.g., once per second, per minute), defaulting to once per minute.

            *Acceptance Criteria*
            # buffer_pool_auto_scale_method=RAM_percentage activates RAM-based scaling.
            # Example: 16GB RAM, buffer_pool_auto_scale_factor=0.75, innodb_buffer_pool_size=12GB.
            # buffer_pool_auto_scale_factor configurable between 0.1–1.0.
            # Respects buffer_pool_min_size and buffer_pool_max_size.
            # Manual SET GLOBAL innodb_buffer_pool_size fails with an error.             		*Description*
            Enable the buffer pool to adjust innodb_buffer_pool_size based on available RAM, targeting 75-80% by default, with configurable scaling factors and limits for stability. See https://jira.mariadb.org/browse/MDEV-36194 for more information on use case.

            *Tasks*
            # Implement periodic RAM detection (e.g., via syscalls or OS notifications).
            Add system variables:
            # buffer_pool_auto_scale_method={OFF, RAM_percentage} (default: OFF).
            # buffer_pool_auto_scale_factor=N (default: 0.75, range: 0.1–1.0).
            # buffer_pool_min_size (default: 128MB), buffer_pool_max_size (default: unlimited).
            # Set innodb_buffer_pool_size = RAM * buffer_pool_auto_scale_factor, respecting limits.
            # Reject manual SET GLOBAL innodb_buffer_pool_size when auto-scaling is active.
            # Make polling frequency configurable (e.g., once per second, per minute), defaulting to once per minute.

            *Acceptance Criteria*
            # buffer_pool_auto_scale_method=RAM_percentage activates RAM-based scaling.
            # Example: 16GB RAM, buffer_pool_auto_scale_factor=0.75, innodb_buffer_pool_size=12GB.
            # buffer_pool_auto_scale_factor configurable between 0.1–1.0.
            # Respects buffer_pool_min_size and buffer_pool_max_size.
            # Manual SET GLOBAL innodb_buffer_pool_size fails with an error.
            adamluciano Adam Luciano made changes -
            Labels kubernetes performance server
            danblack Daniel Black made changes -
            danblack Daniel Black made changes -
            danblack Daniel Black made changes -
            marko Marko Mäkelä made changes -
            marko Marko Mäkelä made changes -
            adamluciano Adam Luciano made changes -
            Description *Description*
            Enable the buffer pool to adjust innodb_buffer_pool_size based on available RAM, targeting 75-80% by default, with configurable scaling factors and limits for stability. See https://jira.mariadb.org/browse/MDEV-36194 for more information on use case.

            *Tasks*
            # Implement periodic RAM detection (e.g., via syscalls or OS notifications).
            Add system variables:
            # buffer_pool_auto_scale_method={OFF, RAM_percentage} (default: OFF).
            # buffer_pool_auto_scale_factor=N (default: 0.75, range: 0.1–1.0).
            # buffer_pool_min_size (default: 128MB), buffer_pool_max_size (default: unlimited).
            # Set innodb_buffer_pool_size = RAM * buffer_pool_auto_scale_factor, respecting limits.
            # Reject manual SET GLOBAL innodb_buffer_pool_size when auto-scaling is active.
            # Make polling frequency configurable (e.g., once per second, per minute), defaulting to once per minute.

            *Acceptance Criteria*
            # buffer_pool_auto_scale_method=RAM_percentage activates RAM-based scaling.
            # Example: 16GB RAM, buffer_pool_auto_scale_factor=0.75, innodb_buffer_pool_size=12GB.
            # buffer_pool_auto_scale_factor configurable between 0.1–1.0.
            # Respects buffer_pool_min_size and buffer_pool_max_size.
            # Manual SET GLOBAL innodb_buffer_pool_size fails with an error.             		*Description*
            Enable the buffer pool to adjust innodb_buffer_pool_size based on available RAM, targeting 75-80% by default, with configurable scaling factors and limits for stability. See https://jira.mariadb.org/browse/MDEV-36194 for more information on use case.

            *Tasks*
            # Implement periodic RAM detection (e.g., via syscalls or OS notifications).
            Add system variables:
            # buffer_pool_auto_scale_method={OFF, RAM_percentage} (default: OFF).
            # buffer_pool_auto_scale_factor=N (default: 0.75, range: 0.1–1.0).
            # buffer_pool_min_size (default: 128MB), buffer_pool_max_size (default: unlimited).
            # Set innodb_buffer_pool_size = RAM * buffer_pool_auto_scale_factor, respecting limits.
            # Reject manual SET GLOBAL innodb_buffer_pool_size when auto-scaling is active.
            # Make polling frequency configurable (e.g., once per second, per minute), defaulting to once per minute.

            *Acceptance Criteria*
            # buffer_pool_auto_scale_method=RAM_percentage activates RAM-based scaling.
            # Example: 16GB RAM, buffer_pool_auto_scale_factor=0.75, innodb_buffer_pool_size=12GB.
            # buffer_pool_auto_scale_factor configurable between 0.1–1.0.
            # Respects buffer_pool_min_size and buffer_pool_max_size.
            # Manual SET GLOBAL innodb_buffer_pool_size fails with an error.
            # Ensure the system variables are manually configurable.
            adamluciano Adam Luciano made changes -
            Fix Version/s 12.3 [ 30147 ]

            People

              marko Marko Mäkelä
              adamluciano Adam Luciano
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.