Details
-
Task
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
Description
The InnoDB buffer pool had been allocated in multiple chunks, because SET GLOBAL innodb_buffer_pool_size would extend the buffer pool in chunks. This would lead to many limitations, such as the inability to shrink the buffer pool below innodb_buffer_pool_chunk_size.
It would be cleaner to:
- allocate a contiguous virtual address range for a maximum supported size of buffer pool (a new parameter innodb_buffer_pool_size_max, which defaults to the initially specified innodb_buffer_pool_size)
- allow the innodb_buffer_pool_size to be changed in increments of 1 megabyte
- define a fixed mapping between the virtual memory addresses of buffer page descriptors page frames, to fix bugs like
MDEV-34677andMDEV-35485 - refactor the shrinking of the buffer pool to provide more meaningful progress output and to avoid hangs
The complicated logic of having multiple buffer pool chunks can be removed, and the parameter innodb_buffer_pool_chunk_size will be deprecated and ignored.
Attachments
Issue Links
- blocks
-
MDEV-21203 Bad value for the variable "Buffer pool size"
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
MDEV-36197 Implement Buffer Pool Auto-Scaling Based on RAM Availability
-
- Open
-
- includes
-
-
- Closed
-
- is blocked by
-
-
- Closed
-
- relates to
-
-
- Needs Feedback
-
-
-
- Stalled
-
-
-
- Closed
-
-
-
- Open
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Open
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
Activity
| Field | Original Value | New Value |
|---|---|---|
| Link |
This issue relates to |
| Link | This issue relates to MDEV-29432 [ MDEV-29432 ] |
| Priority | Major [ 3 ] | Minor [ 4 ] |
| Link |
This issue blocks |
| Link |
This issue blocks |
| Fix Version/s | 11.1 [ 28549 ] | |
| Assignee | Marko Mäkelä [ marko ] | |
| Labels | energy | energy performance |
| Priority | Minor [ 4 ] | Major [ 3 ] |
| Priority | Major [ 3 ] | Critical [ 2 ] |
| Link |
This issue blocks |
| Fix Version/s | 11.2 [ 28603 ] | |
| Fix Version/s | 11.1 [ 28549 ] |
| Link |
This issue blocks |
| Fix Version/s | 11.3 [ 28565 ] | |
| Fix Version/s | 11.2 [ 28603 ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Link | This issue is blocked by MDEV-25340 [ MDEV-25340 ] |
| Attachment | MDEV-29445-sizes.gnumeric [ 71618 ] |
| Link | This issue relates to MDEV-31976 [ MDEV-31976 ] |
| Link |
This issue blocks |
| Link |
This issue relates to |
| Fix Version/s | 11.4 [ 29301 ] | |
| Fix Version/s | 11.3 [ 28565 ] |
| Link |
This issue relates to |
| Link |
This issue blocks |
| Link | This issue relates to MDEV-32544 [ MDEV-32544 ] |
| Fix Version/s | 11.5 [ 29506 ] | |
| Fix Version/s | 11.4 [ 29301 ] |
| Status | In Progress [ 3 ] | Stalled [ 10000 ] |
| Status | Stalled [ 10000 ] | In Progress [ 3 ] |
| Link |
This issue is blocked by |
| Link |
This issue relates to |
| Status | In Progress [ 3 ] | Stalled [ 10000 ] |
| Fix Version/s | 11.6 [ 29515 ] | |
| Fix Version/s | 11.5 [ 29506 ] |
| Fix Version/s | 11.7 [ 29815 ] | |
| Fix Version/s | 11.6 [ 29515 ] |
| Zendesk Related Tickets | 201628 | |
| Zendesk active tickets | 201628 |
| Fix Version/s | 11.8 [ 29921 ] | |
| Fix Version/s | 11.7 [ 29815 ] |
| Priority | Critical [ 2 ] | Major [ 3 ] |
| Priority | Major [ 3 ] | Critical [ 2 ] |
| Link |
This issue relates to |
| Fix Version/s | 11.9 [ 29945 ] | |
| Fix Version/s | 11.8 [ 29921 ] |
| Status | Stalled [ 10000 ] | In Progress [ 3 ] |
| Link | This issue is blocked by MDEV-25340 [ MDEV-25340 ] |
| Link |
This issue relates to |
| Assignee | Marko Mäkelä [ marko ] | Debarun Banerjee [ JIRAUSER54513 ] |
| Status | In Progress [ 3 ] | In Review [ 10002 ] |
| Link |
This issue relates to |
| Assignee | Debarun Banerjee [ JIRAUSER54513 ] | Marko Mäkelä [ marko ] |
| Status | In Review [ 10002 ] | Stalled [ 10000 ] |
| Link | This issue blocks MDEV-36197 [ MDEV-36197 ] |
| Assignee | Marko Mäkelä [ marko ] | Debarun Banerjee [ JIRAUSER54513 ] |
| Status | Stalled [ 10000 ] | In Review [ 10002 ] |
| Link |
This issue blocks |
| Assignee | Debarun Banerjee [ JIRAUSER54513 ] | Marko Mäkelä [ marko ] |
| Status | In Review [ 10002 ] | Stalled [ 10000 ] |
| Assignee | Marko Mäkelä [ marko ] | Vladislav Vaintroub [ wlad ] |
| Status | Stalled [ 10000 ] | In Review [ 10002 ] |
| Link |
This issue blocks |
| Link |
This issue includes |
| Link |
This issue blocks |
| Fix Version/s | 12.1 [ 29992 ] | |
| Fix Version/s | 12.0 [ 29945 ] |
| Assignee | Vladislav Vaintroub [ wlad ] | Marko Mäkelä [ marko ] |
| Description |
copied from [ * The buf_pool.free as well as the buffer pool blocks that are backing store for the AHI or lock_sys could be doubly linked with each other via bytes allocated within the page frame itself. We do not need a dummy buf_page_t for such blocks. * We could allocate a contiguous virtual address range for the maximum supported size of buffer pool, and let the operating system physically allocate a subset of these addresses. The complicated logic of having multiple buffer pool chunks can be removed. On 32-bit architectures, the maximum size could be about 2GiB. On 64-bit architectures, the virtual address bus often is 48 bits (around 256 TiB). Perhaps we could shift some burden to the user and introduce a startup parameter innodb_buffer_pool_size_max. |
The InnoDB buffer pool had been allocated in multiple chunks, because {{SET GLOBAL innodb_buffer_pool_size}} would extend the buffer pool in chunks. This would lead to many limitations, such as the inability to shrink the buffer pool below {{innodb_buffer_pool_chunk_size}}.
It would be cleaner to allocate a contiguous virtual address range for a maximum supported size of buffer pool (a new parameter {{innodb_buffer_pool_size_max}}, which defaults to the initially specified {{innodb_buffer_pool_size}}) and to allow the {{innodb_buffer_pool_size}} to be changed in increments of 1 megabyte. , and let the operating system physically allocate a subset of these addresses. The complicated logic of having multiple buffer pool chunks can be removed. On 32-bit architectures, the maximum size could be about 2GiB. On 64-bit architectures, the virtual address bus often is 48 bits (around 256 TiB). Perhaps we could shift some burden to the user and introduce a startup parameter innodb_buffer_pool_size_max. |
| Description |
The InnoDB buffer pool had been allocated in multiple chunks, because {{SET GLOBAL innodb_buffer_pool_size}} would extend the buffer pool in chunks. This would lead to many limitations, such as the inability to shrink the buffer pool below {{innodb_buffer_pool_chunk_size}}.
It would be cleaner to allocate a contiguous virtual address range for a maximum supported size of buffer pool (a new parameter {{innodb_buffer_pool_size_max}}, which defaults to the initially specified {{innodb_buffer_pool_size}}) and to allow the {{innodb_buffer_pool_size}} to be changed in increments of 1 megabyte. , and let the operating system physically allocate a subset of these addresses. The complicated logic of having multiple buffer pool chunks can be removed. On 32-bit architectures, the maximum size could be about 2GiB. On 64-bit architectures, the virtual address bus often is 48 bits (around 256 TiB). Perhaps we could shift some burden to the user and introduce a startup parameter innodb_buffer_pool_size_max. |
The InnoDB buffer pool had been allocated in multiple chunks, because {{SET GLOBAL innodb_buffer_pool_size}} would extend the buffer pool in chunks. This would lead to many limitations, such as the inability to shrink the buffer pool below {{innodb_buffer_pool_chunk_size}}.
It would be cleaner to: * allocate a contiguous virtual address range for a maximum supported size of buffer pool (a new parameter {{innodb_buffer_pool_size_max}}, which defaults to the initially specified {{innodb_buffer_pool_size}}) * allow the {{innodb_buffer_pool_size}} to be changed in increments of 1 megabyte * define a fixed mapping between the virtual memory addresses of buffer page descriptors page frames, to fix bugs like * refactor the shrinking of the buffer pool to provide more meaningful progress output and to avoid hangs The complicated logic of having multiple buffer pool chunks can be removed, and the parameter {{innodb_buffer_pool_chunk_size}} will be deprecated and ignored. |
| Fix Version/s | 10.11 [ 27614 ] | |
| Fix Version/s | 11.4 [ 29301 ] | |
| Fix Version/s | 11.8 [ 29921 ] | |
| Fix Version/s | 12.1 [ 29992 ] |
| Status | In Review [ 10002 ] | In Testing [ 10301 ] |
| Assignee | Marko Mäkelä [ marko ] | Matthias Leich [ mleich ] |
| issue.field.resolutiondate | 2025-03-26 15:45:33.0 | 2025-03-26 15:45:32.995 |
| Fix Version/s | 10.11.12 [ 29998 ] | |
| Fix Version/s | 11.4.6 [ 29999 ] | |
| Fix Version/s | 11.8.2 [ 30001 ] | |
| Fix Version/s | 10.11 [ 27614 ] | |
| Fix Version/s | 11.4 [ 29301 ] | |
| Fix Version/s | 11.8 [ 29921 ] | |
| Assignee | Matthias Leich [ mleich ] | Marko Mäkelä [ marko ] |
| Resolution | Fixed [ 1 ] | |
| Status | In Testing [ 10301 ] | Closed [ 6 ] |