Details
-
New Feature
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
None
Description
systemd file descriptor store + memfd creates an opportunity for the entire innodb buffer pool to remain intack between restarts.
The recovery of these pages would be much faster than full log recovery.
Use cases are:
- changing fixed read-only system variables/option
- offline for maintenance
- minor upgrades
- OOM killed process
- fatal assertion/crash
The caveats around getting the mmap of a buffer pool back are:
- This is just a memory, there's no mutex state accounted for, clean up is required
- need to account for changed options on buffer pool size
- need some compatibility marker that the memory is good (between upgrades of differing MariaDB versions)
notes from discussion:
- Some extra care would be needed to allow upgrades. We’d need some data structure version identifier, so that if an incompatible build of the server is starting up, it’d reinitialize the buffer pool.
- We do have the block descriptors in the same contiguous virtual address range, and there is a fixed mapping between block descriptors and page frames. We’d only have to traverse the memory and populate buf_pool.page_hash based on it. Log-based crash recovery (including a full scan of the log) is actually necessary if there exist any page that was being modified by an incomplete mini-transaction and had already been modified since the latest checkpoint but not written back to the file system. We might be able to consult the buf_page_t::oldest_modification field to determine this.
- mem resize plausible. Either: the state of the resize is discoverable, potentially with local variables in the same memory location; or just unstore the fd, and re-store post resize.
- mysys/my_virtual_mem.c - just add the fd argument to mmap.
- we’d have to determine also the size of the old buffer pool, in case the new server is being started with a smaller innodb_buffer_pool_size_max. Some ftruncate of the memfd might be needed as well.
Attachments
Issue Links
- relates to
-
MDEV-29445 reorganise innodb buffer pool (and remove buffer pool chunks)
-
- Closed
-