Prototype OpenMP in addressing parallel queries and other operations in code
(MDEV-26157)
|
|
| Status: | Needs Feedback |
| Project: | MariaDB Server |
| Component/s: | Storage Engine - InnoDB |
| Affects Version/s: | 10.7 |
| Fix Version/s: | None |
| Type: | Technical task | Priority: | Major |
| Reporter: | Daniel Black | Assignee: | Daniel Black |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Description |
|
Largely a proof of concept. Tencent Cloud has mentioned in their talks that they do it. I do remember starting mariadb on a multi TB RAM machine that I had brief access to an it consumed 10s of minutes to initialize the buffer pool, so its not totally unworthy task. |
| Comments |
| Comment by Daniel Black [ 2021-07-20 ] | ||||
|
https://github.com/MariaDB/server/commit/359a6e6405cf720c0e24e579f74f50d3d7d5f5ae is the POC. No measurable gain in the log file on 1 10G pool/1M chunksize.
However I'll test this again on a bigger machine. | ||||
| Comment by Marko Mäkelä [ 2021-07-20 ] | ||||
|
It could be that the removal of buf_block_t::mutex in Is this work motivated by or related to PR#1548? | ||||
| Comment by Daniel Black [ 2021-07-20 ] | ||||
|
it was mainly a place to attempt a POC of doing something in parallel using omp that wasn't complicated, very little code changes, and would fail impressively if I got it wrong (in CI). The Tencent folks mentioning it made me think of it. There wasn't any real thought as to intermediate changes that improved this already. But yes, doing something lazy like MDEV-25340 might be something better to benchmark for a real change | ||||
| Comment by Marko Mäkelä [ 2023-08-21 ] | ||||
|
The buffer pool initialization in MariaDB Server 10.6 is already pretty fast according to tests that I conducted: less than 1 second to start up, bootstrap and shut down a server with innodb_buffer_pool_size=96g. Would this contribution be made redundant by MDEV-29445 (removing the current-form buf_pool.chunks) and MDEV-25340 (lazy initialization of the buf_pool.free list)? |