[MDEV-26157] Prototype OpenMP in addressing parallel queries and other operations in code Created: 2021-07-16 Updated: 2023-03-09 |
|
| Status: | Open |
| Project: | MariaDB Server |
| Component/s: | Optimizer |
| Fix Version/s: | None |
| Type: | Epic | Priority: | Major |
| Reporter: | Daniel Black | Assignee: | Daniel Black |
| Resolution: | Unresolved | Votes: | 1 |
| Labels: | foundation | ||
| Issue Links: |
|
|||||||||||||||||||||||||
| Sub-Tasks: |
|
|||||||||||||||||||||||||
| Epic Name: | Prototype OpenMP | |||||||||||||||||||||||||
| Description |
|
There is a strong and persistent demand for queries to use more than a single core in various MariaDB Foundation ServerFest presentations by users and in other forums. The ideal that autoparallelism will magicly appear when required as so far proved to be false. An investigation of potential solutions arrives at OpenMP as one of the better options due the the following attributes: OpenMP supports virtually all compilers including:
These compilers are readily available of supported Operating Systems. Being part of the compiler and the compiler runtime its not a significant dependency addition. Even the 8+ year old OpenMP-4.0 specification supports a significant set of directives that will aid in the prototyping. The introduction of parallel workloads can be done using largely #pragma omp directives. A lightweight standardised runtime component in #include <omp.h> provides access to lower level constructs. The richness of these #pragma directives allows the introduction of parallel programming without any significant changes to the existing code supporting an easy rebase if proved to be mature. OpenMP implement operate a threadpooling based approach consistent to the existing server codebase. Postgres has lead the way in what can be achieved with parallel queries. OpenMP has NUMA aware directives if required. A significant number of training resources and commercially available training are available for OpenMP. Using a number of scenarios (in subtasks) starting with stand alone operations like filesort we'll prototype and benchmark what is required to achieve parallel operations, the level of portability and what performance gains, can be achieved. |