[MCOL-5043] Reduce a number of pre-spawned ExeMgr threads Created: 2022-04-06 Updated: 2023-12-21 |
|
| Status: | Stalled |
| Project: | MariaDB ColumnStore |
| Component/s: | ExeMgr, PrimProc |
| Affects Version/s: | 6.2.3 |
| Fix Version/s: | 23.10 |
| Type: | New Feature | Priority: | Major |
| Reporter: | Roman | Assignee: | Roman |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | rm_perf | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Sprint: | 2021-17, 2022-22, 2022-23, 2023-4, 2023-5, 2023-6, 2023-7, 2023-8, 2023-10, 2023-11, 2023-12 | ||||||||||||||||||||||||
| Description |
|
MCS spawns lots of idle thread pool jobs for parallel query execution, e.g. every 2nd phase of a parallel 2-step aggregation spawns 24 threads and parallel sorting spawns 16 threads by default. The pool threads are just idle until data starts to flow from the lower parts of the executed query. Every thread uses sync primitives, e.g. mutex-es or cond_variable. When multiple queries are processed by an MCS cluster the concurrency sync primitives overhead is enormous and can reach 25% of non-virtualized CPU horsepower. |