Parallel Query Execution

Goal

Add parallel query execution to MariaDB Server so that long-running analytical statements can scale across multiple cores by splitting work across worker threads, with linear-ish speedup proportional to thread count (~70% of the theoretical maximum, per Monty's estimate in MDEV-6096).
In scope: pure {{SELECT}}s on InnoDB tables with one or more tables, including group-by, where the optimizer estimates the serial cost above a threshold and where executing in parallel does not increase user-perceived latency.
Out of scope (for the first ship): write paths, parallel DML, plans involving variable assignment (@a := value), engines that cannot share a read snapshot, queries cheaper than parallel_query_min_cost.

Why an Epic instead of using MDEV-6096

MDEV-6096 has been the de-facto thinking-space for parallel query since 2014. Its title is "Research - Ideas about…" and its description is a design memo, not a feature brief. Treating it as the umbrella confuses "research phase" with "feature shipment". This Epic now owns the feature; MDEV-6096 stays as the research artifact and is one of the Epic's children.

Workstreams

The current implementation is split between two devs. Stories under this Epic group as follows:
Storage-engine plumbing — Oleg Smirnov

MDEV-39489 — study how to use the same read snapshot across workers
MDEV-39491 — split the table/index into chunks to read (storage-engine "position at N%")
MDEV-39652 — query plan shape: parallel table scan + group

Worker-thread plumbing & result shipping — Rex Johnston

MDEV-39492 — how to create worker threads (thread pool? INSERT-DELAYED pattern?)
MDEV-39495 — use temporary work tables to ship results from worker → primary
MDEV-39638 — open per-worker handler/TABLE objects

Pre-existing related stories (older OPEN siblings)

MDEV-5004 — parallel read transactions on the same snapshot (precondition for snapshot work above)
MDEV-18705 — parallel index range scan
MDEV-27717 — parallel execution on partitions where multiple partitions are needed

Research artifact

MDEV-6096 — original 2014 research/ideas ticket

Related but not children

MDEV-26157 — Prototype OpenMP in addressing parallel queries (already an Epic of its own, hence not a child here). Treat as a sibling exploration; revisit if OpenMP becomes a viable substrate.
TODO-5462 — review queue for the older replay MDEVs (separate Epic MDEV-39836); listed here only because some of the same engineers are involved.

Status snapshot (as of creation)

Total logged effort across children to date:

• roughly 40+ engineer-days, mostly on the two STALLED studies (MDEV-39492 ~16d, MDEV-39491 ~13d).

Success criteria

• An analytical query crossing parallel_query_min_cost on InnoDB runs across N worker threads with a measurable speedup vs. serial, at ≥70% of the theoretical N× ceiling on the reference benchmark.
• The feature is gated by server variables (parallel_max_running_threads, parallel_worker_threads, parallel_query_min_cost) — off by default for the first release.
• Worker threads are visible in INFORMATION_SCHEMA.PROCESSLIST and performance_schema.threads.
  KILL of either the primary or any worker terminates the whole parallel query cleanly.
• All existing MTR tests pass with the feature enabled (no regressions to serial plans).
• Documentation page covering tuning variables, when the optimizer chooses a parallel plan, and known limitations.

Non-goals for this Epic

• Cross-engine parallelism (MyISAM/RocksDB) — out of scope; would need separate snapshot-sharing work per engine.
• Spider-like cross-node parallelism — covered separately by Spider/MaxScale work.
• Parallel DML or parallel index build — separate Epics if pursued.

Target release

Fix Version: 13.1. To be re-evaluated mid-Q3/2026 once the two STALLED studies are unblocked.

Tracking

This Epic is the single ticket to follow for "is parallel query done?". When all children are CLOSED, the feature ships. New parallel-query bugs and stories should be labeled parallel-query-2026 and linked here as Epic children.

Phases by version

Phase 0: studies and plumbing (shipped in 13.1 / 13.2)

• MDEV-39491 (split table/index into chunks, study): closed, 13.1.1
• MDEV-39859 (chunked execution in parallel workers): closed, 13.1.2
• MDEV-39489 (same read snapshot across workers, study): closed, 13.2.1
• MDEV-39492 (create worker threads, study), MDEV-39495 (temp work tables for results), MDEV-39638 (open tables for workers): closed, no fix version

Phase 1: parallel join and scans (targeted at 13.3)

• MDEV-40012 (execute the join in worker threads): in progress, 13.3
• MDEV-39652 (parallel table scan + group plan): open, 13.3
• MDEV-40003 (parallel scans on clustered-index ranges) and MDEV-40005 (parallel scans on secondary indexes): closed, 13.3.1
• MDEV-38801 (Item_cache_year shallow_copy), a prerequisite labelled parallel-query-2026, was backported to 10.11.20, 11.4.14, 11.8.10, 12.3.4, 13.1.2, 13.2.1 and 13.3.1.

Phase 1 regressions (open against 13.2, fix in 13.3)

All four are caused by MDEV-40012:

• MDEV-41492: intermittent signal 11 with parallel execution, Critical, in progress
• MDEV-41399: performance degradation on a JOIN query, in progress
• MDEV-41497: UNION query 80% slower
• MDEV-41499: degradation for range predicates returning few rows

Phase 2: remaining scope (14.1)

• MDEV-18705 (parallel index range scan): open, no version
• MDEV-27717 (parallel execution across partitions): open, Critical, ROADMAP
• MDEV-5004 (parallel reads on the same snapshot): open, no version
• MDEV-6096 (research ticket): open, ROADMAP
• MDEV-41292 (stress-test InnoDB part, added 2026-09-30): open, no version

Solution architecture

• Suppose we're computing a two-table query with GROUP BY:

select sum(o_ordertotal)

from

  customer join orders on c_custkey=o_custkey

where

  c_country ='Finland'

group by

  c_custkey;

The join order is customer->orders.

Single-threaded execution would look like this:

Multi-threaded execution would look like this (different colors denote execution by different threads):

• The first table is scanned with a special "parallel scan".
• Other tables are re-opened by the worker threads and we can make lookups there independently on other workers
  • Yes, this will not work for BNL[H] joins or any access methods that have startup steps.
• GROUP BY handling for eligible aggregate functions
  • Each worker will compute partial group.
  • Then, temporary tables are passed over to the coordinator thread
  • Which sums them up to make final aggregates.

Partitioning of table orders follows the way MySQL did it for their COUNT(*) computation and is based on the shape of the BTREE index we're scanning.
The page range to scan is split into page ranges that are given to worker threads.

There is a tradeoff between giving few big page ranges (Plus: no need for coordination, Minus: some ranges can be done faster than others, leaving some workers without work to do) and giving many smaller page ranges (Plus: work is evenly distributed between workers, Minus: there is coordination overhead and double read operations at work piece bounds).