Details
-
Bug
-
Status: Open (View Workflow)
-
Critical
-
Resolution: Unresolved
-
23.02.25
-
None
-
2026-12
Description
Problem
PrimProc gets jemalloc only via LD_PRELOAD in the mcs-primproc systemd unit. If ldconfig can't find libjemalloc, the unit logs "Please install jemalloc to avoid ColumnStore performance degradation and unexpected service interruptions." and starts PrimProc on glibc malloc anyway.
The RPM doesn't guarantee jemalloc is installed:
- Server-side storage/columnstore/CMakeLists.txt declares it as Recommends: (weak dependency) since 979af00b7c0 (2020-08-04).
- 23.02 engine cmake/cpack_manage.cmake (82b9046b1) adds Requires: snappy jemalloc procps-ng gawk, but that doesn't reach the built package. Verified on the signed ES RPM MariaDB-columnstore-engine-10.6.15_10_23.02.25-1.el9: rpm -qR MariaDB-columnstore-engine | grep jemalloc returns nothing.
- Community MariaDB-columnstore-engine-11.8.9_25.10.6-1.el9 has Recommends: jemalloc, no Requires.
- cmake/cpackEngineRPM.cmake also lists jemalloc but isn't included anywhere.
A weak dependency isn't installed when install_weak_deps=False, and dnf can remove it when nothing hard-requires it.
Impact (see MCOL-6694)
3-node cluster with the identical 23.02.25 RPM on all nodes. On the one node where jemalloc had gone missing during an upgrade/downgrade:
- Parallel wide GROUP BY rollups pushed PrimProc RSS past the 95% watchdog before disk-based aggregation spilled.
- Result: CAL0044 14x, OOM kills 12x, 2 core dumps within hours.
The other two nodes, which have jemalloc, ran the same workload without crashing. Installing jemalloc on the bad node and restarting stopped the OOMs.
Expected
- A ColumnStore installation can't end up without jemalloc: a hard Requires: jemalloc (RPM) / Depends: (DEB) in the shipped engine package for 23.02 and 25.10, community and enterprise. This means revisiting the 2020 decision in 979af00b7c0 with the server packaging owners.
- Fix or remove the 23.02 cpack_manage.cmake dependency logic so the declared dependencies actually reach the package, and drop the dead cpackEngineRPM.cmake.
- (Under discussion with the team, not yet agreed.) Running without jemalloc should be visible, not just one journal line. For example, CMAPI / mcs cluster status reports it, or PrimProc refuses to start unless an explicit override is set.
How to check an installed system
rpm -qR MariaDB-columnstore-engine | grep jemalloc
|
grep -c jemalloc /proc/$(pidof PrimProc)/maps # should be > 0
|
Attachments
Issue Links
- relates to
-
MCOL-6694 PrimProc was OOM-killed without spill despite DiskBasedAggregation=Y
-
- Open
-