RSS Memory Growth During Long-Duration Stored Procedure Retention Test

Summary

The process under test shows a consistent and linear increase in RSS memory during a long-duration stored procedure workload. The growth rate is approximately 800 to 900 MB per hour, with no corresponding increase in working set or workload complexity. This behavior suggests allocator retention, fragmentation, or a subsystem failing to return memory to the OS.

Note: MariaDB cache setting = 30GB, total host RAM 60GB.

Background and Motivation

This test was performed because prior research identified memory retention issues in stored procedure workloads in other MySQL variants. Percona documented this behavior in detail in the following article:

https://www.percona.com/blog/stored-procedures-memory-consumption-in-percona-server-for-mysql/

Their investigation showed that repeated stored procedure and cursor operations could cause sustained RSS growth over time. Percona tracked the issue under PS-11472:

https://perconadev.atlassian.net/browse/PS-11472

To determine whether similar behavior exists in our implementation, we reproduced the same class of workload using HammerDB TPROC-C with stored procedures enabled. The goal was to validate whether our allocator correctly recycles memory or whether it exhibits the same retention pattern described in the Percona analysis.

Test Configuration

The workload was executed using HammerDB TPROC-C with the following configuration:

• 200 warehouses
• 36 virtual users
• Stored procedures enabled
• Threadpool enabled (TP)
• REPEATABLE-READ isolation level (RR)
• Long-duration execution to expose allocator behavior
• Stable schema, stable warehouse count, stable VU count
• No external load, no schema changes, no workload variation

This configuration ensures that memory demand should stabilize after warmup. Any monotonic RSS growth is therefore allocator-driven, not workload-driven.

Problem Description

During the multi-hour retention run, RSS grows from approximately 8.9 GB to approximately 31.4 GB. The workload is stable and repeatable, and the growth pattern is monotonic. No spikes or sawtooth patterns are present. VSZ remains stable. The allocator is expected to recycle memory aggressively under this workload, but RSS continues to expand.

Observed Behavior

• RSS increases at a near-linear rate.
• No corresponding increase in workload complexity or memory demand.
• No drop phases indicating allocator release.
• No OS-level memory pressure events.
• No change in process behavior or query mix.
• Growth pattern matches the retention behavior described in PS-11472.

Expected Behavior

Allocator should stabilize RSS after initial warmup. Memory should be recycled internally and returned to the OS when unused. RSS should plateau, not grow indefinitely.

*Impact
*
Long-duration runs will consume excessive memory, eventually leading to OOM conditions, degraded performance, or system-level throttling. This prevents reliable multi-hour or overnight testing and invalidates retention benchmarks.

Additional Notes

The growth pattern strongly suggests:

• Memory fragmentation inside the allocator, or
• A subsystem leaking memory slowly over time, or
• A failure to return freed memory to the OS.

Starting RSS logging for PID 1862321
Timestamp, RSS_MB
2026-08-10 23:09:57, 8935
2026-08-10 23:19:57, 9842
2026-08-10 23:29:57, 10944
2026-08-10 23:39:57, 11824
2026-08-10 23:49:57, 12812
2026-08-10 23:59:57, 13811
2026-08-11 00:09:57, 14856
2026-08-11 00:19:57, 15932
2026-08-11 00:29:57, 16991
2026-08-11 00:39:57, 17974
2026-08-11 00:49:57, 18931
2026-08-11 00:59:57, 19869
2026-08-11 01:09:57, 20801
2026-08-11 01:19:57, 21746
2026-08-11 01:29:57, 22679
2026-08-11 01:39:57, 23613
2026-08-11 01:49:57, 24539
2026-08-11 01:59:57, 25482
2026-08-11 02:09:57, 26416
2026-08-11 02:19:57, 27235
2026-08-11 02:29:57, 28043
2026-08-11 02:39:57, 28741
2026-08-11 02:49:57, 29425
2026-08-11 02:59:57, 30110
2026-08-11 03:09:57, 30708
2026-08-11 03:19:57, 31275
2026-08-11 03:29:57, 31401
2026-08-11 03:39:57, 31402
2026-08-11 03:49:57, 31402