[MXS-4787] Benchmark overhead of session_trace Created: 2023-09-29  Updated: 2023-10-17  Resolved: 2023-10-16

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: None
Fix Version/s: 24.02.0

Type: Task Priority: Major
Reporter: markus makela Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: None

Sprint: MXS-SPRINT-192

 Description   

Knowing how much CPU and memory overhead there is to enabling session_trace would make it easier to evaluate how reasonable it would be to automatically turn it on when unexpected situations occur that could be caused due to a bug in MaxScale.

With the recent addition of session_trace_match, this becomes even more important as it is the only way to debug some problems in production environments where the overhead of enabling log_info is unacceptable. Knowing in advance how much this tracing costs would also inform us of how to possibly improve it so that an in-memory log could always be on by default.



 Comments   
Comment by markus makela [ 2023-10-09 ]

On a single-threaded MaxScale (threads=1) the overhead appears to be around 63% of the baseline QPS on a CPU-bound workload (sysbench read-only, 128 threads, UNIX-domain sockets). This does not take into account the global mutex that locatime_r has so with more threads the cost will be greater due to the serialization that each log message causes.

With some optimizations, the performance could be improved to about 84% of baseline QPS while also removing the global mutex in localtime_r.

Version QPS Relative Performance
No tracing 41846 100%
Baseline 26152 63%
Optimized 34775 84%

The remaining overhead appears to be the copying that is done to both generate the messages and to store them in memory for the session tracing.

Generated at Thu Feb 08 04:31:09 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.