[MCOL-4733] Memory allocations are kept longer than needed Created: 2021-05-24  Updated: 2023-07-25  Resolved: 2023-05-24

Status: Closed
Project: MariaDB ColumnStore
Component/s: DDLProc, DMLProc, ExeMgr, PrimProc
Affects Version/s: 5.5.1
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: David Hall (Inactive) Assignee: alexey vorovich (Inactive)
Resolution: Fixed Votes: 1
Labels: None

Issue Links:
Issue split
split from MCOL-4626 Columnstore cluster becomes non opera... Stalled
Relates
relates to MCOL-4746 Prevent MCS cluster from starting on ... Open
Epic Link: ColumnStore Performance Improvements

 Description   

Many of objects, such as BatchPrimitiveProcessor, were designed to be destroyed when the thread finished. Later, threadpools were added and these objects are not destroyed, but rather re-used.

These objects appear to allocate a lot of internal structures. When they got put into the pool, no one thought to empty them first, so lots of memory is being consumed from previous queries until a new query comes in that uses those objects. Then those pointers are reset to new values and the memory is finally released.

Add a reset() function to these objects that clears these allocations. Then add a call to reset() just before it gets sent back to the pool.

BatchPrimitiveProcessor is getting this upgrade as part of MCOL-4626, but the code must be scrubbed for other places where this happens.

Generally, this doesn't cause problems except in memory constrained environments. Here, an object may have large amounts of memory tied up for an expired query which may cause memory limits to be exceeded when there could be memory available.

During this effort, any tracked memory as part of TotalUMMemory must also decrement the tracker when released.



 Comments   
Comment by Roman [ 2023-05-24 ]

Closed in favor of MCOL-5499.

Generated at Thu Feb 08 02:52:34 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.