Many of objects, such as BatchPrimitiveProcessor, were designed to be destroyed when the thread finished. Later, threadpools were added and these objects are not destroyed, but rather re-used.
These objects appear to allocate a lot of internal structures. When they got put into the pool, no one thought to empty them first, so lots of memory is being consumed from previous queries until a new query comes in that uses those objects. Then those pointers are reset to new values and the memory is finally released.
Add a reset() function to these objects that clears these allocations. Then add a call to reset() just before it gets sent back to the pool.
BatchPrimitiveProcessor is getting this upgrade as part of MCOL-4626, but the code must be scrubbed for other places where this happens.
Generally, this doesn't cause problems except in memory constrained environments. Here, an object may have large amounts of memory tied up for an expired query which may cause memory limits to be exceeded when there could be memory available.
During this effort, any tracked memory as part of TotalUMMemory must also decrement the tracker when released.