[MCOL-559] Uncontrolled ExeMgr memory utilization (other than hash-join memory) ? Created: 2017-02-09  Updated: 2019-07-10  Resolved: 2019-07-10

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: None
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: Daniel Lee (Inactive) Assignee: Andrew Hutchings (Inactive)
Resolution: Duplicate Votes: 0
Labels: None

Issue Links:
Relates
relates to MCOL-558 Possible memory leak in ExeMgr Closed
Sprint: 2017-15

 Description   

Build tested: 1.0.7-1

This issue is related to the 1TB test and issue ported in MCOL-558. Please refer to that ticket for additional details.

With totalUMMemory se at 50%, I noticed that ExeMgr's memory utilization went up to 70% before ColumnStore self-restarted due to swapping issue. Since I did not get the hash join memory exhausted error so I assume that we had not hit that 50% yet. That means ExeMgr used over 20% for processing other than hash operations. If the system did not get restarted, I don't know how high this memory utilization will go.

As far as I know, there is no parameter for controlling this memory utilization.

Is this memory utilization strictly based on dataset size?
Is there any mechanism built-in to cap/control ever increasing memory usage?

For this test, disk based join was not enabled. If you are think about that, please check the another ticket that I will be writing right after this. I will post the ticket number in a follow up comment.



 Comments   
Comment by Daniel Lee (Inactive) [ 2017-02-09 ]

The ticket regarding disk based join is MCOL-560.

Comment by Andrew Hutchings (Inactive) [ 2017-07-27 ]

As with MCOL-558, please try with 1.1.0. I suspect this is fixed now.

Comment by David Thompson (Inactive) [ 2017-07-31 ]

Moving into sprint for retest

Comment by Daniel Lee (Inactive) [ 2017-08-02 ]

Performed another test on 1.1.0 branch and test results has been posted in MCOL-558.

Comment by Andrew Hutchings (Inactive) [ 2017-08-02 ]

So this problem is now gone and MCOL-558 remains?

Comment by Andrew Hutchings (Inactive) [ 2017-08-11 ]

I have not been able to reproduce this, just MCOL-558. I suspect this would have been a large result set buffering ready for transmission to mysqld.

Comment by Andrew Hutchings (Inactive) [ 2019-07-10 ]

Handling elsewhere

Generated at Thu Feb 08 02:21:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.