[MCOL-3585] update some defaults in config file Created: 2019-11-04  Updated: 2019-12-05  Resolved: 2019-11-22

Status: Closed
Project: MariaDB ColumnStore
Component/s: ExeMgr
Affects Version/s: 1.4.0
Fix Version/s: 1.4.1

Type: Task Priority: Major
Reporter: Patrick LeBlanc (Inactive) Assignee: Patrick LeBlanc (Inactive)
Resolution: Done Votes: 0
Labels: None

Sprint: 2019-06

 Description   

Some of the defaults are more appropriate to the machines ~ 10-15 years ago. Time to update...

1) PM join threshold. With ssds it makes sense to take some mem from cache and use it for larger pm joins. I would suggest a min of 1GB. The key is named PmMaxMemorySmallSide.
2) Number of aggregation threads & buckets. For some reason this is 4. Would be good to automate this setting, but if time is short, let's just set them higher. As a rough guideline, make the # of buckets 4 X num threads. Keys of interest are RowAggrThreads and RowAggrBuckets.
3) degree of parallelism. This would also be a good candidate for automation. MaxOutstandingRequests is how many extents worth of jobs can be in flight at once. With higher core count, bigger mem, and potential for scale-out, should probably be increased. Automation would be a function of core count across all PMs, ProcessorThreadsPerScan, etc.

There also isn't much reason to have large buffers between jobsteps, so I would suggest trimming FifoSizeLargeSide, and FifoSize. Benchmark a little to confirm.



 Comments   
Comment by Jose Rojas (Inactive) [ 2019-11-21 ]

Updated PmMaxMemorySmall, FifoSizeLargeSize, and FifoSize settings from config file.

RowAggrThreads and RowAggrBuckets are already being automated. I updated automation. Now automating MaxOutstandingRequests as well.

Decreasing FifoSize and FifoSizeLargeSide did not cause any performance hit.

Comment by Patrick LeBlanc (Inactive) [ 2019-11-22 ]

Merged it

Generated at Thu Feb 08 02:43:49 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.