[MCOL-4001] Can't set the value of MaxOutstandingRequests Created: 2020-05-13  Updated: 2023-10-26  Resolved: 2023-10-26

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: 1.2.5
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: ravi dhingra Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

POC


 Description   

Hi,
We are doing a POC and the machines are each 64core (hyperthreaded) with 512G RAM and local SSD.
The test we have is for 500 concurrency and we have 2 PMs (total of 128 cores)
We have tried setting MaxOutstandingRequests parameter and it's just not making any change...
$ configxml.sh setconfig JobList MaxOutstandingRequests 64
Old value of JobList / MaxOutstandingRequests is 20
JobList / MaxOutstandingRequests now set to 64
$ configxml.sh getconfig JobList MaxOutstandingRequests
Current value of JobList / MaxOutstandingRequests is 64

We have tried it with setconfig also
$ setConfig JobList MaxOutstandingRequests 64

$getConfig JobList MaxOutstandingRequests
64

And getconfig and all else reads it right. We have verified in Columnstore.xml and it reflects there too.
But it is not running more than 20 queries...

MariaDB > select calgetsqlcount();
------------------------------------------------------

calgetsqlcount()

------------------------------------------------------

Running SQL statements 20, Waiting SQL statments 230

------------------------------------------------------

We ran 250 concurrent queries per server and 20 ran and 230 waited.
Replication is on and we are sending 250 queries on pm1 and pm2.

mcsadmin> getsystemi
getsysteminfo Tue May 12 17:43:28 2020

System columnstore-1

System and Module statuses

Component Status Last Status Change
------------ -------------------------- ------------------------
System ACTIVE Tue May 12 17:10:06 2020

Module pm1 ACTIVE Tue May 12 17:10:04 2020
Module pm2 ACTIVE Tue May 12 17:10:00 2020

Active Parent OAM Performance Module is 'pm1'
Primary Front-End MariaDB ColumnStore Module is 'pm1'
MariaDB ColumnStore Replication Feature is enabled
MariaDB ColumnStore set for Non-Distributed Install

MariaDB ColumnStore Process statuses

Process Module Status Last Status Change Process ID
------------------ ------ --------------- ------------------------ ----------
ProcessMonitor pm1 ACTIVE Tue May 12 17:09:06 2020 38104
ProcessManager pm1 ACTIVE Tue May 12 17:09:13 2020 38359
DBRMControllerNode pm1 ACTIVE Tue May 12 17:09:41 2020 39478
ServerMonitor pm1 ACTIVE Tue May 12 17:09:43 2020 39530
DBRMWorkerNode pm1 ACTIVE Tue May 12 17:09:43 2020 39584
PrimProc pm1 ACTIVE Tue May 12 17:09:48 2020 39758
ExeMgr pm1 ACTIVE Tue May 12 17:09:53 2020 40471
WriteEngineServer pm1 ACTIVE Tue May 12 17:09:57 2020 40621
DDLProc pm1 ACTIVE Tue May 12 17:10:01 2020 40755
DMLProc pm1 ACTIVE Tue May 12 17:10:05 2020 40906
mysqld pm1 ACTIVE Tue May 12 17:09:55 2020 39330

ProcessMonitor pm2 ACTIVE Tue May 12 17:09:31 2020 19696
ProcessManager pm2 HOT_STANDBY Tue May 12 17:09:36 2020 19916
DBRMControllerNode pm2 COLD_STANDBY Tue May 12 17:09:42 2020
ServerMonitor pm2 ACTIVE Tue May 12 17:09:45 2020 20261
DBRMWorkerNode pm2 ACTIVE Tue May 12 17:09:45 2020 20337
PrimProc pm2 ACTIVE Tue May 12 17:09:50 2020 20358
ExeMgr pm2 ACTIVE Tue May 12 17:09:54 2020 20727
WriteEngineServer pm2 ACTIVE Tue May 12 17:09:58 2020 20796
DDLProc pm2 COLD_STANDBY Tue May 12 17:10:00 2020
DMLProc pm2 COLD_STANDBY Tue May 12 17:10:00 2020
mysqld pm2 ACTIVE Tue May 12 17:09:44 2020 20199

Active Alarm Counts: Critical = 0, Major = 0, Minor = 0, Warning = 0, Info = 0
mcsadmin>



 Comments   
Comment by David Hall (Inactive) [ 2020-05-14 ]

For a large system, increasing JobList / MaxOutstandingRequests may help performance. Some experimentation may show better resource usage.
However, this is not where the number of concurrent queries is set.
use <ExeMgr1>
<ExecQueueSize>nn</ ExecQueueSize> to set the throttling threshold for number of concurrent queries to nn.

Comment by ravi dhingra [ 2020-05-14 ]

Thank you David for a quick response. Really appreciate it. Where can I find a comprehensive documentation for Columnstore.xml parameters and settings?
I'll try this out and will update the ticket.

Comment by David Hall (Inactive) [ 2020-05-14 ]

https://mariadb.com/kb/en/mariadb-columnstore-performance-related-configuration-settings/ can help, but it is by no means comprehensive.

Comment by ravi dhingra [ 2020-05-14 ]

David, that's the reason I requested a comprehensive. Even a dirty document with no editing will work. Any pointers. For us to successfully take this on a production stack, we need to know what all we can tweak. We are expecting very high concurrency and live queries from tableau.

Comment by David Hall (Inactive) [ 2020-05-14 ]

That document can help tune a system. In general those are the only settings we recommend to play with.

Remember that Columnstore is designed to apply as many resources as possible to each query. It doesn't try to tone one down to keep others happy. That's why the default is 20 at a time.

Also be aware of <=> (NULL safe operator) not working correctly for joins. See MCOL-1216. It's possible that tableau uses this. If you need this fixed fast, talk to your rep.

Comment by ravi dhingra [ 2020-05-14 ]

Got it! Thanks David. But again, a complete list and description of Columnstore.xml parameters will be really helpful down the line!

Generated at Thu Feb 08 02:47:02 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.