[MCOL-153] Making the max concurrent query parameter configurable Created: 2016-06-20 Updated: 2017-12-01 Resolved: 2016-10-31 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | ExeMgr |
| Affects Version/s: | 1.0.1 |
| Fix Version/s: | Icebox |
| Type: | New Feature | Priority: | Critical |
| Reporter: | Daniel Lee (Inactive) | Assignee: | Daniel Lee (Inactive) |
| Resolution: | Not a Bug | Votes: | 1 |
| Labels: | None | ||
| Description |
|
By the original design, Columnstore allows only 10 concurrent MySQL queries. Additional queries are placed in a queue and to be processed on a first-come-first-served basis. This max queries mechanism was put in place because of memory requirement on the UM(s) for hashing joins. Evidently, it is not good for all hardware configurations, especially for hardware configurations that provide high CPU power and large memory. When disk-join is enabled, this restriction becomes less relevant. I believe this max value of 10 is not configurable in Columnstore. I suggest that we change this hard-coded value to a configurable parameter in Calpont.xml. By default, it should be set to 0 for no limit. It can be set by the user when the throttling is required for the user’s environment. It should still be independent from the disk-join setting. |
| Comments |
| Comment by David Hall (Inactive) [ 2016-10-20 ] |
|
The max concurrent query functionality is maintained in ExeMgr by statementsRunningCount, an instance of ActiveStatementCounter. The maximum is controlled by the (undocumented) ExeMgr1:ExecQueueSize value in Columnstore.xml. The default since 3.5.11 is 20. Other seldom used XML values that may apply are: PrimitiveServers:ServerQueueSize (default 10) which controls how many primitives can be in queue waiting for a thread (available threads are randomly assigned). When exceeded, a primitive is held in abeyance until the queue is no longer full. This allows for semi-serializing. The number of threads available is controlled by PrimitiveServers:ProcessorThreads, which defaults to #cores*2. DBBC:NumThreads (default #cores*2) which controls how many threads work in the data cache. Testing has shown that modifying this is counter-productive. |
| Comment by Daniel Lee (Inactive) [ 2016-10-27 ] |
|
Please confirm if there is already a parameter for controlling max concurrent queries and it is working. Thanks |
| Comment by David Hall (Inactive) [ 2016-10-27 ] |
|
I have verified the functionality of ExeMgr1:ExecQueueSize. I have added a UDF – calgetsqlcount that will show the number of running statements and the number of waiting statements. In conjunction with test002 this can be used to verify ExecQueueSize. Changing ExecQueueSize requires a restart of ExeMgr. |
| Comment by Daniel Lee (Inactive) [ 2016-10-28 ] |
|
So the requested functionality is already in place. Show we mark this ticket invalid? Some how we need to close it. |