[MCOL-4770] Distribute queries processing b/w ExeMgrs/primprocs in the cluster. Created: 2021-06-22  Updated: 2022-10-26  Resolved: 2022-08-19

Status: Closed
Project: MariaDB ColumnStore
Component/s: cmapi
Affects Version/s: 6.1.1
Fix Version/s: cmapi-22.08.1

Type: Bug Priority: Major
Reporter: Roman Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
causes MCOL-4860 Disable multiple ExeMgrs in the cluster Closed
Relates
relates to MCOL-4593 Multiple concurrent queries with aggr... Stalled
Sprint: 2021-9

 Description   

Despite the fact CMAPI starts ExeMgr on every cluster node the Columnstore.xml contains a description of a single ExeMgr that resides at the primary only.
The preliminary testing in AWS demonstrates 20% performance improvement if there workload is distributed across multiple EMs so CMAPI now adds a new ExeMgr section for every node added into the cluster and removes it when the node is deleted.

 Comments   
Comment by Roman [ 2021-06-28 ]

4QA Previously any MDB in the cluster uses a single EM that resides at the primary. With this patch the queries issued at the node using a local MDB goes through the local EM only. One can monitor cpu utilization using top at non primary nodes running queries using the node's MDB.

Comment by Roman [ 2021-06-30 ]

With this patch the queries issued at the node are processed with the local EM only.
Talking about the schema replication you missed the point that MaxScale takes care about schema replication in normal installation.

Comment by Daniel Lee (Inactive) [ 2021-07-01 ]

Build tested: 6.1.1 ( Drone #2635 )

Executing a query on any node uses the ExeMgr on the same node and covers data for the entire database (not just data on the local node, as "local query" used to do)
Columnstore.xml updated fine when removing and adding nodes.

Comment by Daniel Lee (Inactive) [ 2022-08-19 ]

Build verified: 22.08-1 (#5324), cmapi-22.08-1 (#743)

Verified installation and cmapi operations:

Using this build of the cmapi, installed 5.6.1-1, 6.4.4-2, and 22.08-1 on VMs
installed 6.4.4-2, and 22.08-1 on docker containers.

Generated at Thu Feb 08 02:52:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.