[MCOL-4679] Simplify configuration of EM to PM connections Created: 2021-04-16  Updated: 2021-07-08  Resolved: 2021-07-02

Status: Closed
Project: MariaDB ColumnStore
Component/s: ExeMgr
Affects Version/s: None
Fix Version/s: 6.1.1

Type: New Feature Priority: Minor
Reporter: Roman Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MCOL-4593 Multiple concurrent queries with aggr... Stalled
Sprint: 2021-8, 2021-9

 Description   

To describe a connection from UM to PM one needs to add a section into Columnstore.xml like this for every connection

<PMS1>
 
       <IPAddr>127.0.0.1</IPAddr>
 
       <Port>8620</Port>
 
</PMS1>

If the number of connections is significant it might be tedious to describe them and wasteful to store them in the config.

The suggested way to describe now is to add a single section per PM exists in the cluster and set PrimitiveServers.ConnectionsPerPrimProc to describe how many PM-EM connections must be created by DEC::Setup() method.

It is worth to note the issue will also cover CMAPI part that manipulates the XML configuration adding or removing a node from the cluster.

 Comments   
Comment by Roman [ 2021-04-16 ]

Plz review.

Comment by Roman [ 2021-04-19 ]

Now. We need to address CMAPI part.

Comment by Roman [ 2021-05-26 ]

CMAPI when adds a node creates additional PMSX sections in Columnstore.xml where X = number of nodes * PrimitiveServers.ConnectionsPerPrimProc parameter value from Columnstore.xml. leonid.fedorov plz remove the multiplication so that there will be only N PMSX sections in Columnstore.xml where N = number of nodes.

Comment by Roman [ 2021-06-18 ]

gdorman Yes it is.

Comment by Roman [ 2021-06-18 ]

Plz review.

Comment by Roman [ 2021-06-25 ]

4QA One needs to have a single PMS section in Columnstore.xml per PM and set PrimitiveServers.ConnectionsPerPrimProc to whatever number of connections needed. EM automatically create PrimitiveServers.ConnectionsPerPrimProc connections in its startup.
Start MCS count the output lines for ss -tenp | grep PrimProc. Set PrimitiveServers.ConnectionsPerPrimProc to 10, restart MCS and count lines for ss -tenp | grep PrimProc again.

Comment by Daniel Lee (Inactive) [ 2021-06-28 ]

Build tested: 6.1.1 ( Drone #2640), CMAPI ( Drone #502 )

CMAPI DELETE is not update PMS<x> in Columnstore.xml correctly

after installation

	<PMS1>
 
        <IPAddr>s1pm1</IPAddr>
 
        <Port>8620</Port>
 
    </PMS1>
 
    <PMS2>
 
        <IPAddr>s1pm2</IPAddr>
 
        <Port>8620</Port>
 
    </PMS2>
 
    <PMS3>
 
        <IPAddr>s1pm3</IPAddr>
 
        <Port>8620</Port>
 
    </PMS3>

After deleting node s1pm3
There should be 1 entry per PM, a total of 2
But there are 2 entries per PM, a total of 4

    <PMS1>
 
        <IPAddr>s1pm1</IPAddr>
 
        <Port>8620</Port>
 
    </PMS1>
 
    <PMS2>
 
        <IPAddr>s1pm2</IPAddr>
 
        <Port>8620</Port>
 
    </PMS2>
 
    <PMS3>
 
        <IPAddr>s1pm1</IPAddr>
 
        <Port>8620</Port>
 
    </PMS3>
 
    <PMS4>
 
        <IPAddr>s1pm2</IPAddr>
 
        <Port>8620</Port>
 
    </PMS4>

After putting back node s1pm3

    <PMS1>
 
        <IPAddr>s1pm1</IPAddr>
 
        <Port>8620</Port>
 
    </PMS1>
 
    <PMS2>
 
        <IPAddr>s1pm2</IPAddr>
 
        <Port>8620</Port>
 
    </PMS2>
 
    <PMS3>
 
        <IPAddr>s1pm3</IPAddr>
 
        <Port>8620</Port>
 
    </PMS3>

Comment by Daniel Lee (Inactive) [ 2021-06-28 ]

Build tested: 6.1.1 ( Drone #2640), CMAPI ( Drone #502 )

Number of connections don't seem to be correspond to what's configured.
Used "ss -tenp |grep PrimProc"

After installation
ConnectionsPerPrimProc=2
pm1=7
pm2=10
pm3=7

After change
ConnectionsPerPrimProc=5
pm1=7
pm2=21
pm3=7

Comment by Roman [ 2021-06-29 ]

dleeyh The command 'ss -tenp |grep PrimProc' returns all connections that has a line PrimProc so there are additional connections at PP2. If you checked its output right after adding a node it is clear that some of EM to PP connections were not established and you need to run any query that re-establish all connections that EM couldn't create on its startup.

Comment by Roman [ 2021-07-02 ]

The issue with a number of PMS sections on node removal had been fixed. The connections number factor had been explained earlier.

Comment by Daniel Lee (Inactive) [ 2021-07-02 ]

Build verified: 6.1.1 ( Drone #2687), CMAPI ( Drone #505 )

Generated at Thu Feb 08 02:52:10 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.