Self adjusting Columnstore monitor

Currently double book-keeping is required when using the MaxScale Columnstore monitor for monitoring a Columnstore cluster.

Although the Columnstore cluster (obviously) is aware of what nodes are part of the cluster, MaxScale needs to be explicitly told which those nodes/servers are.

This arrangement is

• fragile
• error prone and
• userhostile.

It should be so that you only need to tell MaxScale how to get in contact with the Columnstore cluster, after which everything is automatic. As new nodes are added to the cluster or existing nodes removed, irrespective of whether it is done indirectly via MaxScale or directly using Columnstore, MaxScale (the monitor + all routers using the information provided by the monitor) should adapt and things should just work.

Essentially, the Columnstore monitor should work just the way the Clustrix monitor currently works. In practice this means that

• the servers the Columnstore monitor is configured with are only used by the monitor in order to get in contact the Columnstore cluster (one is sufficient, but more can be provided),
• the Columnstore monitor fetches the Columnstore configuration and creates transient server objects (inside MaxScale) according to the configuration, and
• services do not explicitly refer to servers, but to the Columnstore monitor.

The end-result is that the addition or removal of a node to/from the Columnstore cluster requires no changes whatsoever to the MaxScale configuration.

A sample configuration could look like:

[CS-bootstrap-node-1]

type=server

address=10.10.10.10

port=3306

protocol=mariadbbackend

[CS-bootstrap-node-2]

type=server

address=10.10.10.11

port=3306

protocol=mariadbbackend

This server entries are exactly like all other server entries, but they are only used for bootstrapping the monitor.

[CsMonitor]

type=monitor

module=csmon

version=1.5

servers=CS-bootstrap-node-1, CS-bootstrap-node-2

The servers the monitor is explicitly provided with are only used by the monitor for getting in contact with the Columnstore cluster. One entry is sufficient, but in case that happens to be down at startup, it's advisable to specify additional.

Using that information, the Columnstore monitor connects to the Columnstore cluster and asks for its configuration. Using the configuration the monitor

• stores persistently information about the nodes and their IPs, and
• creates a dynamic server instance for node.

The former, so that the monitor at subsequent starts has connection information for all nodes. That is, the bootstrap servers are essentially only needed for the very first startup when the monitor has no knowledge of the cluster. At subsequent startups the bootstrap information is essentially not needed.

The dynamic server instances will be named like:

@@CsMonitor:node1

@@CsMonitor:node1

The @@ prefix is to ensure that there cannot be a name-clash with a server object manually created by a user. As these server instances are transient to their nature, that is, will disappear if the corresponding node is removed, they should never be explicitly referred to.

Services should neither list the bootstrap servers, nor dynamic ones, but should simply refer to the monitor.

[My-Service]

type=service

cluster=CsMonitor

```

That is, no servers entry, but instead a cluster entry that refers to the monitor.

With this setup, MaxScale + Columnstore will just work without a need for any kind of manual reconfiguration, regardless of the changes made to the Columnstore cluster.