[MCOL-4292] columnstore-docker-cluster: ColumnStore engine not accessible during restart of a cluster member in HA mode Created: 2020-09-03  Updated: 2020-09-23  Resolved: 2020-09-07

Status: Closed
Project: MariaDB ColumnStore
Component/s: N/A
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Jens Röwekamp (Inactive) Assignee: Todd Stoffel (Inactive)
Resolution: Not a Bug Votes: 0
Labels: skysql
Environment:

mariadb/skysql-enterprise-columnstore-handoff sky-1.5.4-1 95e34753d40f



 Description   

Don't know if this is a bug or expected behaviour.

Scenario: A multi node ColumnStore cluster with 3 nodes and shared metadata storage (activated HA feature)

In case one cluster member is restarted read and write queries to the ColumnStore engine aren't possible for less than a minute. Once the restarted node is back up again reads and writes are possible again.

How to reproduce:

jens@ubuntu18:~/columnstore-docker-cluster$ docker-compose up -d
Creating network "columnstore-docker-cluster_default" with the default driver
Creating volume "columnstore-docker-cluster_data1" with default driver
Creating volume "columnstore-docker-cluster_data2" with default driver
Creating volume "columnstore-docker-cluster_data3" with default driver
Creating volume "columnstore-docker-cluster_storagemanager" with default driver
Creating mcs3 ... done
Creating mcs2 ... done
Creating mcs1 ... done
Creating mx1  ... done

jens@ubuntu18:~/columnstore-docker-cluster$ docker exec -it mcs1 demo
Waiting for PM1 to be initialized ............... done
Adding PM1 to cluster ... done
Adding PM2 to cluster ... done
Adding PM3 to cluster ... done
Validating ... done
Adding PM1 to MaxScale ... done
Adding PM2 to MaxScale ... done
Adding PM3 to MaxScale ... done
Adding service ... done
Adding listener ... done
Adding monitor ... done

jens@ubuntu18:~/columnstore-docker-cluster$ docker exec -it mcs1 mariadb -e "CREATE DATABASE test; CREATE TABLE test.tmp1 (i int) engine=columnstore; insert into test.tmp1 values (42); select * FROM test.tmp1"
+------+
| i    |
+------+
|   42 |
+------+

jens@ubuntu18:~/columnstore-docker-cluster$ docker restart mcs2; for i in $(seq 1 10); do echo $i; docker exec -it mcs1 mariadb -e "SELECT * FROM test.t
mp1"; sleep 1; done
mcs2
1
ERROR 1815 (HY000) at line 1: Internal error: DistributedEngineComm::write: Broken Pipe error
2
ERROR 1815 (HY000) at line 1: Internal error: DistributedEngineComm::write: Broken Pipe error
3
ERROR 1815 (HY000) at line 1: Internal error: IDB-2039: Data file does not exist, please contact your system administrator for more information.
4
ERROR 1815 (HY000) at line 1: Internal error: IDB-2039: Data file does not exist, please contact your system administrator for more information.
5
ERROR 1815 (HY000) at line 1: Internal error: IDB-2039: Data file does not exist, please contact your system administrator for more information.
6
ERROR 1815 (HY000) at line 1: Internal error: IDB-2045: At least one PrimProc closed the connection unexpectedly.
7
+------+
| i    |
+------+
|   42 |
+------+
8
+------+
| i    |
+------+
|   42 |
+------+
9
+------+
| i    |
+------+
|   42 |
+------+
10
+------+
| i    |
+------+
|   42 |
+------+



 Comments   
Comment by Todd Stoffel (Inactive) [ 2020-09-07 ]

This is expected behavior. Columnstore will temporarily put itself into readonly mode while nodes are reconfigured. An appropriate message is displayed indicating this process.

Generated at Thu Feb 08 02:49:16 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.