Details
-
Bug
-
Status: Closed (View Workflow)
-
Blocker
-
Resolution: Fixed
-
None
-
EC2 AWS 3 node cluster
-
2022-22
Description
In the latest version cs 22.08.x with cmpai 22.08.x
When losing a pm, selects continue fine on two nodes but the cluster doesnt shuffle to a 2 node cluster. Create table statments do not work during this lost pm timeframe.
Furthermore, when the 3rd PM does come back online, no selects work as the 3rd node is still part of the cluster but no subprocesses are online. until a manual mcsShutdown/mcsStart occurs.
steps to reproduce
Used ansible to setup 3 node nfs cluste with CS 22.08.X and cmapi 22.08.x |
|
|
create database test;
|
use test;
|
create table t1 ( a int) engine=columnstore; |
insert into t1 values(1); |
insert into t1 values(2); |
insert into t1 values(3); |
select * from t1;
|
exit
|
|
|
mcsStatus
|
|
|
# now shutdown node 3 |
|
|
mcsStatus # notice it doesnt work
|
{
|
"error": "Got an error retrieving status from node ip-172-31-27-124.us-west-2.compute.internal" |
}
|
|
|
mariadb test -e "select * from t1;"; # notice it works |
|
|
mariadb test -e "create table t2 ( a int) engine=columnstore;" # notice it doesnt work |
|
|
|
|
# start up node 3 node |
|
|
mcsStatus
|
|
|
# notice cs subprocesses are offline on node 3 |
|
|
mariadb test -e "select * from t1;"; # notice it fails and errors |
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Labels | triage |
| Priority | Major [ 3 ] | Critical [ 2 ] |
| Rank | Ranked higher |
| Fix Version/s | Icebox [ 22302 ] |
| Assignee | Alan Mologorsky [ JIRAUSER49150 ] |
| Rank | Ranked lower |
| Priority | Critical [ 2 ] | Blocker [ 1 ] |
| Rank | Ranked higher |
| Sprint | 2022-22 [ 672 ] |
| Rank | Ranked lower |
| Fix Version/s | 22.11.01 [ 28458 ] | |
| Fix Version/s | Icebox [ 22302 ] |
| Status | Open [ 1 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | In Review [ 10002 ] |
| Summary | HA Failing when losing a node | HA Failing when losing a node - restart secondary - pm3 |
| Fix Version/s | cmapi-22.08.2 [ 28534 ] | |
| Fix Version/s | 22.11.1 [ 28458 ] |
| Assigned for Testing | Todd Stoffel [ toddstoffel ] |
| Component/s | cmapi [ 16117 ] | |
| Resolution | Fixed [ 1 ] | |
| Status | In Review [ 10002 ] | Closed [ 6 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 36763 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 36856 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 36891 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 36921 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 36943 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 37003 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 37029 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 37059 ] |
| Remote Link | This issue links to "Page (MariaDB Confluence)" [ 37090 ] |