Client facing error

SQLSTATE[HY000]: General error: 1815 Internal error: IDB-2039: Data file does not exist, please contact your system administrator for more information. 

/var/log/messages from PM2 (complaining about a file in data3)

Jan  4 08:28:57 ip-10-111-0-55 env: PrimProc[43018]: 57.151838 |0|0|0| C 28 CAL0053: PrimProc could not open file for OID 571515; /var/lib/columnstore/data3/000.dir/008.dir/184.dir/123.dir/000.dir/FILE000.cdf:No such file or directory

Solution: deconstruct the cluster and readd the nodes 1 by 1 in desired order
https://mariadbcorp.atlassian.net/wiki/spaces/Support/pages/1551925327/Change+Cluster+Master+and+or+Change+DBroot+Order

Error message missleading to the user, says it cant find a file ( in the bucket) but really its the .meta file it cant find

While Broken, maxsacle showed the master orphaned with mcs3 as master, once we fixed the dbroot assignment maxscale list servers fixed itself