System status stayed as "BUSY_INIT" after failover process has been completed

Build tested: 1.1.6-1 source

/root/columnstore/mariadb-columnstore-server
commit 513775738f72ec990d055a5d47e2511e3c0e34dd
Merge: 3c37210 9236098
Author: Andrew Hutchings <andrew@linuxjedi.co.uk>
Date: Wed Jul 18 09:37:17 2018 +0100

Merge pull request #123 from drrtuy/MCOL-970

MCOL-970 Slow query log now contains original query even in vtable mode

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 150171b714c16bd91ef620ea483f6200ad775038
Merge: 1068679 8a42949
Author: benthompson15 <ben.thompson@mariadb.com>
Date: Thu Aug 9 17:49:01 2018 -0500

Merge pull request #535 from mariadb-corporation/MCOL-1605

MCOL-1605 - changed error to debug, alarms trying to get issued befor…

debug.log from PM1 has been attached.

Environment: 1um4pm with gluster

The issue was encountered while testing MCOL-1610. I simply suspended the VM for PM2. Failover took place, PM2 was in AUTO_OFFLINE state. The system status never changed back to ACTIVE from BUSY_INIT. I was able to run queries at during this time.

mcsadmin> getsystemstatus
getsystemstatus Mon Aug 13 19:38:37 2018

System vagrantTestStack

System and Module statuses

Component Status Last Status Change
------------ -------------------------- ------------------------
System BUSY_INIT Mon Aug 13 19:28:00 2018

Module um1 ACTIVE Mon Aug 13 17:12:05 2018
Module pm1 ACTIVE Mon Aug 13 17:11:36 2018
Module pm2 AUTO_DISABLED/DEGRADED Mon Aug 13 19:28:05 2018
Module pm3 ACTIVE Mon Aug 13 19:29:22 2018
Module pm4 ACTIVE Mon Aug 13 19:29:23 2018

Active Parent OAM Performance Module is 'pm1'
mcsadmin> getstorage
getstorageconfig Mon Aug 13 19:38:50 2018

System Storage Configuration

Performance Module (DBRoot) Storage Type = DataRedundancy
System Assigned DBRoot Count = 4
DBRoot IDs assigned to 'pm1' = 1
DBRoot IDs assigned to 'pm2' =
DBRoot IDs assigned to 'pm3' = 2, 3
DBRoot IDs assigned to 'pm4' = 4

Data Redundant Configuration

Copies Per DBroot = 3
DBRoot #1 has copies on PMs = 1 2 3
DBRoot #2 has copies on PMs = 2 3 4
DBRoot #3 has copies on PMs = 1 3 4
DBRoot #4 has copies on PMs = 1 2 4