mcsadmin reports an Active State w/o raising any Alarms but inserts are rejected with Internal error: CAL0001: dmlprocessor.cpp PackageHandler::run() package type 0 IDB-2043: An internal error occurred. Check the error log file & contact support.

[root@pm1 ~]# mcsadmin getsystemi

getsysteminfo   Tue Jan 15 11:10:01 2019

System columnstore-1

System and Module statuses

Component     Status                       Last Status Change

------------  --------------------------   ------------------------

System        ACTIVE                       Tue Jan 15 11:04:52 2019

Module pm1    ACTIVE                       Tue Jan 15 11:04:45 2019

MariaDB ColumnStore Process statuses

Process             Module    Status            Last Status Change        Process ID

------------------  ------    ---------------   ------------------------  ----------

ProcessMonitor      pm1       ACTIVE            Tue Jan 15 11:04:11 2019       11855

ProcessManager      pm1       ACTIVE            Tue Jan 15 11:04:18 2019       12029

DBRMControllerNode  pm1       ACTIVE            Tue Jan 15 11:04:26 2019       12523

ServerMonitor       pm1       ACTIVE            Tue Jan 15 11:04:29 2019       12570

DBRMWorkerNode      pm1       ACTIVE            Tue Jan 15 11:04:29 2019       12588

PrimProc            pm1       ACTIVE            Tue Jan 15 11:04:32 2019       12665

ExeMgr              pm1       ACTIVE            Tue Jan 15 11:04:36 2019       12754

WriteEngineServer   pm1       ACTIVE            Tue Jan 15 11:04:40 2019       12851

DDLProc             pm1       ACTIVE            Tue Jan 15 11:04:44 2019       12934

DMLProc             pm1       ACTIVE            Tue Jan 15 11:04:52 2019       13007

mysqld              pm1       ACTIVE            Tue Jan 15 11:04:23 2019       12417

Active Alarm Counts: Critical = 0, Major = 0, Minor = 0, Warning = 0, Info = 0

MariaDB [(none)]> insert into mytest.lineitem1 values() ;

ERROR 1815 (HY000): Internal error: CAL0001: dmlprocessor.cpp PackageHandler::run() package type 0 IDB-2043: An internal error occurred.  Check the error log file & contact support.

[root@pm1 ~]# cat /var/log/mariadb/columnstore/err.log

Jan 15 11:06:14 pm1 joblist[12523]: 14.186697 |2147483648|0|0| C 05 CAL0000: st: 0 TupleBPS::sendPrimitiveMessages() caught an exception: IDB-2034: At least one DBRoot required for that query is offline.

Jan 15 11:06:14 pm1 controllernode[12523]: 14.187158 |0|0|0| C 29 CAL0000: IDB-2034: At least one DBRoot required for that query is offline.

Jan 15 11:16:29 pm1 joblist[13007]: 29.427530 |2147483648|0|0| C 05 CAL0000: IDB-2034: At least one DBRoot required for that query is offline.

Jan 15 11:16:29 pm1 joblist[13007]: 29.427624 |2147483648|0|0| C 05 CAL0000: IDB-2034: At least one DBRoot required for that query is offline.

Jan 15 11:16:29 pm1 dmlpackageproc[13007]: 29.428497 |0|0|0| E 21 CAL0001: dmlprocessor.cpp PackageHandler::run() package type 0 IDB-2043: An internal error occurred.  Check the error log file & contact support.

mcsadmin> getModuleTypeConfig

getmoduletypeconfig   Tue Jan 15 11:19:26 2019

Module Type Configuration

ModuleType 'pm' Configuration information

ModuleDesc = Performance Module

RunType = SIMPLEX

ModuleCount = 1

ModuleHostName and ModuleIPAddr for NIC ID 1 on  module 'pm1' = localhost , 127.0.0.1

DBRootIDs assigned to module 'pm1' = 5

ModuleCPUCriticalThreshold % = 0

ModuleCPUMajorThreshold % = 0

ModuleCPUMinorThreshold % = 0

ModuleCPUMinorClearThreshold % = 0

ModuleDiskCriticalThreshold % = 90

ModuleDiskMajorThreshold % = 80

ModuleDiskMinorThreshold % = 70

ModuleMemCriticalThreshold % = 90

ModuleMemMajorThreshold % = 0

ModuleMemMinorThreshold % = 0

ModuleSwapCriticalThreshold % = 90

ModuleSwapMajorThreshold % = 80

ModuleSwapMinorThreshold % = 70

ModuleDiskMonitorFileSystem#1 = /

mcsadmin> getModuleConfig

getmoduleconfig   Tue Jan 15 11:19:58 2019

Module Name Configuration

Module 'pm1' Configuration information

ModuleType = pm

ModuleDesc = Performance Module #1

ModuleIPAdd NIC ID 1 = 127.0.0.1

ModuleHostName NIC ID 1 = localhost

DBRootIDs assigned  = 5