CAL0055: ERROR: ExeMgr has caught an exception. Resource temporarily unavailable

prod.ERROR: SQLSTATE[HY000]: General error: 1815 Internal error: IDB-2004: Cannot connect to ExeMgr.

Jul 22 12:46:26 ip-10-111-0-54 joblist[9737]: 26.282984 |198936|0|0| C 05 CAL0000: TupleAggregateStep::doThreadedAggregate() caught Throw location unknown (consider using BOOST_THROW_EXCEPTION)#012Dynamic exception type: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::thread_resource_error> >#012std::exception::what: boost::thread_resource_error: Resource temporarily unavailable

Jul 22 12:46:26 ip-10-111-0-54 joblist[9737]: 26.324209 |198936|0|0| C 05 CAL0000: TupleAggregateStep::doThreadedAggregate() caught Throw location unknown (consider using BOOST_THROW_EXCEPTION)#012Dynamic exception type: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::thread_resource_error> >#012std::exception::what: boost::thread_resource_error: Resource temporarily unavailable

Jul 22 12:46:26 ip-10-111-0-54 ExeMgr[9737]: 26.359785 |198936|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. Resource temporarily unavailable

Jul 22 12:46:35 ip-10-111-0-54 ExeMgr[9737]: 35.473785 |198913|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. Resource temporarily unavailable

Jul 22 12:46:39 ip-10-111-0-54 ExeMgr[9737]: 39.722336 |198930|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. Resource temporarily unavailable

Jul 22 13:00:12 ip-10-111-0-54 threadpool[9737]: 12.171413 |0|0|0| E 22 CAL0005: ThreadPool: Caught exception during execution:  boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:01:10 ip-10-111-0-54 ExeMgr[9737]: 10.819957 |2147683103|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:01:11 ip-10-111-0-54 ExeMgr[9737]: 11.007345 |2147683105|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:01:12 ip-10-111-0-54 ExeMgr[9737]: 12.039356 |2147683111|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:05:58 ip-10-111-0-54 ExeMgr[9737]: 58.503954 |2147683259|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:06:57 ip-10-111-0-54 ExeMgr[9737]: 57.268840 |2147683295|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:06:57 ip-10-111-0-54 ExeMgr[9737]: 57.636075 |2147683294|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 13:07:19 ip-10-111-0-54 ExeMgr[9737]: 19.297657 |2147683301|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:14 ip-10-111-0-54 ExeMgr[9737]: 14.409662 |2147684316|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:15 ip-10-111-0-54 ExeMgr[9737]: 15.246773 |2147684822|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='app' and tablename='client14spenddata' --columnRIDs/FE; ||

Jul 22 14:01:15 ip-10-111-0-54 ExeMgr[9737]: 15.265702 |2147684822|0|0| D 16 CAL0042: End SQL statement

Jul 22 14:01:15 ip-10-111-0-54 ExeMgr[9737]: 15.265770 |2147684822|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:16 ip-10-111-0-54 ExeMgr[9737]: 16.928010 |2147684823|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='app' and tablename='client14spenddata' --columnRIDs/FE; ||

Jul 22 14:01:16 ip-10-111-0-54 ExeMgr[9737]: 16.947776 |2147684823|0|0| D 16 CAL0042: End SQL statement

Jul 22 14:01:16 ip-10-111-0-54 ExeMgr[9737]: 16.947852 |2147684823|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:23 ip-10-111-0-54 ExeMgr[9737]: 23.982106 |2147684824|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='app' and tablename='client1spenddata' --columnRIDs/FE; ||

Jul 22 14:01:24 ip-10-111-0-54 ExeMgr[9737]: 24.004953 |2147684824|0|0| D 16 CAL0042: End SQL statement

Jul 22 14:01:24 ip-10-111-0-54 ExeMgr[9737]: 24.005033 |2147684824|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:30 ip-10-111-0-54 ExeMgr[9737]: 30.015599 |2147684827|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='app' and tablename='client540spenddata' --columnRIDs/FE; ||

Jul 22 14:01:30 ip-10-111-0-54 ExeMgr[9737]: 30.036076 |2147684827|0|0| D 16 CAL0042: End SQL statement

Jul 22 14:01:30 ip-10-111-0-54 ExeMgr[9737]: 30.036159 |2147684827|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:31 ip-10-111-0-54 ExeMgr[9737]: 31.475184 |2147684829|0|0| D 16 CAL0041: Start SQL statement: select objectid,columnname from syscolumn where schema='app' and tablename='client540spenddata' --columnRIDs/FE; ||

Jul 22 14:01:31 ip-10-111-0-54 ExeMgr[9737]: 31.493704 |2147684829|0|0| D 16 CAL0042: End SQL statement

Jul 22 14:01:31 ip-10-111-0-54 ExeMgr[9737]: 31.493773 |2147684829|0|0| C 16 CAL0055: ERROR: ExeMgr has caught an exception. boost::thread_resource_error: Resource temporarily unavailable

Jul 22 14:01:41 ip-10-111-0-54 DMLProc[9769]: 41.976756 |0|0|0| I 20 CAL0002: DMLProc has been told to rollback all DML transactions.

Jul 22 14:02:01 ip-10-111-0-54 IDBFile[9424]: 01.311265 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so

Jul 22 14:02:02 ip-10-111-0-54 StorageManager[9416]: SessionManager Caught Signal 15

Jul 22 14:02:02 ip-10-111-0-54 StorageManager[9416]: Shutdown StorageManager...

Jul 22 14:02:02 ip-10-111-0-54 controllernode[9185]: 02.375516 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::getSystemState() failed (network)

Jul 22 14:02:02 ip-10-111-0-54 controllernode[9185]: 02.375800 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::getSystemState() failed (network)

Jul 22 14:02:02 ip-10-111-0-54 StorageManager[9416]: Ownership: releasing ownership of data1

Jul 22 14:02:02 ip-10-111-0-54 StorageManager[9416]: StorageManager Shutdown Complete.

Jul 22 14:03:44 ip-10-111-0-54 controllernode[9876]: 44.722037 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::getSystemState() failed (network)

Jul 22 14:03:45 ip-10-111-0-54 controllernode[9876]: 45.909634 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::getSystemState() failed (network)

Jul 22 14:03:46 ip-10-111-0-54 controllernode[9876]: 46.298849 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::getSystemState() failed (network)

Jul 22 14:03:47 ip-10-111-0-54 controllernode[9876]: 47.698537 |0|0|0| E 29 CAL0000: DBRM: error: SessionManager::getSystemState() failed (network)

Jul 22 14:03:56 ip-10-111-0-54 StorageManager[10100]: Using the config file found at /etc/columnstore/storagemanager.cnf