17/Apr/2023 13:43:28 [INFO] (root) {MainThread} CMAPI Version: 23.02.1b
17/Apr/2023 13:43:28 [INFO] (root) {MainThread} Detected 7 MCS services.MCS version is 22.08.x and higher
17/Apr/2023 13:43:28 [DEBUG] (root) {MainThread} Waiting command to finish.
17/Apr/2023 13:43:28 [INFO] (root) {MainThread} Detected 239 SYSTEMD version.
17/Apr/2023 13:43:28 [DEBUG] (root) {MainThread} get_module_net_address Module 1 network address 127.0.0.1
17/Apr/2023 13:43:28 [INFO] (root) {MainThread} Loaded the config file, my name is None
17/Apr/2023 13:43:28 [INFO] (root) {MainThread} Failover is turned ON by default or in CMAPI config file.
17/Apr/2023 13:43:28 [INFO] (cherrypy.error) {MainThread} ENGINE Bus STARTING
17/Apr/2023 13:43:28 [INFO] (cherrypy.error) {MainThread} ENGINE Serving on https://0.0.0.0:8640
17/Apr/2023 13:43:28 [INFO] (cherrypy.error) {MainThread} ENGINE Bus STARTED
17/Apr/2023 13:43:28 [INFO] (root) {MainThread} get_current_config_file(): seeking the current config file
17/Apr/2023 13:43:28 [DEBUG] (root) {MainThread} get_module_net_address Module 1 network address 127.0.0.1
17/Apr/2023 13:43:28 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 127.0.0.1 ACCESS "GET /cmapi/ready HTTP/1.1" code 503, bytes 48, user-agent "curl/7.61.1"
17/Apr/2023 13:43:28 [INFO] (root) {MainThread} wait_for_deactivation_or_put_config: Was removed from the cluster, safe to continue.
17/Apr/2023 13:43:29 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 127.0.0.1 ACCESS "GET /cmapi/ready HTTP/1.1" code 503, bytes 48, user-agent "curl/7.61.1"
17/Apr/2023 13:43:30 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 127.0.0.1 ACCESS "GET /cmapi/ready HTTP/1.1" code 503, bytes 48, user-agent "curl/7.61.1"
17/Apr/2023 13:43:31 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 127.0.0.1 ACCESS "GET /cmapi/ready HTTP/1.1" code 503, bytes 48, user-agent "curl/7.61.1"
17/Apr/2023 13:43:32 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 127.0.0.1 ACCESS "GET /cmapi/ready HTTP/1.1" code 503, bytes 48, user-agent "curl/7.61.1"
17/Apr/2023 13:43:33 [INFO] (root) {MainThread} Detected "new" DBRM bytestream protocol
17/Apr/2023 13:43:33 [INFO] (cherrypy.error) {MainThread} ENGINE Starting Failover monitor thread.
17/Apr/2023 13:43:33 [INFO] (heartbeater) {HeartBeater} Starting the heartbeat listener.
17/Apr/2023 13:43:33 [INFO] (node_monitor) {NodeMonitor} Starting the monitor logic
17/Apr/2023 13:43:33 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 127.0.0.1 ACCESS "GET /cmapi/ready HTTP/1.1" code 200, bytes 17, user-agent "curl/7.61.1"
17/Apr/2023 13:43:34 [INFO] (node_monitor) {NodeMonitor} Failover support is inactive; requires at least 3 nodes and a shared storage system
17/Apr/2023 13:43:36 [DEBUG] (root) {NodeMonitor} get_module_net_address Module 1 network address 127.0.0.1
17/Apr/2023 13:43:36 [INFO] (root) {NodeMonitor} Loaded the config file, my name is None
17/Apr/2023 13:43:56 [INFO] (access_logger) {('CP Server Thread-5',)} Got incoming PUT request from "127.0.0.1" to "/cmapi/0.4.0/node/apikey-set". uid: 041e62f5-cf38-416b-8f82-002254d3e409
17/Apr/2023 13:43:56 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} Start setting new node API key.
17/Apr/2023 13:43:56 [INFO] (cmapi_server) {('CP Server Thread-5',)} API key successfully updated.
17/Apr/2023 13:43:56 [INFO] (access_logger) {('CP Server Thread-5',)} Finished processing incoming PUT request from "127.0.0.1" to "/cmapi/0.4.0/node/apikey-set" in 0.0010 seconds. uid: 041e62f5-cf38-416b-8f82-002254d3e409
17/Apr/2023 13:43:56 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 127.0.0.1 ACCESS "PUT /cmapi/0.4.0/node/apikey-set HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:01 [INFO] (access_logger) {('CP Server Thread-7',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin". uid: 9ba34d02-1af4-4d53-83b7-0f5cb09a2436
17/Apr/2023 13:44:01 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} put_begin starts
17/Apr/2023 13:44:01 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} put_begin JSON body {'id': 243595, 'timeout': 299}
17/Apr/2023 13:44:01 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} put_begin returns {'timestamp': '2023-04-17 13:44:01.665198'}
17/Apr/2023 13:44:01 [INFO] (access_logger) {('CP Server Thread-7',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin" in 0.0005 seconds. uid: 9ba34d02-1af4-4d53-83b7-0f5cb09a2436
17/Apr/2023 13:44:01 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/begin HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:01 [INFO] (access_logger) {('CP Server Thread-9',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config". uid: fc65971a-b225-4287-8867-286fb7afe790
17/Apr/2023 13:44:01 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} put_config starts
17/Apr/2023 13:44:01 [INFO] (root) {('CP Server Thread-9',)}  -- Matching against ModuleIPAddr1-1-3, which says 10.0.0.11
17/Apr/2023 13:44:01 [INFO] (root) {('CP Server Thread-9',)} Wrote "pm1" to /var/lib/columnstore/local/module
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Stopping "mcs-ddlproc".
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Call "stop" on service "mcs-ddlproc" with "systemctl stop mcs-ddlproc".
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:01 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:01 [INFO] (root) {('CP Server Thread-9',)} Trying to gracefully stop DMLProc using DBRM commands.
17/Apr/2023 13:44:01 [INFO] (root) {('CP Server Thread-9',)} DBRM._send_command: Command set_system_state was requested with value 24
17/Apr/2023 13:44:01 [INFO] (root) {('CP Server Thread-9',)} Waiting for DMLProc to stop in 10 seconds
17/Apr/2023 13:44:01 [INFO] (root) {('CP Server Thread-9',)} Waiting for DMLProc to stop. Seconds left 10.
17/Apr/2023 13:44:02 [INFO] (root) {NodeMonitor} Loaded the config file, my name is s1pm1
17/Apr/2023 13:44:02 [INFO] (root) {('CP Server Thread-9',)} Waiting for DMLProc to stop. Seconds left 9.
17/Apr/2023 13:44:03 [INFO] (root) {('CP Server Thread-9',)} Waiting for DMLProc to stop. Seconds left 8.
17/Apr/2023 13:44:03 [INFO] (root) {('CP Server Thread-9',)} DMLProc gracefully stopped by DBRM command.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Stopping "mcs-primproc".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Call "stop" on service "mcs-primproc" with "systemctl stop mcs-primproc".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Stopping "mcs-writeengineserver".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Call "stop" on service "mcs-writeengineserver" with "systemctl stop mcs-writeengineserver".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Stopping "mcs-controllernode".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Call "stop" on service "mcs-controllernode" with "systemctl stop mcs-controllernode".
17/Apr/2023 13:44:03 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "disable" on service "mcs-workernode@1.service" with "systemctl disable mcs-workernode@1.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Stopping "mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "stop" on service "mcs-workernode@1.service mcs-workernode@2.service" with "systemctl stop mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-10',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:04 [INFO] (access_logger) {('CP Server Thread-11',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: 70361b31-2ef3-47b3-9a07-fa01f4a0b65c
17/Apr/2023 13:44:04 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_primary starts
17/Apr/2023 13:44:04 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:44:04 [INFO] (access_logger) {('CP Server Thread-11',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0311 seconds. uid: 70361b31-2ef3-47b3-9a07-fa01f4a0b65c
17/Apr/2023 13:44:04 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-workernode" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-workernode" with "systemctl show -p ActiveState --value mcs-workernode".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-workernode" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Stopping "mcs-storagemanager".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "stop" on service "mcs-storagemanager" with "systemctl stop mcs-storagemanager".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-storagemanager" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-storagemanager" with "systemctl show -p ActiveState --value mcs-storagemanager".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-storagemanager" is in "inactive" state
17/Apr/2023 13:44:04 [INFO] (root) {('CP Server Thread-9',)} Skip starting StorageManager with systemd dispatcher.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "enable" on service "mcs-workernode@1.service" with "systemctl enable mcs-workernode@1.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-workernode@1.service" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Starting "mcs-workernode@1.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "start" on service "mcs-workernode@1.service" with "systemctl start mcs-workernode@1.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Successfully started mcs-workernode@1.service.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-workernode@1.service" is in "active" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting for all workernodes to come up before starting controllernode on a primary.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting for "['DBRM_Worker1']"....30
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Starting "mcs-controllernode".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "start" on service "mcs-controllernode" with "systemctl start mcs-controllernode".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Successfully started mcs-controllernode.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-controllernode" is in "active" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Starting "mcs-primproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "start" on service "mcs-primproc" with "systemctl start mcs-primproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Successfully started mcs-primproc.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-primproc" is in "active" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Starting "mcs-writeengineserver".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "start" on service "mcs-writeengineserver" with "systemctl start mcs-writeengineserver".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Successfully started mcs-writeengineserver.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-writeengineserver" is in "active" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-dmlproc" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Starting "mcs-dmlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "start" on service "mcs-dmlproc" with "systemctl start mcs-dmlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Successfully started mcs-dmlproc.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-dmlproc" is in "active" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Starting "mcs-ddlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Call "start" on service "mcs-ddlproc" with "systemctl start mcs-ddlproc".
17/Apr/2023 13:44:04 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-9',)} Attempt to run dbbuilder on primary node
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:05 [ERROR] (root) {('CP Server Thread-9',)} Calling "su -s /bin/sh -c "/usr/bin/dbbuilder 7" mysql" finished with return code: "1" and stderr+stdout "".
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-9',)} Successfully started mcs-ddlproc.
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-9',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-9',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-9',)} Waiting command to finish.
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-9',)} Service "mcs-ddlproc" is in "active" state
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} Node is ready to accept queries.
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-9',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config" in 3.3772 seconds. uid: fc65971a-b225-4287-8867-286fb7afe790
17/Apr/2023 13:44:05 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/config HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-3',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit". uid: b862cad0-4768-4b63-9644-daa1cca38691
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} put_commit starts
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} put_commit returns {'timestamp': '2023-04-17 13:44:05.058832'}
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-3',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit" in 0.0004 seconds. uid: b862cad0-4768-4b63-9644-daa1cca38691
17/Apr/2023 13:44:05 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/commit HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-5',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin". uid: 25e6980b-41fe-4845-a9da-7ded391be7d9
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} put_begin starts
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} put_begin JSON body {'id': 70064, 'timeout': 299}
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} put_begin returns {'timestamp': '2023-04-17 13:44:05.245504'}
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-5',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin" in 0.0003 seconds. uid: 25e6980b-41fe-4845-a9da-7ded391be7d9
17/Apr/2023 13:44:05 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/begin HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-7',)} Got incoming GET request from "10.0.0.11" to "/cmapi/0.4.0/node/new_primary". uid: 3246400f-bba1-44f2-83e1-b2f1d27dbb9f
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} get_new_primary starts
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} get_new_primary returns {'is_primary': True}
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-7',)} Finished processing incoming GET request from "10.0.0.11" to "/cmapi/0.4.0/node/new_primary" in 0.0103 seconds. uid: 3246400f-bba1-44f2-83e1-b2f1d27dbb9f
17/Apr/2023 13:44:05 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 10.0.0.11 ACCESS "GET /cmapi/0.4.0/node/new_primary HTTP/1.1" code 200, bytes 20, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:05 [INFO] (access_logger) {('CP Server Thread-10',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config". uid: 3186fbdd-1e80-46d0-8847-3fbe8a3a7c09
17/Apr/2023 13:44:05 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} put_config starts
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-10',)}  -- Matching against ModuleIPAddr1-1-3, which says 10.0.0.11
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-10',)} Wrote "pm1" to /var/lib/columnstore/local/module
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-ddlproc".
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-ddlproc" with "systemctl stop mcs-ddlproc".
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:05 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-10',)} Trying to gracefully stop DMLProc using DBRM commands.
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-10',)} DBRM._send_command: Command set_system_state was requested with value 24
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop in 10 seconds
17/Apr/2023 13:44:05 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop. Seconds left 10.
17/Apr/2023 13:44:06 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 10.0.0.12 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-9',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/em". uid: db51ebe9-7d79-48e7-9c3e-2a9513f37110
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} get_brm_bytes starts
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} get_brm_bytes returns em from local storage.
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} get_brm_bytes returns.
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-9',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/em" in 0.0012 seconds. uid: db51ebe9-7d79-48e7-9c3e-2a9513f37110
17/Apr/2023 13:44:06 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" code 200, bytes 4844, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-12',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/journal". uid: 98883640-f444-40d0-8266-34b3ff96289b
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes starts
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes returns journal from local storage.
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes returns.
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-12',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/journal" in 0.0008 seconds. uid: 98883640-f444-40d0-8266-34b3ff96289b
17/Apr/2023 13:44:06 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-12',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" code 200, bytes 5, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-3',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vbbm". uid: 062a8856-4d01-4cad-99d4-91c820781370
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_brm_bytes starts
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_brm_bytes returns vbbm from local storage.
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_brm_bytes returns.
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-3',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vbbm" in 0.0006 seconds. uid: 062a8856-4d01-4cad-99d4-91c820781370
17/Apr/2023 13:44:06 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" code 200, bytes 12, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-4',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vss". uid: c7f2421d-e466-4752-8acb-a8b5cf79bf2d
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes starts
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes returns vss from local storage.
17/Apr/2023 13:44:06 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes returns.
17/Apr/2023 13:44:06 [INFO] (access_logger) {('CP Server Thread-4',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vss" in 0.0006 seconds. uid: c7f2421d-e466-4752-8acb-a8b5cf79bf2d
17/Apr/2023 13:44:06 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-4',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" code 200, bytes 8, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:06 [INFO] (root) {NodeMonitor} Loaded the config file, my name is s1pm1
17/Apr/2023 13:44:06 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop. Seconds left 9.
17/Apr/2023 13:44:07 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop. Seconds left 8.
17/Apr/2023 13:44:07 [INFO] (root) {('CP Server Thread-10',)} DMLProc gracefully stopped by DBRM command.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-primproc".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-primproc" with "systemctl stop mcs-primproc".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-writeengineserver".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-writeengineserver" with "systemctl stop mcs-writeengineserver".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-controllernode".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-controllernode" with "systemctl stop mcs-controllernode".
17/Apr/2023 13:44:07 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Call "disable" on service "mcs-workernode@1.service" with "systemctl disable mcs-workernode@1.service".
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-workernode@1.service mcs-workernode@2.service" with "systemctl stop mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:44:21 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:22 [INFO] (access_logger) {('CP Server Thread-6',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: 5e755428-8652-4e71-ad2d-d458d49eebb7
17/Apr/2023 13:44:22 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_primary starts
17/Apr/2023 13:44:22 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:44:22 [INFO] (access_logger) {('CP Server Thread-6',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0376 seconds. uid: 5e755428-8652-4e71-ad2d-d458d49eebb7
17/Apr/2023 13:44:22 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-6',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-workernode" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-workernode" with "systemctl show -p ActiveState --value mcs-workernode".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-workernode" is in "inactive" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-storagemanager".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-storagemanager" with "systemctl stop mcs-storagemanager".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-storagemanager" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-storagemanager" with "systemctl show -p ActiveState --value mcs-storagemanager".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-storagemanager" is in "inactive" state
17/Apr/2023 13:44:22 [INFO] (root) {('CP Server Thread-10',)} Skip starting StorageManager with systemd dispatcher.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "enable" on service "mcs-workernode@1.service" with "systemctl enable mcs-workernode@1.service".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-workernode@1.service" is in "inactive" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-workernode@1.service".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-workernode@1.service" with "systemctl start mcs-workernode@1.service".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:22 [INFO] (access_logger) {('CP Server Thread-8',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: 339286b4-7e43-4740-8e13-e152e552c18b
17/Apr/2023 13:44:22 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_primary starts
17/Apr/2023 13:44:22 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:44:22 [INFO] (access_logger) {('CP Server Thread-8',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0255 seconds. uid: 339286b4-7e43-4740-8e13-e152e552c18b
17/Apr/2023 13:44:22 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-8',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-workernode@1.service.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-workernode@1.service" is in "active" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for all workernodes to come up before starting controllernode on a primary.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for "['DBRM_Worker1', 'DBRM_Worker2']"....30
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-controllernode".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-controllernode" with "systemctl start mcs-controllernode".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-controllernode.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-controllernode" is in "active" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-primproc".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-primproc" with "systemctl start mcs-primproc".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-primproc.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-primproc" is in "active" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-writeengineserver".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-writeengineserver" with "systemctl start mcs-writeengineserver".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-writeengineserver.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-writeengineserver" is in "active" state
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:44:22 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-dmlproc" is in "inactive" state
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-dmlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-dmlproc" with "systemctl start mcs-dmlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-dmlproc.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-dmlproc" is in "active" state
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-ddlproc" with "systemctl start mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-10',)} Attempt to run dbbuilder on primary node
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [ERROR] (root) {('CP Server Thread-10',)} Calling "su -s /bin/sh -c "/usr/bin/dbbuilder 7" mysql" finished with return code: "1" and stderr+stdout "".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-ddlproc.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-ddlproc" is in "active" state
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} Node is ready to accept queries.
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-10',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config" in 17.8690 seconds. uid: 3186fbdd-1e80-46d0-8847-3fbe8a3a7c09
17/Apr/2023 13:44:23 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-10',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/config HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-9',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit". uid: 2532a516-57e7-4650-aa56-02e66febc178
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} put_commit starts
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} put_commit returns {'timestamp': '2023-04-17 13:44:23.161593'}
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-9',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit" in 0.0003 seconds. uid: 2532a516-57e7-4650-aa56-02e66febc178
17/Apr/2023 13:44:23 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/commit HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-3',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin". uid: 4c90fb22-b4cb-4c6e-956d-45cc7ff1d004
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} put_begin starts
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} put_begin JSON body {'id': 994243, 'timeout': 299}
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} put_begin returns {'timestamp': '2023-04-17 13:44:23.366327'}
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-3',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin" in 0.0005 seconds. uid: 4c90fb22-b4cb-4c6e-956d-45cc7ff1d004
17/Apr/2023 13:44:23 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/begin HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-5',)} Got incoming GET request from "10.0.0.11" to "/cmapi/0.4.0/node/new_primary". uid: 6b994863-381f-4056-997d-8d73ce28b315
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_new_primary starts
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_new_primary returns {'is_primary': True}
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-5',)} Finished processing incoming GET request from "10.0.0.11" to "/cmapi/0.4.0/node/new_primary" in 0.0091 seconds. uid: 6b994863-381f-4056-997d-8d73ce28b315
17/Apr/2023 13:44:23 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 10.0.0.11 ACCESS "GET /cmapi/0.4.0/node/new_primary HTTP/1.1" code 200, bytes 20, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:23 [INFO] (access_logger) {('CP Server Thread-7',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config". uid: 1aab8100-6e0b-4687-b4ca-dc2e1aa89168
17/Apr/2023 13:44:23 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} put_config starts
17/Apr/2023 13:44:23 [INFO] (root) {NodeMonitor} Loaded the config file, my name is s1pm1
17/Apr/2023 13:44:23 [INFO] (node_monitor) {NodeMonitor} Failover support is active, monitoring nodes ['s1pm1', 's1pm2', 's1pm3']
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-7',)}  -- Matching against ModuleIPAddr1-1-3, which says 10.0.0.11
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-7',)} Wrote "pm1" to /var/lib/columnstore/local/module
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Stopping "mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Call "stop" on service "mcs-ddlproc" with "systemctl stop mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:23 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-7',)} Trying to gracefully stop DMLProc using DBRM commands.
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-7',)} DBRM._send_command: Command set_system_state was requested with value 24
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-7',)} Waiting for DMLProc to stop in 10 seconds
17/Apr/2023 13:44:23 [INFO] (root) {('CP Server Thread-7',)} Waiting for DMLProc to stop. Seconds left 10.
17/Apr/2023 13:44:24 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-8',)} 10.0.0.13 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-10',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/em". uid: 5951fd81-9c00-47df-8952-81bfd79ad93b
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} get_brm_bytes starts
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} get_brm_bytes returns em from local storage.
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} get_brm_bytes returns.
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-10',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/em" in 0.0039 seconds. uid: 5951fd81-9c00-47df-8952-81bfd79ad93b
17/Apr/2023 13:44:24 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-10',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" code 200, bytes 4844, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-11',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/journal". uid: 8209a34a-6fc7-4cdd-a7dc-c1d07495727c
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_brm_bytes starts
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_brm_bytes returns journal from local storage.
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_brm_bytes returns.
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-11',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/journal" in 0.0019 seconds. uid: 8209a34a-6fc7-4cdd-a7dc-c1d07495727c
17/Apr/2023 13:44:24 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" code 200, bytes 5, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-9',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vbbm". uid: ae5211ba-ab78-4322-9108-29246a7f3045
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} get_brm_bytes starts
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} get_brm_bytes returns vbbm from local storage.
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-9',)} get_brm_bytes returns.
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-9',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vbbm" in 0.0033 seconds. uid: ae5211ba-ab78-4322-9108-29246a7f3045
17/Apr/2023 13:44:24 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" code 200, bytes 12, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-12',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vss". uid: 43468316-9409-44a3-bc67-d4610ee6e66e
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes starts
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes returns vss from local storage.
17/Apr/2023 13:44:24 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes returns.
17/Apr/2023 13:44:24 [INFO] (access_logger) {('CP Server Thread-12',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vss" in 0.0009 seconds. uid: 43468316-9409-44a3-bc67-d4610ee6e66e
17/Apr/2023 13:44:24 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-12',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" code 200, bytes 8, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:24 [INFO] (root) {('CP Server Thread-7',)} Waiting for DMLProc to stop. Seconds left 9.
17/Apr/2023 13:44:25 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 10.0.0.12 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-4',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/em". uid: c10b0c74-f0de-4be5-89f2-c67a0329e8ad
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes starts
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes returns em from local storage.
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes returns.
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-4',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/em" in 0.0013 seconds. uid: c10b0c74-f0de-4be5-89f2-c67a0329e8ad
17/Apr/2023 13:44:25 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-4',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" code 200, bytes 4844, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-5',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/journal". uid: 5b8c13de-d332-4f3c-a0e0-d483f1211510
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_brm_bytes starts
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_brm_bytes returns journal from local storage.
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_brm_bytes returns.
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-5',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/journal" in 0.0034 seconds. uid: 5b8c13de-d332-4f3c-a0e0-d483f1211510
17/Apr/2023 13:44:25 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" code 200, bytes 5, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-6',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vbbm". uid: 607815f1-d935-4b2b-b16a-59876593dfc0
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_brm_bytes starts
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_brm_bytes returns vbbm from local storage.
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_brm_bytes returns.
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-6',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vbbm" in 0.0013 seconds. uid: 607815f1-d935-4b2b-b16a-59876593dfc0
17/Apr/2023 13:44:25 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-6',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" code 200, bytes 12, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-8',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vss". uid: e5c73f92-b411-4fbd-9db0-158e616c9e64
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_brm_bytes starts
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_brm_bytes returns vss from local storage.
17/Apr/2023 13:44:25 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_brm_bytes returns.
17/Apr/2023 13:44:25 [INFO] (access_logger) {('CP Server Thread-8',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vss" in 0.0006 seconds. uid: e5c73f92-b411-4fbd-9db0-158e616c9e64
17/Apr/2023 13:44:25 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-8',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" code 200, bytes 8, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:25 [INFO] (root) {('CP Server Thread-7',)} Waiting for DMLProc to stop. Seconds left 8.
17/Apr/2023 13:44:25 [INFO] (root) {('CP Server Thread-7',)} DMLProc gracefully stopped by DBRM command.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Stopping "mcs-primproc".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Call "stop" on service "mcs-primproc" with "systemctl stop mcs-primproc".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Stopping "mcs-writeengineserver".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Call "stop" on service "mcs-writeengineserver" with "systemctl stop mcs-writeengineserver".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Stopping "mcs-controllernode".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Call "stop" on service "mcs-controllernode" with "systemctl stop mcs-controllernode".
17/Apr/2023 13:44:25 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:40 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:40 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:40 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:40 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:44:40 [DEBUG] (root) {('CP Server Thread-7',)} Call "disable" on service "mcs-workernode@1.service" with "systemctl disable mcs-workernode@1.service".
17/Apr/2023 13:44:40 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Stopping "mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "stop" on service "mcs-workernode@1.service mcs-workernode@2.service" with "systemctl stop mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-10',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:41 [INFO] (access_logger) {('CP Server Thread-11',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: cffb07d9-ce83-4874-aa8e-886195e222f8
17/Apr/2023 13:44:41 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_primary starts
17/Apr/2023 13:44:41 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:44:41 [INFO] (access_logger) {('CP Server Thread-11',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0270 seconds. uid: cffb07d9-ce83-4874-aa8e-886195e222f8
17/Apr/2023 13:44:41 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-workernode" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-workernode" with "systemctl show -p ActiveState --value mcs-workernode".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-workernode" is in "inactive" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Stopping "mcs-storagemanager".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "stop" on service "mcs-storagemanager" with "systemctl stop mcs-storagemanager".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-storagemanager" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-storagemanager" with "systemctl show -p ActiveState --value mcs-storagemanager".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-storagemanager" is in "inactive" state
17/Apr/2023 13:44:41 [INFO] (root) {('CP Server Thread-7',)} Skip starting StorageManager with systemd dispatcher.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "enable" on service "mcs-workernode@1.service" with "systemctl enable mcs-workernode@1.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-workernode@1.service" is in "inactive" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Starting "mcs-workernode@1.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "start" on service "mcs-workernode@1.service" with "systemctl start mcs-workernode@1.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:41 [INFO] (access_logger) {('CP Server Thread-12',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: 985b85a3-c317-4481-9697-10b05c13e7ac
17/Apr/2023 13:44:41 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_primary starts
17/Apr/2023 13:44:41 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:44:41 [INFO] (access_logger) {('CP Server Thread-12',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0336 seconds. uid: 985b85a3-c317-4481-9697-10b05c13e7ac
17/Apr/2023 13:44:41 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-12',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Successfully started mcs-workernode@1.service.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-workernode@1.service" is in "active" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting for all workernodes to come up before starting controllernode on a primary.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting for "['DBRM_Worker1', 'DBRM_Worker2', 'DBRM_Worker3']"....30
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Starting "mcs-controllernode".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "start" on service "mcs-controllernode" with "systemctl start mcs-controllernode".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Successfully started mcs-controllernode.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-controllernode" is in "active" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Starting "mcs-primproc".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "start" on service "mcs-primproc" with "systemctl start mcs-primproc".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Successfully started mcs-primproc.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-primproc" is in "active" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Starting "mcs-writeengineserver".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Call "start" on service "mcs-writeengineserver" with "systemctl start mcs-writeengineserver".
17/Apr/2023 13:44:41 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Successfully started mcs-writeengineserver.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-writeengineserver" is in "active" state
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-dmlproc" is in "inactive" state
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Starting "mcs-dmlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "start" on service "mcs-dmlproc" with "systemctl start mcs-dmlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Successfully started mcs-dmlproc.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-dmlproc" is in "active" state
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Starting "mcs-ddlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "start" on service "mcs-ddlproc" with "systemctl start mcs-ddlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [INFO] (root) {('CP Server Thread-7',)} Attempt to run dbbuilder on primary node
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [ERROR] (root) {('CP Server Thread-7',)} Calling "su -s /bin/sh -c "/usr/bin/dbbuilder 7" mysql" finished with return code: "1" and stderr+stdout "".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Successfully started mcs-ddlproc.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Waiting command to finish.
17/Apr/2023 13:44:42 [DEBUG] (root) {('CP Server Thread-7',)} Service "mcs-ddlproc" is in "active" state
17/Apr/2023 13:44:42 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} Node is ready to accept queries.
17/Apr/2023 13:44:42 [INFO] (access_logger) {('CP Server Thread-7',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config" in 18.7944 seconds. uid: 1aab8100-6e0b-4687-b4ca-dc2e1aa89168
17/Apr/2023 13:44:42 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/config HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:42 [INFO] (access_logger) {('CP Server Thread-4',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit". uid: 41f42935-2d2c-43cf-b471-7ef7d7fd9c83
17/Apr/2023 13:44:42 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} put_commit starts
17/Apr/2023 13:44:42 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} put_commit returns {'timestamp': '2023-04-17 13:44:42.200493'}
17/Apr/2023 13:44:42 [INFO] (access_logger) {('CP Server Thread-4',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit" in 0.0003 seconds. uid: 41f42935-2d2c-43cf-b471-7ef7d7fd9c83
17/Apr/2023 13:44:42 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-4',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/commit HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:44 [INFO] (access_logger) {('CP Server Thread-6',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin". uid: 0e435309-e19c-4faf-a4fa-3cd9bdaca72e
17/Apr/2023 13:44:44 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} put_begin starts
17/Apr/2023 13:44:44 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} put_begin JSON body {'id': 492999, 'timeout': 299}
17/Apr/2023 13:44:44 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} put_begin returns {'timestamp': '2023-04-17 13:44:44.525393'}
17/Apr/2023 13:44:44 [INFO] (access_logger) {('CP Server Thread-6',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/begin" in 0.0004 seconds. uid: 0e435309-e19c-4faf-a4fa-3cd9bdaca72e
17/Apr/2023 13:44:44 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-6',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/begin HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:44:44 [INFO] (access_logger) {('CP Server Thread-10',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config". uid: e945996a-3b71-4d21-b8e5-01500b6a0efa
17/Apr/2023 13:44:44 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} put_config starts
17/Apr/2023 13:44:44 [INFO] (root) {('CP Server Thread-10',)}  -- Matching against ModuleIPAddr1-1-3, which says 10.0.0.11
17/Apr/2023 13:44:44 [INFO] (root) {('CP Server Thread-10',)} Wrote "pm1" to /var/lib/columnstore/local/module
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-ddlproc".
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-ddlproc" with "systemctl stop mcs-ddlproc".
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:44 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:44:44 [INFO] (root) {('CP Server Thread-10',)} Trying to gracefully stop DMLProc using DBRM commands.
17/Apr/2023 13:44:44 [INFO] (root) {('CP Server Thread-10',)} DBRM._send_command: Command set_system_state was requested with value 24
17/Apr/2023 13:44:44 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop in 10 seconds
17/Apr/2023 13:44:44 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop. Seconds left 10.
17/Apr/2023 13:44:44 [INFO] (root) {NodeMonitor} Loaded the config file, my name is s1pm1
17/Apr/2023 13:44:45 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop. Seconds left 9.
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 10.0.0.12 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 10.0.0.13 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-12',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/em". uid: 2fcfdc4c-c379-4d98-bb5c-7bde655a43cc
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes returns em from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-12',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/em" in 0.0014 seconds. uid: 2fcfdc4c-c379-4d98-bb5c-7bde655a43cc
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-12',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" code 200, bytes 4844, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-7',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/em". uid: c5658fe9-3271-45c9-a01e-1e721eedad1c
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} get_brm_bytes returns em from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-7',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-7',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/em" in 0.0024 seconds. uid: c5658fe9-3271-45c9-a01e-1e721eedad1c
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" code 200, bytes 4844, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-3',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/journal". uid: 897bbf20-683b-4d75-b3e3-586e30f665c7
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_brm_bytes returns journal from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-3',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/journal" in 0.0017 seconds. uid: 897bbf20-683b-4d75-b3e3-586e30f665c7
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" code 200, bytes 5, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-4',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/journal". uid: 697772b3-4c09-47c0-8559-d89ca909e1f7
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes returns journal from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-4',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-4',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/journal" in 0.0019 seconds. uid: 697772b3-4c09-47c0-8559-d89ca909e1f7
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-4',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" code 200, bytes 5, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-5',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vbbm". uid: 269329e6-f925-4ee2-b669-5ff367b212d1
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_brm_bytes returns vbbm from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-5',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vbbm" in 0.0007 seconds. uid: 269329e6-f925-4ee2-b669-5ff367b212d1
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" code 200, bytes 12, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-6',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vbbm". uid: 39f7130e-77f9-4927-a9a5-aca511795a8f
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_brm_bytes returns vbbm from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-6',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-6',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vbbm" in 0.0006 seconds. uid: 39f7130e-77f9-4927-a9a5-aca511795a8f
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-6',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" code 200, bytes 12, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-8',)} Got incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vss". uid: 34e3f28b-b3b6-49bc-90c7-c2a334dc92cd
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_brm_bytes returns vss from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-8',)} Finished processing incoming GET request from "10.0.0.12" to "/cmapi/0.4.0/node/meta/vss" in 0.0006 seconds. uid: 34e3f28b-b3b6-49bc-90c7-c2a334dc92cd
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-8',)} 10.0.0.12 ACCESS "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" code 200, bytes 8, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-11',)} Got incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vss". uid: b621e33b-598e-43fc-afd3-ec8f79460e28
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_brm_bytes starts
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_brm_bytes returns vss from local storage.
17/Apr/2023 13:44:46 [DEBUG] (cmapi_server) {('CP Server Thread-11',)} get_brm_bytes returns.
17/Apr/2023 13:44:46 [INFO] (access_logger) {('CP Server Thread-11',)} Finished processing incoming GET request from "10.0.0.13" to "/cmapi/0.4.0/node/meta/vss" in 0.0006 seconds. uid: b621e33b-598e-43fc-afd3-ec8f79460e28
17/Apr/2023 13:44:46 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-11',)} 10.0.0.13 ACCESS "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" code 200, bytes 8, user-agent "Python-urllib/3.6"
17/Apr/2023 13:44:46 [INFO] (root) {('CP Server Thread-10',)} Waiting for DMLProc to stop. Seconds left 8.
17/Apr/2023 13:44:46 [INFO] (root) {('CP Server Thread-10',)} DMLProc gracefully stopped by DBRM command.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-primproc".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-primproc" with "systemctl stop mcs-primproc".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-writeengineserver".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-writeengineserver" with "systemctl stop mcs-writeengineserver".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-controllernode".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-controllernode" with "systemctl stop mcs-controllernode".
17/Apr/2023 13:44:46 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "disable" on service "mcs-workernode@1.service" with "systemctl disable mcs-workernode@1.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-workernode@1.service mcs-workernode@2.service" with "systemctl stop mcs-workernode@1.service mcs-workernode@2.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-9',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:45:02 [INFO] (access_logger) {('CP Server Thread-12',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: 327df2f5-1a4a-4b4d-a578-5ea9ac22c4e2
17/Apr/2023 13:45:02 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_primary starts
17/Apr/2023 13:45:02 [DEBUG] (cmapi_server) {('CP Server Thread-12',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:45:02 [INFO] (access_logger) {('CP Server Thread-12',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0355 seconds. uid: 327df2f5-1a4a-4b4d-a578-5ea9ac22c4e2
17/Apr/2023 13:45:02 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-12',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-workernode" is running.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-workernode" with "systemctl show -p ActiveState --value mcs-workernode".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-workernode" is in "inactive" state
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Stopping "mcs-storagemanager".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "stop" on service "mcs-storagemanager" with "systemctl stop mcs-storagemanager".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-storagemanager" is running.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-storagemanager" with "systemctl show -p ActiveState --value mcs-storagemanager".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-storagemanager" is in "inactive" state
17/Apr/2023 13:45:02 [INFO] (root) {('CP Server Thread-10',)} Skip starting StorageManager with systemd dispatcher.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "enable" on service "mcs-workernode@1.service" with "systemctl enable mcs-workernode@1.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-workernode@1.service" is in "inactive" state
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-workernode@1.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-workernode@1.service" with "systemctl start mcs-workernode@1.service".
17/Apr/2023 13:45:02 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:02 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-7',)} 127.0.0.1 ACCESS "POST /notfound HTTP/1.1" code 404, bytes 513, user-agent "Python-urllib/3.6"
17/Apr/2023 13:45:02 [INFO] (access_logger) {('CP Server Thread-3',)} Got incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary". uid: 83d73853-c39d-4584-a375-bf9c01580488
17/Apr/2023 13:45:02 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_primary starts
17/Apr/2023 13:45:02 [DEBUG] (cmapi_server) {('CP Server Thread-3',)} get_primary returns {'is_primary': 'True'}
17/Apr/2023 13:45:02 [INFO] (access_logger) {('CP Server Thread-3',)} Finished processing incoming GET request from "127.0.0.1" to "/cmapi/0.4.0/node/primary" in 0.0399 seconds. uid: 83d73853-c39d-4584-a375-bf9c01580488
17/Apr/2023 13:45:02 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-3',)} 127.0.0.1 ACCESS "GET /cmapi/0.4.0/node/primary HTTP/1.1" code 200, bytes 22, user-agent "Python-urllib/3.6"
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-workernode@1.service.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-workernode@1.service" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-workernode@1.service" with "systemctl show -p ActiveState --value mcs-workernode@1.service".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-workernode@1.service" is in "active" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for all workernodes to come up before starting controllernode on a primary.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for "['DBRM_Worker1', 'DBRM_Worker2', 'DBRM_Worker3']"....30
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-controllernode" is in "inactive" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-controllernode".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-controllernode" with "systemctl start mcs-controllernode".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-controllernode.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-controllernode" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-controllernode" with "systemctl show -p ActiveState --value mcs-controllernode".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-controllernode" is in "active" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-primproc" is in "inactive" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-primproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-primproc" with "systemctl start mcs-primproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-primproc.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-primproc" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-primproc" with "systemctl show -p ActiveState --value mcs-primproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-primproc" is in "active" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-writeengineserver" is in "inactive" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-writeengineserver".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-writeengineserver" with "systemctl start mcs-writeengineserver".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-writeengineserver.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-writeengineserver" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-writeengineserver" with "systemctl show -p ActiveState --value mcs-writeengineserver".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-writeengineserver" is in "active" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-dmlproc" is in "inactive" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-dmlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-dmlproc" with "systemctl start mcs-dmlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-dmlproc.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-dmlproc" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-dmlproc" with "systemctl show -p ActiveState --value mcs-dmlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-dmlproc" is in "active" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-ddlproc" is in "inactive" state
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Starting "mcs-ddlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "start" on service "mcs-ddlproc" with "systemctl start mcs-ddlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [INFO] (root) {('CP Server Thread-10',)} Attempt to run dbbuilder on primary node
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [ERROR] (root) {('CP Server Thread-10',)} Calling "su -s /bin/sh -c "/usr/bin/dbbuilder 7" mysql" finished with return code: "1" and stderr+stdout "".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Successfully started mcs-ddlproc.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Checking "mcs-ddlproc" is running.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Call "show -p ActiveState --value" on service "mcs-ddlproc" with "systemctl show -p ActiveState --value mcs-ddlproc".
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Waiting command to finish.
17/Apr/2023 13:45:03 [DEBUG] (root) {('CP Server Thread-10',)} Service "mcs-ddlproc" is in "active" state
17/Apr/2023 13:45:03 [DEBUG] (cmapi_server) {('CP Server Thread-10',)} Node is ready to accept queries.
17/Apr/2023 13:45:03 [INFO] (access_logger) {('CP Server Thread-10',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/config" in 18.8995 seconds. uid: e945996a-3b71-4d21-b8e5-01500b6a0efa
17/Apr/2023 13:45:03 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-10',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/config HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:45:03 [INFO] (access_logger) {('CP Server Thread-5',)} Got incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit". uid: 820f13aa-0d2f-4e6a-9b32-126ddef29233
17/Apr/2023 13:45:03 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} put_commit starts
17/Apr/2023 13:45:03 [DEBUG] (cmapi_server) {('CP Server Thread-5',)} put_commit returns {'timestamp': '2023-04-17 13:45:03.441429'}
17/Apr/2023 13:45:03 [INFO] (access_logger) {('CP Server Thread-5',)} Finished processing incoming PUT request from "10.0.0.11" to "/cmapi/0.4.0/node/commit" in 0.0003 seconds. uid: 820f13aa-0d2f-4e6a-9b32-126ddef29233
17/Apr/2023 13:45:03 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-5',)} 10.0.0.11 ACCESS "PUT /cmapi/0.4.0/node/commit HTTP/1.1" code 200, bytes 43, user-agent "python-requests/2.27.1"
17/Apr/2023 13:45:08 [INFO] (access_logger) {('CP Server Thread-8',)} Got incoming GET request from "10.0.0.11" to "/cmapi/0.4.0/node/status". uid: ef6a00be-fcd9-4313-bd4c-59fb4f6b1187
17/Apr/2023 13:45:08 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_status starts
17/Apr/2023 13:45:08 [INFO] (root) {('CP Server Thread-8',)} DBRM._send_command: Command readonly was requested with value None
17/Apr/2023 13:45:08 [DEBUG] (cmapi_server) {('CP Server Thread-8',)} get_status returns {'timestamp': '2023-04-17 13:45:08.707301', 'uptime': 166, 'dbrm_mode': 'master', 'cluster_mode': 'readwrite', 'dbroots': ['1'], 'module_id': 1, 'services': [{'name': 'workernode', 'pid': 13081}, {'name': 'controllernode', 'pid': 13092}, {'name': 'PrimProc', 'pid': 13109}, {'name': 'WriteEngineServer', 'pid': 13152}, {'name': 'DMLProc', 'pid': 13161}, {'name': 'DDLProc', 'pid': 13191}]}
17/Apr/2023 13:45:08 [INFO] (access_logger) {('CP Server Thread-8',)} Finished processing incoming GET request from "10.0.0.11" to "/cmapi/0.4.0/node/status" in 0.0197 seconds. uid: ef6a00be-fcd9-4313-bd4c-59fb4f6b1187
17/Apr/2023 13:45:08 [INFO] (cherrypy.access.140159421382608) {('CP Server Thread-8',)} 10.0.0.11 ACCESS "GET /cmapi/0.4.0/node/status HTTP/1.1" code 200, bytes 389, user-agent "python-requests/2.27.1"