Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4289

columnstore-docker-cluster: cmapi shows error messages when used with custom replication user credentials

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • 5.4.1
    • None
    • mariadb/skysql-enterprise-columnstore-handoff sky-1.5.4-1 95e34753d40f
    • 2020-8

    Description

      If non defaults for the replication user are used, error messages are shown in /var/log/mariadb/columnstore/cmapi_server.log.

      [02/Sep/2020 19:13:46] root  Waiting for Maxscale to choose the new repl master...
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:46] root  New repl master has not been chosen yet
      

      How to reproduce:

      jens@ubuntu18:~/columnstore-docker-cluster$ cat .env
      # S3 Variables
      ASYNC_CONN=0
      ANALYTICS_ONLY=1
      USE_S3_STORAGE=0
      S3_BUCKET=
      S3_ENDPOINT=
      S3_REGION=
      S3_ACCESS_KEY_ID=
      S3_SECRET_ACCESS_KEY=
       
      # Host Variables
      PM1=mcs1
      PM2=mcs2
      PM3=mcs3
      MX1=mx1
       
      # Credentials
      CMAPI_KEY=aAbB44!!!E2
      ADMIN_USER=skysql_admin
      ADMIN_PASS=aAbB44!!!E3
      CEJ_USER=skysql_cej
      CEJ_PASS=aAbB44!!!E4
      MAX_USER=skysql_maxscale
      MAX_PASS=aAbB44!!!E5
      REP_USER=skysql_replication
      REP_PASS=aAbB44!!!E6
      

      jens@ubuntu18:~/columnstore-docker-cluster$ docker-compose up -d
      Creating network "columnstore-docker-cluster_default" with the default driver
      Creating volume "columnstore-docker-cluster_data1" with default driver
      Creating volume "columnstore-docker-cluster_data2" with default driver
      Creating volume "columnstore-docker-cluster_data3" with default driver
      Creating volume "columnstore-docker-cluster_storagemanager" with default driver
      Creating mcs3 ... done                                                                                                                                                                                                                       Creating mcs2 ... done                                                                                                                                                                                                                       Creating mcs1 ... done                                                                                                                                                                                                                       Creating mx1  ... done                                                                                                                                                                                                                       jens@ubuntu18:~/columnstore-docker-cluster$ docker logs mxs1 -f
      

      jens@ubuntu18:~/columnstore-docker-cluster$ docker exec -it mcs1 demo
      Waiting for PM1 to be initialized ... done
      Adding PM1 to cluster ... done
      Adding PM2 to cluster ... done
      Adding PM3 to cluster ... done
      Validating ... done
      Adding PM1 to MaxScale ... done
      Adding PM2 to MaxScale ... done
      Adding PM3 to MaxScale ... done
      Adding service ... done
      Adding listener ... done
      Adding monitor ... done
      

      jens@ubuntu18:~/columnstore-docker-cluster$ docker exec -it mcs1 bash
      [root@mcs1 /]# mariadb -e "SELECT user,host from mysql.user"
      +--------------------+-----------+
      | User               | Host      |
      +--------------------+-----------+
      | cej                | %         |
      | skysql_admin       | %         |
      | skysql_maxscale    | %         |
      | skysql_replication | %         |
      | mariadb.sys        | localhost |
      | mysql              | localhost |
      | root               | localhost |
      +--------------------+-----------+
      

      [root@mcs1 /]# cat /var/log/mariadb/columnstore/cmapi_server.log
      127.0.0.1 - - [02/Sep/2020 19:13:35] cmapi_server DEBUG    Using container dispatcher module
      [02/Sep/2020 19:13:35] root  CMAPI Version: 1.5.3.1
      [02/Sep/2020:19:13:35] ENGINE Bus STARTING
      [02/Sep/2020:19:13:35] ENGINE Serving on https://0.0.0.0:8640
      [02/Sep/2020:19:13:35] ENGINE Bus STARTED
      [02/Sep/2020 19:13:35] root  get_current_config_file(): seeking the current config file
      [02/Sep/2020 19:13:35] root  get_module_net_address Module 1 network address 127.0.0.1
      [02/Sep/2020 19:13:35] root  wait_for_deactivation_or_put_config: Was removed from the cluster, safe to continue.
      [02/Sep/2020 19:13:35] root  get_module_net_address Module 1 network address 127.0.0.1
      [02/Sep/2020 19:13:35] root  Loaded the config file, my name is None
      [02/Sep/2020 19:13:35] root  Starting the heartbeat listener
      [02/Sep/2020 19:13:35] root  starting the monitor logic
      [02/Sep/2020 19:13:36] root  Failover support is inactive; requires at least 3 nodes and a shared storage system
      172.18.0.4 - - [02/Sep/2020 19:13:38] cmapi_server DEBUG    put_add_node starts
      172.18.0.4 - - [02/Sep/2020 19:13:38] cmapi_server DEBUG    put_begin starts
      172.18.0.4 - - [02/Sep/2020 19:13:38] cmapi_server DEBUG    put_begin JSON body {'id': 206126, 'timeout': 299}
      172.18.0.4 - - [02/Sep/2020 19:13:38] cmapi_server DEBUG    put_begin returns {'timestamp': '2020-09-02 19:13:38.941489'}
      172.18.0.4 - - [02/Sep/2020:19:13:38] "PUT /cmapi/0.4.0/node/begin HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [02/Sep/2020 19:13:38] root  get_module_net_address Module 1 network address 127.0.0.1
      [02/Sep/2020 19:13:38] root  add_node(): replacing 127.0.0.1/localhost with 172.18.0.4/mcs1 as this node's name. Be sure mcs1 resolves to 172.18.0.4 on all other nodes in the cluster.
      172.18.0.4 - - [02/Sep/2020 19:13:38] cmapi_server DEBUG    put_config starts
      [02/Sep/2020 19:13:39] root   -- Matching against ModuleIPAddr1-1-3, which says 172.18.0.4
      [02/Sep/2020 19:13:39] root  Wrote 'pm1' to /var/lib/columnstore/local/module
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-dmlproc. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-dmlproc 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-ddlproc. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-ddlproc 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-primproc. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-primproc 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-writeengineserver. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-writeengineserver 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-exemgr. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-exemgr 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-controllernode. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-controllernode 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-workernode. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-workernode 1
      [02/Sep/2020 19:13:39] root  apply Running stop on mcs-storagemanager. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-storagemanager 1
      [02/Sep/2020 19:13:39] root  apply Running start on mcs-workernode. With sudo False and is_primary True
      [02/Sep/2020 19:13:39] root  start running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh start mcs-workernode 1
      [02/Sep/2020 19:13:39] root  Loaded the config file, my name is mcs1
      [02/Sep/2020 19:13:40] root  apply Waiting for all workernodes to come up before starting controllernode on the primary.
      [02/Sep/2020 19:13:40] root  apply Trying...
      [02/Sep/2020 19:13:41] root  apply Trying...
      [02/Sep/2020 19:13:41] root  apply Running start on mcs-controllernode. With sudo False and is_primary True
      [02/Sep/2020 19:13:41] root  start running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh start mcs-controllernode 1
      [02/Sep/2020 19:13:45] root  apply Running start on mcs-primproc. With sudo False and is_primary True
      [02/Sep/2020 19:13:45] root  apply Running start on mcs-exemgr. With sudo False and is_primary True
      [02/Sep/2020 19:13:45] root  apply Running start on mcs-writeengineserver. With sudo False and is_primary True
      [02/Sep/2020 19:13:45] root  apply Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
      [02/Sep/2020 19:13:45] root  apply Trying...
      [02/Sep/2020 19:13:45] root  apply Running start on mcs-dmlproc. With sudo False and is_primary True
      [02/Sep/2020 19:13:45] root  apply Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
      [02/Sep/2020 19:13:45] root  apply Trying...
      [02/Sep/2020 19:13:45] root  apply Running start on mcs-ddlproc. With sudo False and is_primary True
      172.18.0.4 - - [02/Sep/2020:19:13:45] "PUT /cmapi/0.4.0/node/config HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [02/Sep/2020 19:13:45] root  broadcast_new_config(): successfully pushed the new config file to ['mcs1']
      172.18.0.4 - - [02/Sep/2020 19:13:45] cmapi_server DEBUG    put_commit starts
      172.18.0.4 - - [02/Sep/2020 19:13:45] cmapi_server DEBUG    put_commit returns {'timestamp': '2020-09-02 19:13:45.997857'}
      172.18.0.4 - - [02/Sep/2020:19:13:45] "PUT /cmapi/0.4.0/node/commit HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:13:45] cmapi_server DEBUG    put_add_node returns {'timestamp': '2020-09-02 19:13:38.929365', 'node_id': 'mcs1'}
      172.18.0.4 - - [02/Sep/2020:19:13:46] "PUT /cmapi/0.4.0/cluster/add-node HTTP/1.1" 200 62 "" "curl/7.61.1"
      172.18.0.4 - - [02/Sep/2020 19:13:46] cmapi_server DEBUG    put_add_node starts
      172.18.0.4 - - [02/Sep/2020 19:13:46] cmapi_server DEBUG    put_begin starts
      172.18.0.4 - - [02/Sep/2020 19:13:46] cmapi_server DEBUG    put_begin JSON body {'id': 561028, 'timeout': 299}
      172.18.0.4 - - [02/Sep/2020 19:13:46] cmapi_server DEBUG    put_begin returns {'timestamp': '2020-09-02 19:13:46.087717'}
      172.18.0.4 - - [02/Sep/2020:19:13:46] "PUT /cmapi/0.4.0/node/begin HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [02/Sep/2020 19:13:46] root  _add_Module_entries(): using ip address 172.18.0.3 and hostname mcs2
      [02/Sep/2020 19:13:46] root  Waiting for Maxscale to choose the new repl master...
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:46] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:47] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:48] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:49] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:50] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:51] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:52] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:53] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:54] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:55] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:57] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:58] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:13:59] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:00] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:01] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:02] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:03] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:04] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:05] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:06] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:07] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:08] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:09] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:10] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:11] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:12] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:13] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:14] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:15] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:16] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:17] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:18] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:19] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:20] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:21] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:22] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:23] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:24] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:25] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:26] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:27] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:29] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:30] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:31] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:32] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:33] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:34] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:35] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:36] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:37] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:38] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:39] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:40] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:41] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:42] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:43] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:44] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:45] root  New repl master has not been chosen yet
      [02/Sep/2020 19:14:46] root  Maxscale has not reconfigured repl master, continuing...
      [02/Sep/2020 19:14:46] root  _move_primary_node(): dbroot 1 is assigned to ('mcs1', '172.18.0.4')
      172.18.0.4 - - [02/Sep/2020 19:14:46] cmapi_server DEBUG    put_config starts
      [02/Sep/2020 19:14:46] root   -- Matching against ModuleIPAddr1-1-3, which says 172.18.0.4
      [02/Sep/2020 19:14:46] root  Wrote 'pm1' to /var/lib/columnstore/local/module
      [02/Sep/2020 19:14:46] root  apply Running stop on mcs-dmlproc. With sudo False and is_primary True
      [02/Sep/2020 19:14:46] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-dmlproc 1
      [02/Sep/2020 19:14:46] root  apply Running stop on mcs-ddlproc. With sudo False and is_primary True
      [02/Sep/2020 19:14:46] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-ddlproc 1
      [02/Sep/2020 19:14:46] root  apply Running stop on mcs-primproc. With sudo False and is_primary True
      [02/Sep/2020 19:14:46] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-primproc 1
      [02/Sep/2020 19:14:46] root  apply Running stop on mcs-writeengineserver. With sudo False and is_primary True
      [02/Sep/2020 19:14:46] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-writeengineserver 1
      [02/Sep/2020 19:14:46] root  apply Running stop on mcs-exemgr. With sudo False and is_primary True
      [02/Sep/2020 19:14:46] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-exemgr 1
      [02/Sep/2020 19:14:46] root  apply Running stop on mcs-controllernode. With sudo False and is_primary True
      [02/Sep/2020 19:14:46] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-controllernode 1
      [02/Sep/2020 19:14:46] root  Loaded the config file, my name is mcs1
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns em from local storage.
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:14:48] "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" 200 3436 "" "python-requests/2.23.0"
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns journal from local storage.
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:14:48] "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" 200 - "" "python-requests/2.23.0"
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns vbbm from local storage.
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:14:48] "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" 200 12 "" "python-requests/2.23.0"
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns vss from local storage.
      172.18.0.3 - - [02/Sep/2020 19:14:48] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:14:48] "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" 200 8 "" "python-requests/2.23.0"
      [02/Sep/2020 19:14:50] root  apply Running stop on mcs-workernode. With sudo False and is_primary True
      [02/Sep/2020 19:14:50] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-workernode 1
      [02/Sep/2020 19:14:50] root  apply Running stop on mcs-storagemanager. With sudo False and is_primary True
      [02/Sep/2020 19:14:50] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-storagemanager 1
      [02/Sep/2020 19:14:50] root  apply Running start on mcs-workernode. With sudo False and is_primary True
      [02/Sep/2020 19:14:50] root  start running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh start mcs-workernode 1
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns em from local storage.
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:14:51] "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" 200 3436 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns journal from local storage.
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:14:51] "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" 200 - "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns vbbm from local storage.
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:14:51] "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" 200 12 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns vss from local storage.
      172.18.0.4 - - [02/Sep/2020 19:14:51] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:14:51] "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" 200 8 "" "python-requests/2.23.0"
      [02/Sep/2020 19:14:51] root  apply Waiting for all workernodes to come up before starting controllernode on the primary.
      [02/Sep/2020 19:14:51] root  apply Trying...
      [02/Sep/2020 19:14:52] root  apply Trying...
      [02/Sep/2020 19:14:52] root  apply Running start on mcs-controllernode. With sudo False and is_primary True
      [02/Sep/2020 19:14:53] root  start running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh start mcs-controllernode 1
      [02/Sep/2020 19:14:57] root  apply Running start on mcs-primproc. With sudo False and is_primary True
      [02/Sep/2020 19:14:57] root  apply Running start on mcs-exemgr. With sudo False and is_primary True
      [02/Sep/2020 19:14:57] root  apply Running start on mcs-writeengineserver. With sudo False and is_primary True
      [02/Sep/2020 19:14:57] root  apply Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
      [02/Sep/2020 19:14:57] root  apply Trying...
      [02/Sep/2020 19:14:57] root  apply Running start on mcs-dmlproc. With sudo False and is_primary True
      [02/Sep/2020 19:14:57] root  apply Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
      [02/Sep/2020 19:14:57] root  apply Trying...
      [02/Sep/2020 19:14:57] root  apply Running start on mcs-ddlproc. With sudo False and is_primary True
      172.18.0.4 - - [02/Sep/2020:19:14:57] "PUT /cmapi/0.4.0/node/config HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [02/Sep/2020 19:14:57] root  broadcast_new_config(): successfully pushed the new config file to ['mcs2', 'mcs1']
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_commit starts
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_commit returns {'timestamp': '2020-09-02 19:14:57.142281'}
      172.18.0.4 - - [02/Sep/2020:19:14:57] "PUT /cmapi/0.4.0/node/commit HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_add_node returns {'timestamp': '2020-09-02 19:13:46.062222', 'node_id': 'mcs2'}
      172.18.0.4 - - [02/Sep/2020:19:14:57] "PUT /cmapi/0.4.0/cluster/add-node HTTP/1.1" 200 62 "" "curl/7.61.1"
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_add_node starts
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_begin starts
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_begin JSON body {'id': 356266, 'timeout': 299}
      172.18.0.4 - - [02/Sep/2020 19:14:57] cmapi_server DEBUG    put_begin returns {'timestamp': '2020-09-02 19:14:57.262050'}
      172.18.0.4 - - [02/Sep/2020:19:14:57] "PUT /cmapi/0.4.0/node/begin HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [02/Sep/2020 19:14:57] root  _add_Module_entries(): using ip address 172.18.0.2 and hostname mcs3
      [02/Sep/2020 19:14:57] root  Waiting for Maxscale to choose the new repl master...
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:57] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:58] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:14:59] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:00] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:01] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:02] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:03] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:04] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:05] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:06] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:07] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:08] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:09] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:10] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:11] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:12] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:13] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:14] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:15] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:16] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:17] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:18] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:19] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:21] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:22] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:23] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:24] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:25] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:26] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:27] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:28] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:29] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:30] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:31] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:32] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:33] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:34] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:35] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:36] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:37] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:38] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:39] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:40] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:41] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:42] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:43] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:44] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:45] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:46] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:47] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:48] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:49] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:50] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:51] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:52] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:54] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:55] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:56] root  New repl master has not been chosen yet
      ERROR 1698 (28000): Access denied for user 'idbrep'@'mcs1'
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      ERROR 1045 (28000): Access denied for user 'idbrep'@'mcs1.columnstore-docker-cluster_default' (using password: YES)
      [02/Sep/2020 19:15:57] root  New repl master has not been chosen yet
      [02/Sep/2020 19:15:58] root  Maxscale has not reconfigured repl master, continuing...
      [02/Sep/2020 19:15:58] root  _move_primary_node(): dbroot 1 is assigned to ('mcs1', '172.18.0.4')
      172.18.0.4 - - [02/Sep/2020 19:15:58] cmapi_server DEBUG    put_config starts
      [02/Sep/2020 19:15:58] root   -- Matching against ModuleIPAddr1-1-3, which says 172.18.0.4
      [02/Sep/2020 19:15:58] root  Wrote 'pm1' to /var/lib/columnstore/local/module
      [02/Sep/2020 19:15:58] root  apply Running stop on mcs-dmlproc. With sudo False and is_primary True
      [02/Sep/2020 19:15:58] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-dmlproc 1
      [02/Sep/2020 19:15:58] root  apply Running stop on mcs-ddlproc. With sudo False and is_primary True
      [02/Sep/2020 19:15:58] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-ddlproc 1
      [02/Sep/2020 19:15:58] root  apply Running stop on mcs-primproc. With sudo False and is_primary True
      [02/Sep/2020 19:15:58] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-primproc 1
      [02/Sep/2020 19:15:58] root  apply Running stop on mcs-writeengineserver. With sudo False and is_primary True
      [02/Sep/2020 19:15:58] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-writeengineserver 1
      [02/Sep/2020 19:15:58] root  apply Running stop on mcs-exemgr. With sudo False and is_primary True
      [02/Sep/2020 19:15:58] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-exemgr 1
      [02/Sep/2020 19:15:58] root  apply Running stop on mcs-controllernode. With sudo False and is_primary True
      [02/Sep/2020 19:15:58] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-controllernode 1
      [02/Sep/2020 19:15:58] root  Loaded the config file, my name is mcs1
      [02/Sep/2020 19:15:58] root  Failover support is active, monitoring nodes ['mcs1', 'mcs2', 'mcs3']
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns em from local storage.
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.2 - - [02/Sep/2020:19:16:00] "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" 200 3436 "" "python-requests/2.23.0"
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns journal from local storage.
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.2 - - [02/Sep/2020:19:16:00] "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" 200 - "" "python-requests/2.23.0"
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns vbbm from local storage.
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.2 - - [02/Sep/2020:19:16:00] "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" 200 12 "" "python-requests/2.23.0"
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns vss from local storage.
      172.18.0.2 - - [02/Sep/2020 19:16:00] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.2 - - [02/Sep/2020:19:16:00] "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" 200 8 "" "python-requests/2.23.0"
      [02/Sep/2020 19:16:02] root  apply Running stop on mcs-workernode. With sudo False and is_primary True
      [02/Sep/2020 19:16:02] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-workernode 1
      [02/Sep/2020 19:16:02] root  apply Running stop on mcs-storagemanager. With sudo False and is_primary True
      [02/Sep/2020 19:16:02] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-storagemanager 1
      [02/Sep/2020 19:16:02] root  apply Running start on mcs-workernode. With sudo False and is_primary True
      [02/Sep/2020 19:16:02] root  start running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh start mcs-workernode 1
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns em from local storage.
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" 200 3436 "" "python-requests/2.23.0"
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns journal from local storage.
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" 200 - "" "python-requests/2.23.0"
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns vbbm from local storage.
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" 200 12 "" "python-requests/2.23.0"
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns vss from local storage.
      172.18.0.3 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.3 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" 200 8 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns em from local storage.
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/em HTTP/1.1" 200 3436 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns journal from local storage.
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/journal HTTP/1.1" 200 - "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns vbbm from local storage.
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/vbbm HTTP/1.1" 200 12 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes starts
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns vss from local storage.
      172.18.0.4 - - [02/Sep/2020 19:16:03] cmapi_server DEBUG    get_brm_bytes returns.
      172.18.0.4 - - [02/Sep/2020:19:16:03] "GET /cmapi/0.4.0/node/meta/vss HTTP/1.1" 200 8 "" "python-requests/2.23.0"
      [02/Sep/2020 19:16:03] root  apply Waiting for all workernodes to come up before starting controllernode on the primary.
      [02/Sep/2020 19:16:03] root  apply Trying...
      [02/Sep/2020 19:16:04] root  apply Trying...
      [02/Sep/2020 19:16:04] root  apply Running start on mcs-controllernode. With sudo False and is_primary True
      [02/Sep/2020 19:16:04] root  start running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh start mcs-controllernode 1
      [02/Sep/2020 19:16:08] root  apply Running start on mcs-primproc. With sudo False and is_primary True
      [02/Sep/2020 19:16:08] root  apply Running start on mcs-exemgr. With sudo False and is_primary True
      [02/Sep/2020 19:16:08] root  apply Running start on mcs-writeengineserver. With sudo False and is_primary True
      [02/Sep/2020 19:16:08] root  apply Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
      [02/Sep/2020 19:16:08] root  apply Trying...
      [02/Sep/2020 19:16:08] root  apply Running start on mcs-dmlproc. With sudo False and is_primary True
      [02/Sep/2020 19:16:08] root  apply Waiting for controllernode to come up before starting ddlproc/dmlproc on non-primary nodes.
      [02/Sep/2020 19:16:08] root  apply Trying...
      [02/Sep/2020 19:16:08] root  apply Running start on mcs-ddlproc. With sudo False and is_primary True
      172.18.0.4 - - [02/Sep/2020:19:16:08] "PUT /cmapi/0.4.0/node/config HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [02/Sep/2020 19:16:09] root  broadcast_new_config(): successfully pushed the new config file to ['mcs3', 'mcs1', 'mcs2']
      172.18.0.4 - - [02/Sep/2020 19:16:09] cmapi_server DEBUG    put_commit starts
      172.18.0.4 - - [02/Sep/2020 19:16:09] cmapi_server DEBUG    put_commit returns {'timestamp': '2020-09-02 19:16:09.704918'}
      172.18.0.4 - - [02/Sep/2020:19:16:09] "PUT /cmapi/0.4.0/node/commit HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      172.18.0.4 - - [02/Sep/2020 19:16:09] cmapi_server DEBUG    put_add_node returns {'timestamp': '2020-09-02 19:14:57.238878', 'node_id': 'mcs3'}
      172.18.0.4 - - [02/Sep/2020:19:16:09] "PUT /cmapi/0.4.0/cluster/add-node HTTP/1.1" 200 62 "" "curl/7.61.1"
      

      Attachments

        Activity

          People

            toddstoffel Todd Stoffel (Inactive)
            jens.rowekamp Jens Röwekamp (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.