Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4260

cmapi_server doesn't work with FQDN in k8s

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Blocker
    • Resolution: Not a Bug
    • 1.5.3
    • N/A
    • cmapi
    • Kubernetes

      image: mariadb/skysql-enterprise-columnstore-handoff:cluster-dev
      digest: 02b7298cadbd822c88de9db04f34c37f33561cd8ef0c6800acd97fc6bb9d10bb

    Description

      Adding the first node to a ColumnStore cluster through the cmapi server in k8s using its fully qualified domain name fails.

      The same steps however succeed if localhost is used instead of the FQDN.

      localhost works

      [root@cs-test-mdb-cs-node-0 /]# ping localhost -c2
      PING localhost (127.0.0.1) 56(84) bytes of data.
      64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.017 ms
      64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.029 ms
       
      --- localhost ping statistics ---
      2 packets transmitted, 2 received, 0% packet loss, time 54ms
      rtt min/avg/max/mdev = 0.017/0.023/0.029/0.006 ms
      

      [root@cs-test-mdb-cs-node-0 /]# curl -s https://localhost:8640/cmapi/0.4.0/cluster/status --header 'Content-Type:application/json' --header "x-api-key:somekey123" -k | jq .
      {
        "timestamp": "2020-08-20 18:02:44.320358",
        "num_nodes": 0
      }
      

      [root@cs-test-mdb-cs-node-0 /]# curl -s -X PUT "https://localhost:8640/cmapi/0.4.0/cluster/add-node" --header 'Content-Type:application/json' --header "x-api-key:somekey123" --data '{"timeout":60, "node": "localhost"}' -k | jq .
      {
        "timestamp": "2020-08-20 18:02:49.877532",
        "node_id": "localhost"
      }
      

      [root@cs-test-mdb-cs-node-0 /]# curl -s https://localhost:8640/cmapi/0.4.0/cluster/status --header 'Content-Type:application/json' --header "x-api-key:somekey123" -k | jq .
      {
        "timestamp": "2020-08-20 18:03:57.401819",
        "localhost": {
          "timestamp": "2020-08-20 18:03:57.408225",
          "uptime": 1379,
          "dbrm_mode": "master",
          "cluster_mode": "readwrite",
          "dbroots": [
            "1"
          ],
          "module_id": 1,
          "services": [
            {
              "name": "workernode",
              "pid": 1215
            },
            {
              "name": "controllernode",
              "pid": 1272
            },
            {
              "name": "PrimProc",
              "pid": 1275
            },
            {
              "name": "ExeMgr",
              "pid": 1300
            },
            {
              "name": "WriteEngine",
              "pid": 1303
            },
            {
              "name": "DMLProc",
              "pid": 1319
            },
            {
              "name": "DDLProc",
              "pid": 1322
            }
          ]
        },
        "num_nodes": 1
      }
      [root@cs-test-mdb-cs-node-0 /]# exit
      

      FQDN cs-test-mdb-cs-node-0.cs-test-mdb-clust doesn't work

      dev-jump:~ $ kubectl logs cs-test-mdb-cs-node-0
      Starting...
      CMAPI PID = 848
      Starting MariaDB.200820 18:06:24 mysqld_safe Logging to '/var/log/mariadb/columnstore/mariadb-error.log'.
      200820 18:06:24 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql
       SUCCESS!
       New Monit id: 6290541950c79652f3904ecad7f4f33c
       Stored in '/root/.monit.id'
      Starting Monit 5.26.0 daemon with http interface at [localhost]:2812
      Monit will delay for 60s on first start after reboot ...
      'cs-test-mdb-cs-node-0' Monit 5.26.0 started
      

      dev-jump:~ $ kubectl exec -it cs-test-mdb-cs-node-0 -- bash
      [root@cs-test-mdb-cs-node-0 /]# ps -aux
      USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
      root           1  0.0  0.0   4340   796 ?        Ss   18:05   0:00 /usr/bin/tini -- docker-entrypoint.sh /bin/sh -c cmapi-start && monit -I
      root           7  0.0  0.0  14816  3300 ?        S    18:05   0:00 /bin/bash /usr/bin/docker-entrypoint.sh /bin/sh -c cmapi-start && monit -I
      root          10  0.4  0.0 197144  2372 ?        Ssl  18:05   0:00 rsyslogd
      root          12  0.0  0.0 109444  5280 ?        Sl   18:05   0:00 monit -I
      root         848  2.6  0.6 1211824 51116 ?       Sl   18:06   0:02 /opt/cmapi/python/bin/python3 -m cmapi_server
      root         905  0.0  0.0  15080  3444 ?        S    18:06   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/cs-test-mdb-cs-node-0.pid
      mysql       1035  0.1  1.3 1700728 99968 ?       Sl   18:06   0:00 /usr/sbin/mariadbd --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/log/mariadb/columnstore/mariadb-error.log --
      root        1092  1.0  0.0  15052  3728 pts/0    Ss   18:07   0:00 bash
      root        1107  0.0  0.0  50436  3884 pts/0    R+   18:07   0:00 ps -aux
      

      [root@cs-test-mdb-cs-node-0 /]# ping cs-test-mdb-cs-node-0.cs-test-mdb-clust -c2
      PING cs-test-mdb-cs-node-0.cs-test-mdb-clust.db00005735.svc.cluster.local (10.0.0.166) 56(84) bytes of data.
      64 bytes from cs-test-mdb-cs-node-0.cs-test-mdb-clust.db00005735.svc.cluster.local (10.0.0.166): icmp_seq=1 ttl=64 time=0.014 ms
      64 bytes from cs-test-mdb-cs-node-0.cs-test-mdb-clust.db00005735.svc.cluster.local (10.0.0.166): icmp_seq=2 ttl=64 time=0.038 ms
       
      --- cs-test-mdb-cs-node-0.cs-test-mdb-clust.db00005735.svc.cluster.local ping statistics ---
      2 packets transmitted, 2 received, 0% packet loss, time 12ms
      rtt min/avg/max/mdev = 0.014/0.026/0.038/0.012 ms
      

      [root@cs-test-mdb-cs-node-0 /]# curl -s https://cs-test-mdb-cs-node-0.cs-test-mdb-clust:8640/cmapi/0.4.0/cluster/status --header 'Content-Type:application/json' --header "x-api-key:somekey123" -k | jq .
      {
        "timestamp": "2020-08-20 18:08:06.374310",
        "num_nodes": 0
      }
      

      [root@cs-test-mdb-cs-node-0 /]# curl -s -X PUT "https://cs-test-mdb-cs-node-0.cs-test-mdb-clust:8640/cmapi/0.4.0/cluster/add-node" --header 'Content-Type:application/json' --header "x-api-key:somekey123" --data '{"timeout":60, "node": "cs-test-mdb-cs-node-0.cs-test-mdb-clust"}' -k | jq .
      {
        "timestamp": "2020-08-20 18:08:12.795427",
        "node_id": "cs-test-mdb-cs-node-0.cs-test-mdb-clust"
      }
      

      [root@cs-test-mdb-cs-node-0 /]# sleep 120
      

      [root@cs-test-mdb-cs-node-0 /]# curl -s https://cs-test-mdb-cs-node-0.cs-test-mdb-clust:8640/cmapi/0.4.0/cluster/status --header 'Content-Type:application/json' --header "x-api-key:somekey123" -k | jq .
      {
        "timestamp": "2020-08-20 18:10:41.507126",
        "cs-test-mdb-cs-node-0.cs-test-mdb-clust": {
          "timestamp": "2020-08-20 18:10:41.515981",
          "uptime": 1783,
          "dbrm_mode": "offline",
          "cluster_mode": "readonly",
          "dbroots": [],
          "module_id": 1,
          "services": []
        },
        "num_nodes": 1
      }
      

      [root@cs-test-mdb-cs-node-0 /]# for i in $(ls /var/log/mariadb/columnstore/); do
      >     echo "------"
      > echo $i
      > cat /var/log/mariadb/columnstore/$i
      > echo
      > done
      ------
      archive
      cat: /var/log/mariadb/columnstore/archive: Is a directory
       
      ------
      cmapi_server.log
      127.0.0.1 - - [20/Aug/2020 18:06:25] cmapi_server DEBUG    Using container dispatcher module
      127.0.0.1 - - [20/Aug/2020 18:06:25] cmapi_server INFO     CMAPI Version: 1.5.3.1
      [20/Aug/2020 18:06:25] root  get_module_net_address Module 1 network address 127.0.0.1
      [20/Aug/2020 18:06:25] root  Loaded the config file, my name is None
      [20/Aug/2020:18:06:25] ENGINE Bus STARTING
      [20/Aug/2020 18:06:25] root  Starting the heartbeat listener
      [20/Aug/2020 18:06:25] root  starting the monitor logic
      [20/Aug/2020:18:06:25] ENGINE Serving on https://0.0.0.0:8640
      [20/Aug/2020:18:06:25] ENGINE Bus STARTED
      [20/Aug/2020 18:06:26] root  Failover monitoring is inactive; requires at least 3 nodes and a shared storage system
      10.0.0.166 - - [20/Aug/2020 18:08:06] cmapi_server DEBUG    get_status starts
      10.0.0.166 - - [20/Aug/2020 18:08:06] cmapi_server DEBUG    get_status returns {'timestamp': '2020-08-20 18:08:06.374310', 'num_nodes': 0}
      10.0.0.166 - - [20/Aug/2020:18:08:06] "GET /cmapi/0.4.0/cluster/status HTTP/1.1" 200 59 "" "curl/7.61.1"
      10.0.0.166 - - [20/Aug/2020 18:08:12] cmapi_server DEBUG    put_add_node starts
      10.0.0.166 - - [20/Aug/2020 18:08:12] cmapi_server DEBUG    put_begin starts
      10.0.0.166 - - [20/Aug/2020 18:08:12] cmapi_server DEBUG    put_begin JSON body {'id': 26562, 'timeout': 299}
      10.0.0.166 - - [20/Aug/2020 18:08:12] cmapi_server DEBUG    put_begin returns {'timestamp': '2020-08-20 18:08:12.806544'}
      10.0.0.166 - - [20/Aug/2020:18:08:12] "PUT /cmapi/0.4.0/node/begin HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [20/Aug/2020 18:08:12] root  get_module_net_address Module 1 network address 127.0.0.1
      [20/Aug/2020 18:08:12] root  add_node(): replacing 127.0.0.1/localhost with 10.0.0.166/cs-test-mdb-cs-node-0.cs-test-mdb-clust as this node's name. Be sure cs-test-mdb-cs-node-0.cs-test-mdb-clust resolves to 10.0.0.166 on all other nodes in the cluster.
      10.0.0.166 - - [20/Aug/2020 18:08:12] cmapi_server DEBUG    put_config starts
      10.0.0.166 - - [20/Aug/2020 18:08:12] cmapi_server DEBUG    put_config JSON body {'manager': 'cs-test-mdb-cs-node-0.cs-test-mdb-clust', 'revision': '1', 'timeout': 300, 'config': '<?xml version="1.0" ?>\n<Columnstore Version="V1.0.0">\n    <ExeMgr1>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8601</Port>\n        <Module>unassigned</Module>\n    </ExeMgr1>\n    <JobProc>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8602</Port>\n    </JobProc>\n    <ProcMgr>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8603</Port>\n    </ProcMgr>\n    <ProcMgr_Alarm>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8606</Port>\n    </ProcMgr_Alarm>\n    <ProcStatusControl>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8604</Port>\n    </ProcStatusControl>\n    <ProcStatusControlStandby>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8605</Port>\n    </ProcStatusControlStandby>\n    <localhost_ProcessMonitor>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8800</Port>\n    </localhost_ProcessMonitor>\n    <dm1_ProcessMonitor>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8800</Port>\n    </dm1_ProcessMonitor>\n    <um1_ProcessMonitor>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8800</Port>\n    </um1_ProcessMonitor>\n    <pm1_ProcessMonitor>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8800</Port>\n    </pm1_ProcessMonitor>\n    <dm1_ServerMonitor>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8622</Port>\n    </dm1_ServerMonitor>\n    <um1_ServerMonitor>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8622</Port>\n    </um1_ServerMonitor>\n    <pm1_ServerMonitor>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8622</Port>\n    </pm1_ServerMonitor>\n    <pm1_WriteEngineServer>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8630</Port>\n    </pm1_WriteEngineServer>\n    <DDLProc>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8612</Port>\n    </DDLProc>\n    <DMLProc>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8614</Port>\n    </DMLProc>\n    <BatchInsert>\n        <RowsPerBatch>10000</RowsPerBatch>\n    </BatchInsert>\n    <PrimitiveServers>\n        <Count>1</Count>\n        <ConnectionsPerPrimProc>2</ConnectionsPerPrimProc>\n        <ProcessorThreshold>128</ProcessorThreshold>\n        <ProcessorQueueSize>10K</ProcessorQueueSize>\n        <DebugLevel>0</DebugLevel>\n        <LBID_Shift>13</LBID_Shift>\n        <ColScanBufferSizeBlocks>512</ColScanBufferSizeBlocks>\n        <ColScanReadAheadBlocks>512</ColScanReadAheadBlocks>\n        <PrefetchThreshold>1</PrefetchThreshold>\n        <PTTrace>0</PTTrace>\n        <RotatingDestination>n</RotatingDestination>\n        <DirectIO>y</DirectIO>\n        <HighPriorityPercentage/>\n        <MediumPriorityPercentage/>\n        <LowPriorityPercentage/>\n    </PrimitiveServers>\n    <PMS1>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS1>\n    <PMS2>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS2>\n    <PMS3>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS3>\n    <PMS4>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS4>\n    <PMS5>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS5>\n    <PMS6>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS6>\n    <PMS7>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS7>\n    <PMS8>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS8>\n    <PMS9>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS9>\n    <PMS10>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS10>\n    <PMS11>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS11>\n    <PMS12>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS12>\n    <PMS13>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS13>\n    <PMS14>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS14>\n    <PMS15>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS15>\n    <PMS16>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS16>\n    <PMS17>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS17>\n    <PMS18>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS18>\n    <PMS19>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS19>\n    <PMS20>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS20>\n    <PMS21>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS21>\n    <PMS22>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS22>\n    <PMS23>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS23>\n    <PMS24>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS24>\n    <PMS25>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS25>\n    <PMS26>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS26>\n    <PMS27>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS27>\n    <PMS28>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS28>\n    <PMS29>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS29>\n    <PMS30>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS30>\n    <PMS31>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS31>\n    <PMS32>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8620</Port>\n    </PMS32>\n    <SystemConfig>\n        <SystemName>columnstore-1</SystemName>\n        <ParentOAMModuleName>pm1</ParentOAMModuleName>\n        <StandbyOAMModuleName>unassigned</StandbyOAMModuleName>\n        <PrimaryUMModuleName>pm1</PrimaryUMModuleName>\n        <ModuleHeartbeatPeriod>1</ModuleHeartbeatPeriod>\n        <ModuleHeartbeatCount>3</ModuleHeartbeatCount>\n        <ModuleProcMonWaitCount>12</ModuleProcMonWaitCount>\n        \t// 2.5 minutes\n        <DBRootCount>1</DBRootCount>\n        <DBRoot1>/var/lib/columnstore/data1</DBRoot1>\n        <DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>\n        <TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>\n        <DBRMTimeOut>15</DBRMTimeOut>\n        <DBRMSnapshotInterval>100000</DBRMSnapshotInterval>\n        <ExternalCriticalThreshold>90</ExternalCriticalThreshold>\n        <ExternalMajorThreshold>80</ExternalMajorThreshold>\n        <ExternalMinorThreshold>70</ExternalMinorThreshold>\n        <TransactionArchivePeriod>10</TransactionArchivePeriod>\n        <NMSIPAddress>0.0.0.0</NMSIPAddress>\n        <TempSaveSize>128M</TempSaveSize>\n        <WaitPeriod>10</WaitPeriod>\n        <ProcessRestartCount>10</ProcessRestartCount>\n        <ProcessRestartPeriod>120</ProcessRestartPeriod>\n        <SwapAction>restartSystem</SwapAction>\n        <ActivePmFailoverDisabled>n</ActivePmFailoverDisabled>\n        <MemoryCheckPercent>95</MemoryCheckPercent>\n        <DataFileLog>OFF</DataFileLog>\n        <hdfsRdwrScratch>/rdwrscratch</hdfsRdwrScratch>\n        <TempFileDir>/columnstore_tmp_files</TempFileDir>\n        <SystemTempFileDir>/tmp/columnstore_tmp_files</SystemTempFileDir>\n        <DataFilePlugin>libcloudio.so</DataFilePlugin>\n    </SystemConfig>\n    <SystemModuleConfig>\n        <ModuleType1>dm</ModuleType1>\n        <ModuleDesc1>Director Module</ModuleDesc1>\n        <RunType1>SIMPLEX</RunType1>\n        <ModuleCount1>0</ModuleCount1>\n        <ModuleIPAddr1-1-1>0.0.0.0</ModuleIPAddr1-1-1>\n        <ModuleHostName1-1-1>unassigned</ModuleHostName1-1-1>\n        <ModuleDisableState1-1>ENABLED</ModuleDisableState1-1>\n        <ModuleCPUCriticalThreshold1>0</ModuleCPUCriticalThreshold1>\n        <ModuleCPUMajorThreshold1>0</ModuleCPUMajorThreshold1>\n        <ModuleCPUMinorThreshold1>0</ModuleCPUMinorThreshold1>\n        <ModuleCPUMinorClearThreshold1>0</ModuleCPUMinorClearThreshold1>\n        <ModuleDiskCriticalThreshold1>90</ModuleDiskCriticalThreshold1>\n        <ModuleDiskMajorThreshold1>80</ModuleDiskMajorThreshold1>\n        <ModuleDiskMinorThreshold1>70</ModuleDiskMinorThreshold1>\n        <ModuleMemCriticalThreshold1>90</ModuleMemCriticalThreshold1>\n        <ModuleMemMajorThreshold1>0</ModuleMemMajorThreshold1>\n        <ModuleMemMinorThreshold1>0</ModuleMemMinorThreshold1>\n        <ModuleSwapCriticalThreshold1>90</ModuleSwapCriticalThreshold1>\n        <ModuleSwapMajorThreshold1>80</ModuleSwapMajorThreshold1>\n        <ModuleSwapMinorThreshold1>70</ModuleSwapMinorThreshold1>\n        <ModuleDiskMonitorFileSystem1-1>/</ModuleDiskMonitorFileSystem1-1>\n        <ModuleDBRootCount1-1>unassigned</ModuleDBRootCount1-1>\n        <ModuleDBRootID1-1-1>unassigned</ModuleDBRootID1-1-1>\n        <ModuleType2>um</ModuleType2>\n        <ModuleDesc2>User Module</ModuleDesc2>\n        <RunType2>SIMPLEX</RunType2>\n        <ModuleCount2>0</ModuleCount2>\n        <ModuleIPAddr1-1-2>0.0.0.0</ModuleIPAddr1-1-2>\n        <ModuleHostName1-1-2>unassigned</ModuleHostName1-1-2>\n        <ModuleDisableState1-2>ENABLED</ModuleDisableState1-2>\n        <ModuleCPUCriticalThreshold2>0</ModuleCPUCriticalThreshold2>\n        <ModuleCPUMajorThreshold2>0</ModuleCPUMajorThreshold2>\n        <ModuleCPUMinorThreshold2>0</ModuleCPUMinorThreshold2>\n        <ModuleCPUMinorClearThreshold2>0</ModuleCPUMinorClearThreshold2>\n        <ModuleDiskCriticalThreshold2>90</ModuleDiskCriticalThreshold2>\n        <ModuleDiskMajorThreshold2>80</ModuleDiskMajorThreshold2>\n        <ModuleDiskMinorThreshold2>70</ModuleDiskMinorThreshold2>\n        <ModuleMemCriticalThreshold2>90</ModuleMemCriticalThreshold2>\n        <ModuleMemMajorThreshold2>0</ModuleMemMajorThreshold2>\n        <ModuleMemMinorThreshold2>0</ModuleMemMinorThreshold2>\n        <ModuleSwapCriticalThreshold2>90</ModuleSwapCriticalThreshold2>\n        <ModuleSwapMajorThreshold2>80</ModuleSwapMajorThreshold2>\n        <ModuleSwapMinorThreshold2>70</ModuleSwapMinorThreshold2>\n        <ModuleDiskMonitorFileSystem1-2>/</ModuleDiskMonitorFileSystem1-2>\n        <ModuleDBRootCount1-2>unassigned</ModuleDBRootCount1-2>\n        <ModuleDBRootID1-1-2>unassigned</ModuleDBRootID1-1-2>\n        <ModuleType3>pm</ModuleType3>\n        <ModuleDesc3>Performance Module</ModuleDesc3>\n        <RunType3>SIMPLEX</RunType3>\n        <ModuleCount3>1</ModuleCount3>\n        <ModuleIPAddr1-1-3>10.0.0.166</ModuleIPAddr1-1-3>\n        <ModuleHostName1-1-3>cs-test-mdb-cs-node-0.cs-test-mdb-clust</ModuleHostName1-1-3>\n        <ModuleDisableState1-3>ENABLED</ModuleDisableState1-3>\n        <ModuleCPUCriticalThreshold3>0</ModuleCPUCriticalThreshold3>\n        <ModuleCPUMajorThreshold3>0</ModuleCPUMajorThreshold3>\n        <ModuleCPUMinorThreshold3>0</ModuleCPUMinorThreshold3>\n        <ModuleCPUMinorClearThreshold3>0</ModuleCPUMinorClearThreshold3>\n        <ModuleDiskCriticalThreshold3>90</ModuleDiskCriticalThreshold3>\n        <ModuleDiskMajorThreshold3>80</ModuleDiskMajorThreshold3>\n        <ModuleDiskMinorThreshold3>70</ModuleDiskMinorThreshold3>\n        <ModuleMemCriticalThreshold3>90</ModuleMemCriticalThreshold3>\n        <ModuleMemMajorThreshold3>0</ModuleMemMajorThreshold3>\n        <ModuleMemMinorThreshold3>0</ModuleMemMinorThreshold3>\n        <ModuleSwapCriticalThreshold3>90</ModuleSwapCriticalThreshold3>\n        <ModuleSwapMajorThreshold3>80</ModuleSwapMajorThreshold3>\n        <ModuleSwapMinorThreshold3>70</ModuleSwapMinorThreshold3>\n        <ModuleDiskMonitorFileSystem1-3>/</ModuleDiskMonitorFileSystem1-3>\n        <ModuleDBRootCount1-3>1</ModuleDBRootCount1-3>\n        <ModuleDBRootID1-1-3>1</ModuleDBRootID1-1-3>\n    </SystemModuleConfig>\n    <SystemExtDeviceConfig>\n        <Count>0</Count>\n        <Name1>unassigned</Name1>\n        <IPAddr1>0.0.0.0</IPAddr1>\n        <DisableState1>ENABLED</DisableState1>\n    </SystemExtDeviceConfig>\n    <SessionManager>\n        <MaxConcurrentTransactions>1000</MaxConcurrentTransactions>\n        <TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>\n    </SessionManager>\n    <VersionBuffer>\n        <VersionBufferFileSize>1GB</VersionBufferFileSize>\n    </VersionBuffer>\n    <OIDManager>\n        <OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>\n        <FirstOID>3000</FirstOID>\n    </OIDManager>\n    <WriteEngine>\n        <BulkRoot>/var/lib/columnstore/data/bulk</BulkRoot>\n        <BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>\n        <MaxFileSystemDiskUsagePct>98</MaxFileSystemDiskUsagePct>\n        <CompressedPaddingBlocks>1</CompressedPaddingBlocks>\n    </WriteEngine>\n    <DBRM_Controller>\n        <NumWorkers>1</NumWorkers>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8616</Port>\n    </DBRM_Controller>\n    <DBRM_Worker1>\n        <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>\n        <Port>8700</Port>\n        <Module>pm1</Module>\n    </DBRM_Worker1>\n    <DBRM_Worker2>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker2>\n    <DBRM_Worker3>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker3>\n    <DBRM_Worker4>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker4>\n    <DBRM_Worker5>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker5>\n    <DBRM_Worker6>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker6>\n    <DBRM_Worker7>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker7>\n    <DBRM_Worker8>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker8>\n    <DBRM_Worker9>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker9>\n    <DBRM_Worker10>\n        <IPAddr>0.0.0.0</IPAddr>\n        <Port>8700</Port>\n        <Module>unassigned</Module>\n    </DBRM_Worker10>\n    <DBBC>\n        <NumCaches>1</NumCaches>\n        <IOMTracing>0</IOMTracing>\n        <BRPTracing>0</BRPTracing>\n        <ReportFrequency>65536</ReportFrequency>\n        <MaxOpenFiles>2K</MaxOpenFiles>\n        <DecreaseOpenFilesCount>200</DecreaseOpenFilesCount>\n        <FDCacheTrace>0</FDCacheTrace>\n        <NumBlocksPct>50</NumBlocksPct>\n    </DBBC>\n    <Installation>\n        <SystemStartupOffline>n</SystemStartupOffline>\n        <InitialInstallFlag>y</InitialInstallFlag>\n        <SingleServerInstall>y</SingleServerInstall>\n        <ServerTypeInstall>2</ServerTypeInstall>\n        <PMwithUM>n</PMwithUM>\n        <MySQLRep>n</MySQLRep>\n        <DBRootStorageType>StorageManager</DBRootStorageType>\n        <UMStorageType>internal</UMStorageType>\n        <EEPackageType>binary</EEPackageType>\n        <SystemLogConfigFile>/etc/rsyslog.d/columnstore.conf</SystemLogConfigFile>\n        <Cloud>unassigned</Cloud>\n        <AmazonVPCNextPrivateIP>autoassign</AmazonVPCNextPrivateIP>\n        <UMInstanceType>unassigned</UMInstanceType>\n        <UMSecurityGroup>unassigned</UMSecurityGroup>\n        <UMVolumeSize>unassigned</UMVolumeSize>\n        <UMVolumeType>gp2</UMVolumeType>\n        <UMVolumeIOPS>unassigned</UMVolumeIOPS>\n        <PMInstanceType>unassigned</PMInstanceType>\n        <PMVolumeSize>unassigned</PMVolumeSize>\n        <PMVolumeType>gp2</PMVolumeType>\n        <PMVolumeIOPS>unassigned</PMVolumeIOPS>\n        <AmazonPMFailover>y</AmazonPMFailover>\n        <AmazonAutoTagging>y</AmazonAutoTagging>\n        <AmazonElasticIPCount>0</AmazonElasticIPCount>\n        <AmazonElasticIPAddr1>unassigned</AmazonElasticIPAddr1>\n        <AmazonElasticModule1>unassigned</AmazonElasticModule1>\n        <DataRedundancyConfig>n</DataRedundancyConfig>\n        <DataRedundancyCopies>0</DataRedundancyCopies>\n        <DataRedundancyStorageType>unassigned</DataRedundancyStorageType>\n        <CoreFileFlag>n</CoreFileFlag>\n        <MySQLPort>3306</MySQLPort>\n        <AmazonDeviceName>/dev/xvd</AmazonDeviceName>\n        <LockFileDirectory>/var/lock/subsys</LockFileDirectory>\n        <ProfileFile>/etc/profile.d/columnstoreAlias.sh</ProfileFile>\n        <DataRedundancyNetworkType/>\n    </Installation>\n    <ExtentMap>\n        <FilesPerColumnPartition>4</FilesPerColumnPartition>\n        <ExtentsPerSegmentFile>2</ExtentsPerSegmentFile>\n        <BRM_UID>0x0</BRM_UID>\n    </ExtentMap>\n    <HashJoin>\n        <MaxBuckets>128</MaxBuckets>\n        <MaxElems>128K</MaxElems>\n        <FifoSizeLargeSide>64</FifoSizeLargeSide>\n        <PmMaxMemorySmallSide>1G</PmMaxMemorySmallSide>\n        <TotalUmMemory>25%</TotalUmMemory>\n        <TotalPmUmMemory>100%</TotalPmUmMemory>\n        <CPUniqueLimit>100</CPUniqueLimit>\n        <AllowDiskBasedJoin>N</AllowDiskBasedJoin>\n        <TempFileCompression>Y</TempFileCompression>\n    </HashJoin>\n    <JobList>\n        <FlushInterval>16K</FlushInterval>\n        <FifoSize>16</FifoSize>\n        <RequestSize>1</RequestSize>\n        <ThreadPoolSize>100</ThreadPoolSize>\n    </JobList>\n    <TupleWSDL>\n        <MaxSize>1M</MaxSize>\n        <InitialCapacity>1M</InitialCapacity>\n        <MaxBuckets>512</MaxBuckets>\n    </TupleWSDL>\n    <RowAggregation>\n\t</RowAggregation>\n    <CrossEngineSupport>\n        <Host>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Host>\n        <Port>3306</Port>\n        <User>cej</User>\n        <Password>C0lumnStore!</Password>\n        <TLSCA/>\n        <TLSClientCert/>\n        <TLSClientKey/>\n    </CrossEngineSupport>\n    <QueryStats>\n        <Enabled>N</Enabled>\n    </QueryStats>\n    <UserPriority>\n        <Enabled>N</Enabled>\n    </UserPriority>\n    <NetworkCompression>\n        <Enabled>Y</Enabled>\n    </NetworkCompression>\n    <QueryTele>\n        <Host>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Host>\n        <Port>0</Port>\n    </QueryTele>\n    <StorageManager>\n        <MaxSockets>30</MaxSockets>\n        <Enabled>Y</Enabled>\n    </StorageManager>\n    <DataRedundancyConfig>\n        <DBRoot1PMs/>\n    </DataRedundancyConfig>\n    <ConfigRevision>1</ConfigRevision>\n    <ClusterManager>cs-test-mdb-cs-node-0.cs-test-mdb-clust</ClusterManager>\n    <ClusterName>MyCluster</ClusterName>\n    <NextNodeId>2</NextNodeId>\n    <NextDBRootId>2</NextDBRootId>\n    <DesiredNodes>\n        <Node>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Node>\n    </DesiredNodes>\n    <ActiveNodes>\n        <Node>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Node>\n    </ActiveNodes>\n    <InactiveNodes/>\n    <PrimaryNode>cs-test-mdb-cs-node-0.cs-test-mdb-clust</PrimaryNode>\n    <SharedStorage>yes</SharedStorage>\n</Columnstore>', 'cs_config_filename': '/etc/columnstore/Columnstore.xml'}
      [20/Aug/2020 18:08:12] root  This node is not in the current ActiveNodes section.  Not starting Columnstore.
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-dmlproc. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-dmlproc 0
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-ddlproc. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-ddlproc 0
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-primproc. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-primproc 0
      [20/Aug/2020 18:08:12] root  This node has no entry in the list of desired nodes
      [20/Aug/2020 18:08:12] root  Loaded the config file, my name is None
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-writeengineserver. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-writeengineserver 0
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-exemgr. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-exemgr 0
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-controllernode. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-controllernode 0
      [20/Aug/2020 18:08:12] root  apply Running stop on mcs-workernode. With sudo False and is_primary False
      [20/Aug/2020 18:08:12] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-workernode 0
      [20/Aug/2020 18:08:14] root  apply Running stop on mcs-storagemanager. With sudo False and is_primary False
      [20/Aug/2020 18:08:14] root  stop running /opt/cmapi/mcs_node_control/custom_dispatchers/container.sh stop mcs-storagemanager 0
      10.0.0.166 - - [20/Aug/2020:18:08:14] "PUT /cmapi/0.4.0/node/config HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      [20/Aug/2020 18:08:14] root  broadcast_new_config(): successfully pushed the new config file to ['cs-test-mdb-cs-node-0.cs-test-mdb-clust']
      10.0.0.166 - - [20/Aug/2020 18:08:14] cmapi_server DEBUG    put_commit starts
      10.0.0.166 - - [20/Aug/2020 18:08:14] cmapi_server DEBUG    put_commit returns {'timestamp': '2020-08-20 18:08:14.211940'}
      10.0.0.166 - - [20/Aug/2020:18:08:14] "PUT /cmapi/0.4.0/node/commit HTTP/1.1" 200 43 "" "python-requests/2.23.0"
      10.0.0.166 - - [20/Aug/2020 18:08:14] cmapi_server DEBUG    put_add_node returns {'timestamp': '2020-08-20 18:08:12.795427', 'node_id': 'cs-test-mdb-cs-node-0.cs-test-mdb-clust'}
      10.0.0.166 - - [20/Aug/2020:18:08:14] "PUT /cmapi/0.4.0/cluster/add-node HTTP/1.1" 200 97 "" "curl/7.61.1"
      10.0.0.166 - - [20/Aug/2020 18:10:41] cmapi_server DEBUG    get_status starts
      10.0.0.166 - - [20/Aug/2020 18:10:41] cmapi_server DEBUG    get_status starts
      [20/Aug/2020 18:10:41] root  Cannot establish or use DBRM connection. [Errno 111] Connection refused
      10.0.0.166 - - [20/Aug/2020 18:10:41] cmapi_server DEBUG    get_status returns {'timestamp': '2020-08-20 18:10:41.515981', 'uptime': 1783, 'dbrm_mode': 'slave', 'cluster_mode': 'readonly', 'dbroots': [], 'module_id': 1, 'services': []}
      10.0.0.166 - - [20/Aug/2020:18:10:41] "GET /cmapi/0.4.0/node/status HTTP/1.1" 200 156 "" "curl/7.61.1"
      10.0.0.166 - - [20/Aug/2020 18:10:41] cmapi_server DEBUG    get_status returns {'timestamp': '2020-08-20 18:10:41.507126', 'cs-test-mdb-cs-node-0.cs-test-mdb-clust': {'timestamp': '2020-08-20 18:10:41.515981', 'uptime': 1783, 'dbrm_mode': 'offline', 'cluster_mode': 'readonly', 'dbroots': [], 'module_id': 1, 'services': []}, 'num_nodes': 1}
      10.0.0.166 - - [20/Aug/2020:18:10:41] "GET /cmapi/0.4.0/cluster/status HTTP/1.1" 200 262 "" "curl/7.61.1"
       
      ------
      columnstore-init.log
      Removing gs://mdb0002756-1595842761951-columnstore-db00005735-79658635/087b1561-a13e-4d19-ae01-70d375532e36_0_2097152_data1~000.dir~000.dir~008.dir~016.dir~000.dir~FILE000.cdf...
      Removing gs://mdb0002756-1595842761951-columnstore-db00005735-79658635/00a05b2e-e403-4dbe-b9e1-50066a396af8_0_2097152_data1~000.dir~000.dir~008.dir~025.dir~000.dir~FILE000.cdf...
      Removing gs://mdb0002756-1595842761951-columnstore-db00005735-79658635/007d515e-8e44-449d-98b2-c18dea916a9b_0_8_data1~systemFiles~dbrm~BRM_savesA_vss...
      Removing gs://mdb0002756-1595842761951-columnstore-db00005735-79658635/00ceca06-c6ea-4377-af91-e69668a1e47e_0_3436_data1~systemFiles~dbrm~BRM_savesA_em...
      Removing gs://mdb0002756-1595842761951-columnstore-db00005735-79658635/0252c561-d480-4b53-ac20-e22c20f0d781_0_1048576_data1~000.dir~000.dir~004.dir~007.dir~000.dir~FILE000.cdf...
      CommandException: 5 files/objects could not be removed.
      Starting Primary Node...
      StorageManager PID = 318
      loading BRM
      Error reading data1/systemFiles/dbrm/BRM_saves_current: No such file or directory
      workernode PID = 333
      controllernode PID = 336
      PrimProc PID = 339
      ExeMgr PID = 371
      WriteEngineServer PID = 374
      DMLProc PID = 389
      DDLProc PID = 392
      Starting MariaDB.200820 18:06:17 mysqld_safe Logging to '/var/log/mariadb/columnstore/mariadb-error.log'.
      200820 18:06:17 mysqld_safe Starting mariadbd daemon with databases from /var/lib/mysql
       SUCCESS!
      Cross engine join user created
      Replication user created
      MaxScale user created
      Stopping Primary Node...
      Shutting down MariaDB. SUCCESS!
      Saved to /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves
      Multi-node with local-storage detected.
      Sending SIGTERM to Columnstore processes
      /usr/bin/columnstore-init: line 262:   339 Terminated              $MCS_INSTALL_BIN/PrimProc &> $LOG_PREFIX/primproc.log
      /usr/bin/columnstore-init: line 262:   371 Terminated              $MCS_INSTALL_BIN/ExeMgr &> $LOG_PREFIX/exemgr.log
      /usr/bin/columnstore-init: line 262:   374 Terminated              $MCS_INSTALL_BIN/WriteEngineServer &> $LOG_PREFIX/writeengineserver.log
      /usr/bin/columnstore-init: line 262:   389 Terminated              $MCS_INSTALL_BIN/DMLProc &> $LOG_PREFIX/dmlproc.log
      /usr/bin/columnstore-init: line 262:   392 Terminated              $MCS_INSTALL_BIN/DDLProc &> $LOG_PREFIX/ddlproc.log
      Sending SIGKILL to ColumnStore processes
      Clearing shared memory...
       
      ------
      controllernode.log
      DBRM Controller: Waiting for threads to finish...
      Exiting...
       
      ------
      corefiles
      cat: /var/log/mariadb/columnstore/corefiles: Is a directory
       
      ------
      dbbuilder.log
      Creating System Catalog...
       
      Creating SYSTABLE
      ---------------------------------------
        Creating TableName column OID: 1001
        Creating TableName column dictionary
        Creating Schema column OID: 1002
        Creating Schema column dictionary
        Creating ObjectId column OID: 1003
        Creating CreateDate column OID: 1004
        Creating LastUpdate column OID: 1005
        Creating INIT column OID: 1006
        Creating NEXT column OID: 1007
        Creating NUMOFROWS column OID: 1008
        Creating AVGROWLEN column OID: 1009
        Creating NUMOFBLOCKS column OID: 1010
        Creating AUTOINCREMENT column OID: 1011
       
      Creating SYSCOLUMN
      ---------------------------------------
        Creating Schema column OID: 1021
        Creating Schema column dictionary...
        Creating TableName column OID: 1022
        Creating TableName column dictionary...
        Creating ColumnName column OID: 1023
        Creating ColumnName column dictionary...
        Creating ObjectID column OID: 1024
        Creating DictOID column OID: 1025
        Creating ListOID column OID: 1026
        Creating TreeOID column OID: 1027
        Creating DataType column OID: 1028
        Creating ColumnLength column OID: 1029
        Creating ColumnPos column OID: 1030
        Creating LastUpdate column OID: 1031
        Creating DefaultValue column OID: 1032
        Creating DefaultValue column dictionary...
        Creating Nullable column OID: 1033
        Creating Scale column OID: 1034
        Creating Precision column OID: 1035
        Creating AutoInc column OID: 1036
        Creating DISTCOUNT column OID: 1037
        Creating NULLCOUNT column OID: 1038
        Creating MINVALUE column OID: 1039
        Creating MINVALUE column dictionary...
        Creating MAXVALUE column OID: 1040
        Creating MAXVALUE column dictionary...
        Creating CompressionType column OID: 1041
        Creating NEXTVALUE column OID: 1042
      System Catalog creation took: 0.452925 seconds to complete.
       
      System Catalog created
       
      Saved to /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves
       
      ------
      ddlproc.log
      DDLProc is ready...
       
      ------
      debug.log
      Aug 20 18:05:57 cs-test-mdb-cs-node-0 StorageManager[318]: Using the config file found at /etc/columnstore/storagemanager.cnf
      Aug 20 18:06:12 cs-test-mdb-cs-node-0 StorageManager[327]: Using the config file found at /etc/columnstore/storagemanager.cnf
      Aug 20 18:06:12 cs-test-mdb-cs-node-0 IDBFile[333]: 12.941968 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:12 cs-test-mdb-cs-node-0 IDBFile[348]: 12.974113 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:12 cs-test-mdb-cs-node-0 StorageManager[318]: Ownership: taking ownership of data1
      Aug 20 18:06:13 cs-test-mdb-cs-node-0 controllernode[348]: 13.034426 |0|0|0| D 29 CAL0000: TableLockServer::load(): could not open the save file/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks
      Aug 20 18:06:13 cs-test-mdb-cs-node-0 IDBFile[339]: 13.036026 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:14 cs-test-mdb-cs-node-0 IDBFile[374]: 14.004685 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 IDBFile[389]: 16.047648 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 IDBFile[392]: 16.057264 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 IDBFile[393]: 16.067991 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 IDBFile[427]: 16.758494 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:19 cs-test-mdb-cs-node-0 IDBFile[831]: 19.771999 |0|0|0| D 35 CAL0002: IDBFactory::installPlugin: installed filesystem plugin libcloudio.so
      Aug 20 18:06:19 cs-test-mdb-cs-node-0 StorageManager[318]: SessionManager Caught Signal 15
      Aug 20 18:06:19 cs-test-mdb-cs-node-0 StorageManager[318]: Shutdown StorageManager...
      Aug 20 18:06:20 cs-test-mdb-cs-node-0 StorageManager[318]: Ownership: releasing ownership of data1
       
      ------
      dmlproc.log
      DMLProc is ready...
       
      ------
      exemgr.log
      Starting ExeMgr: st = 50, qs = 20, mx = 95, cf = /etc/columnstore/Columnstore.xml
       
      ------
      info.log
      Aug 20 18:05:57 cs-test-mdb-cs-node-0 StorageManager[318]: Cache/cache_size = 8589934592
      Aug 20 18:05:58 cs-test-mdb-cs-node-0 StorageManager[318]: S3Storage: S3 connectivity & permissions are OK
      Aug 20 18:05:58 cs-test-mdb-cs-node-0 StorageManager[318]: max_concurrent_downloads = 21
      Aug 20 18:05:58 cs-test-mdb-cs-node-0 StorageManager[318]: max_concurrent_uploads = 21
      Aug 20 18:05:58 cs-test-mdb-cs-node-0 StorageManager[318]: SessionManager waiting for sockets.
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 DMLProc[389]: 16.065888 |0|0|0| I 20 CAL0002: DMLProc starts rollbackAll.
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 DMLProc[389]: 16.154070 |0|0|0| I 20 CAL0002: DMLProc will rollback 0 tables.
      Aug 20 18:06:16 cs-test-mdb-cs-node-0 DMLProc[389]: 16.166109 |0|0|0| I 20 CAL0002: DMLProc finished rollbackAll.
      Aug 20 18:06:20 cs-test-mdb-cs-node-0 StorageManager[318]: StorageManager Shutdown Complete.
       
      ------
      mariadb-error.log
      2020-08-20 18:06:17 0 [Warning] Plugin 'Columnstore' is of maturity level gamma while the server is stable
      2020-08-20 18:06:17 0 [Warning] Plugin 'COLUMNSTORE_COLUMNS' is of maturity level gamma while the server is stable
      2020-08-20 18:06:17 0 [Warning] Plugin 'COLUMNSTORE_TABLES' is of maturity level gamma while the server is stable
      2020-08-20 18:06:17 0 [Warning] Plugin 'COLUMNSTORE_FILES' is of maturity level gamma while the server is stable
      2020-08-20 18:06:17 0 [Warning] Plugin 'COLUMNSTORE_EXTENTS' is of maturity level gamma while the server is stable
      200820 18:06:17 Columnstore: Started; Version: 1.5.4-1
      2020-08-20 18:06:17 0 [Note] mariadbd: Aria engine: starting recovery
      recovered pages: 0% 10% 20% 30% 43% 53% 88% 99% 100% (0.0 seconds); tables to flush: 2 1 0
       (0.0 seconds);
      2020-08-20 18:06:17 0 [Note] mariadbd: Aria engine: recovery done
      2020-08-20 18:06:17 0 [Note] InnoDB: Using Linux native AIO
      2020-08-20 18:06:17 0 [Note] InnoDB: Uses event mutexes
      2020-08-20 18:06:17 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2020-08-20 18:06:17 0 [Note] InnoDB: Number of pools: 1
      2020-08-20 18:06:17 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
      2020-08-20 18:06:17 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
      2020-08-20 18:06:17 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
      2020-08-20 18:06:17 0 [Note] InnoDB: Completed initialization of buffer pool
      2020-08-20 18:06:17 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2020-08-20 18:06:17 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=56438
      2020-08-20 18:06:17 0 [Note] InnoDB: 128 rollback segments are active.
      2020-08-20 18:06:17 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2020-08-20 18:06:17 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2020-08-20 18:06:17 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2020-08-20 18:06:17 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2020-08-20 18:06:17 0 [Note] InnoDB: 10.5.6 started; log sequence number 56450; transaction id 49
      2020-08-20 18:06:17 0 [Note] Plugin 'FEEDBACK' is disabled.
      2020-08-20 18:06:17 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      2020-08-20 18:06:17 0 [Note] InnoDB: Buffer pool(s) load completed at 200820 18:06:17
      2020-08-20 18:06:17 0 [Note] Server socket created on IP: '::'.
      2020-08-20 18:06:17 0 [Note] Reading of all Master_info entries succeeded
      2020-08-20 18:06:17 0 [Note] Added new Master_info '' to hash table
      2020-08-20 18:06:17 0 [Note] /usr/sbin/mariadbd: ready for connections.
      Version: '10.5.6-MariaDB-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
      2020-08-20 18:06:18 0 [Note] /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
      2020-08-20 18:06:18 0 [Note] Event Scheduler: Purging the queue. 0 events
      2020-08-20 18:06:18 0 [Note] InnoDB: FTS optimize thread exiting.
      2020-08-20 18:06:18 0 [Note] InnoDB: Starting shutdown...
      2020-08-20 18:06:18 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
      2020-08-20 18:06:18 0 [Note] InnoDB: Buffer pool(s) dump completed at 200820 18:06:18
      2020-08-20 18:06:19 0 [Note] InnoDB: Shutdown completed; log sequence number 56474; transaction id 52
      2020-08-20 18:06:19 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
      2020-08-20 18:06:19 0 [Note] /usr/sbin/mariadbd: Shutdown complete
       
      2020-08-20 18:06:24 0 [Warning] Plugin 'Columnstore' is of maturity level gamma while the server is stable
      2020-08-20 18:06:24 0 [Warning] Plugin 'COLUMNSTORE_COLUMNS' is of maturity level gamma while the server is stable
      2020-08-20 18:06:24 0 [Warning] Plugin 'COLUMNSTORE_TABLES' is of maturity level gamma while the server is stable
      2020-08-20 18:06:24 0 [Warning] Plugin 'COLUMNSTORE_FILES' is of maturity level gamma while the server is stable
      2020-08-20 18:06:24 0 [Warning] Plugin 'COLUMNSTORE_EXTENTS' is of maturity level gamma while the server is stable
      200820 18:06:24 Columnstore: Started; Version: 1.5.4-1
      2020-08-20 18:06:24 0 [Note] InnoDB: Using Linux native AIO
      2020-08-20 18:06:24 0 [Note] InnoDB: Uses event mutexes
      2020-08-20 18:06:24 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
      2020-08-20 18:06:24 0 [Note] InnoDB: Number of pools: 1
      2020-08-20 18:06:24 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
      2020-08-20 18:06:24 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
      2020-08-20 18:06:24 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
      2020-08-20 18:06:24 0 [Note] InnoDB: Completed initialization of buffer pool
      2020-08-20 18:06:24 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
      2020-08-20 18:06:24 0 [Note] InnoDB: 128 rollback segments are active.
      2020-08-20 18:06:24 0 [Note] InnoDB: Creating shared tablespace for temporary tables
      2020-08-20 18:06:24 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
      2020-08-20 18:06:24 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
      2020-08-20 18:06:24 0 [Note] InnoDB: 10.5.6 started; log sequence number 56474; transaction id 49
      2020-08-20 18:06:24 0 [Note] Plugin 'FEEDBACK' is disabled.
      2020-08-20 18:06:24 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
      2020-08-20 18:06:24 0 [Note] InnoDB: Buffer pool(s) load completed at 200820 18:06:24
      2020-08-20 18:06:24 0 [Note] Server socket created on IP: '::'.
      2020-08-20 18:06:24 0 [Note] Reading of all Master_info entries succeeded
      2020-08-20 18:06:24 0 [Note] Added new Master_info '' to hash table
      2020-08-20 18:06:24 0 [Note] /usr/sbin/mariadbd: ready for connections.
      Version: '10.5.6-MariaDB-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
      2020-08-20 18:06:25 4 [Note] Start binlog_dump to slave_server(126), pos(, 4), using_gtid(1), gtid('')
      2020-08-20 18:06:28 6 [Note] Start binlog_dump to slave_server(126), pos(mariadb-bin.000002, 344), using_gtid(1), gtid('0-125-11')
       
      ------
      primproc.log
      Starting PrimitiveServer: st = 1, sq = 10, pw = 128, pq = 10240, nb = 478531, nt = 4, nc = 1, ra = 512, db = 128, mb = 512, rd = 0, tr = 0, ss = 67108864, bp = 4
       
      ------
      storagemanager.log
       
      ------
      trace
      cat: /var/log/mariadb/columnstore/trace: Is a directory
       
      ------
      workernode.log
      "/tmp/columnstore_tmp_files/rdwrscratch"
       
      ------
      writeengineserver.log
      WriteEngineServer is ready
      

      [root@cs-test-mdb-cs-node-0 /]# cat /etc/columnstore/Columnstore.xml
      <?xml version="1.0" ?>
      <Columnstore Version="V1.0.0">
          <ExeMgr1>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8601</Port>
              <Module>unassigned</Module>
          </ExeMgr1>
          <JobProc>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8602</Port>
          </JobProc>
          <ProcMgr>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8603</Port>
          </ProcMgr>
          <ProcMgr_Alarm>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8606</Port>
          </ProcMgr_Alarm>
          <ProcStatusControl>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8604</Port>
          </ProcStatusControl>
          <ProcStatusControlStandby>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8605</Port>
          </ProcStatusControlStandby>
          <localhost_ProcessMonitor>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8800</Port>
          </localhost_ProcessMonitor>
          <dm1_ProcessMonitor>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8800</Port>
          </dm1_ProcessMonitor>
          <um1_ProcessMonitor>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8800</Port>
          </um1_ProcessMonitor>
          <pm1_ProcessMonitor>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8800</Port>
          </pm1_ProcessMonitor>
          <dm1_ServerMonitor>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8622</Port>
          </dm1_ServerMonitor>
          <um1_ServerMonitor>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8622</Port>
          </um1_ServerMonitor>
          <pm1_ServerMonitor>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8622</Port>
          </pm1_ServerMonitor>
          <pm1_WriteEngineServer>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8630</Port>
          </pm1_WriteEngineServer>
          <DDLProc>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8612</Port>
          </DDLProc>
          <DMLProc>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8614</Port>
          </DMLProc>
          <BatchInsert>
              <RowsPerBatch>10000</RowsPerBatch>
          </BatchInsert>
          <PrimitiveServers>
              <Count>1</Count>
              <ConnectionsPerPrimProc>2</ConnectionsPerPrimProc>
              <ProcessorThreshold>128</ProcessorThreshold>
              <ProcessorQueueSize>10K</ProcessorQueueSize>
              <DebugLevel>0</DebugLevel>
              <LBID_Shift>13</LBID_Shift>
              <ColScanBufferSizeBlocks>512</ColScanBufferSizeBlocks>
              <ColScanReadAheadBlocks>512</ColScanReadAheadBlocks>
              <PrefetchThreshold>1</PrefetchThreshold>
              <PTTrace>0</PTTrace>
              <RotatingDestination>n</RotatingDestination>
              <DirectIO>y</DirectIO>
              <HighPriorityPercentage/>
              <MediumPriorityPercentage/>
              <LowPriorityPercentage/>
          </PrimitiveServers>
          <PMS1>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS1>
          <PMS2>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS2>
          <PMS3>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS3>
          <PMS4>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS4>
          <PMS5>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS5>
          <PMS6>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS6>
          <PMS7>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS7>
          <PMS8>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS8>
          <PMS9>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS9>
          <PMS10>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS10>
          <PMS11>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS11>
          <PMS12>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS12>
          <PMS13>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS13>
          <PMS14>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS14>
          <PMS15>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS15>
          <PMS16>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS16>
          <PMS17>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS17>
          <PMS18>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS18>
          <PMS19>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS19>
          <PMS20>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS20>
          <PMS21>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS21>
          <PMS22>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS22>
          <PMS23>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS23>
          <PMS24>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS24>
          <PMS25>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS25>
          <PMS26>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS26>
          <PMS27>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS27>
          <PMS28>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS28>
          <PMS29>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS29>
          <PMS30>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS30>
          <PMS31>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS31>
          <PMS32>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8620</Port>
          </PMS32>
          <SystemConfig>
              <SystemName>columnstore-1</SystemName>
              <ParentOAMModuleName>pm1</ParentOAMModuleName>
              <StandbyOAMModuleName>unassigned</StandbyOAMModuleName>
              <PrimaryUMModuleName>pm1</PrimaryUMModuleName>
              <ModuleHeartbeatPeriod>1</ModuleHeartbeatPeriod>
              <ModuleHeartbeatCount>3</ModuleHeartbeatCount>
              <ModuleProcMonWaitCount>12</ModuleProcMonWaitCount>
                      // 2.5 minutes
              <DBRootCount>1</DBRootCount>
              <DBRoot1>/var/lib/columnstore/data1</DBRoot1>
              <DBRMRoot>/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves</DBRMRoot>
              <TableLockSaveFile>/var/lib/columnstore/data1/systemFiles/dbrm/tablelocks</TableLockSaveFile>
              <DBRMTimeOut>15</DBRMTimeOut>
              <DBRMSnapshotInterval>100000</DBRMSnapshotInterval>
              <ExternalCriticalThreshold>90</ExternalCriticalThreshold>
              <ExternalMajorThreshold>80</ExternalMajorThreshold>
              <ExternalMinorThreshold>70</ExternalMinorThreshold>
              <TransactionArchivePeriod>10</TransactionArchivePeriod>
              <NMSIPAddress>0.0.0.0</NMSIPAddress>
              <TempSaveSize>128M</TempSaveSize>
              <WaitPeriod>10</WaitPeriod>
              <ProcessRestartCount>10</ProcessRestartCount>
              <ProcessRestartPeriod>120</ProcessRestartPeriod>
              <SwapAction>restartSystem</SwapAction>
              <ActivePmFailoverDisabled>n</ActivePmFailoverDisabled>
              <MemoryCheckPercent>95</MemoryCheckPercent>
              <DataFileLog>OFF</DataFileLog>
              <hdfsRdwrScratch>/rdwrscratch</hdfsRdwrScratch>
              <TempFileDir>/columnstore_tmp_files</TempFileDir>
              <SystemTempFileDir>/tmp/columnstore_tmp_files</SystemTempFileDir>
              <DataFilePlugin>libcloudio.so</DataFilePlugin>
          </SystemConfig>
          <SystemModuleConfig>
              <ModuleType1>dm</ModuleType1>
              <ModuleDesc1>Director Module</ModuleDesc1>
              <RunType1>SIMPLEX</RunType1>
              <ModuleCount1>0</ModuleCount1>
              <ModuleIPAddr1-1-1>0.0.0.0</ModuleIPAddr1-1-1>
              <ModuleHostName1-1-1>unassigned</ModuleHostName1-1-1>
              <ModuleDisableState1-1>ENABLED</ModuleDisableState1-1>
              <ModuleCPUCriticalThreshold1>0</ModuleCPUCriticalThreshold1>
              <ModuleCPUMajorThreshold1>0</ModuleCPUMajorThreshold1>
              <ModuleCPUMinorThreshold1>0</ModuleCPUMinorThreshold1>
              <ModuleCPUMinorClearThreshold1>0</ModuleCPUMinorClearThreshold1>
              <ModuleDiskCriticalThreshold1>90</ModuleDiskCriticalThreshold1>
              <ModuleDiskMajorThreshold1>80</ModuleDiskMajorThreshold1>
              <ModuleDiskMinorThreshold1>70</ModuleDiskMinorThreshold1>
              <ModuleMemCriticalThreshold1>90</ModuleMemCriticalThreshold1>
              <ModuleMemMajorThreshold1>0</ModuleMemMajorThreshold1>
              <ModuleMemMinorThreshold1>0</ModuleMemMinorThreshold1>
              <ModuleSwapCriticalThreshold1>90</ModuleSwapCriticalThreshold1>
              <ModuleSwapMajorThreshold1>80</ModuleSwapMajorThreshold1>
              <ModuleSwapMinorThreshold1>70</ModuleSwapMinorThreshold1>
              <ModuleDiskMonitorFileSystem1-1>/</ModuleDiskMonitorFileSystem1-1>
              <ModuleDBRootCount1-1>unassigned</ModuleDBRootCount1-1>
              <ModuleDBRootID1-1-1>unassigned</ModuleDBRootID1-1-1>
              <ModuleType2>um</ModuleType2>
              <ModuleDesc2>User Module</ModuleDesc2>
              <RunType2>SIMPLEX</RunType2>
              <ModuleCount2>0</ModuleCount2>
              <ModuleIPAddr1-1-2>0.0.0.0</ModuleIPAddr1-1-2>
              <ModuleHostName1-1-2>unassigned</ModuleHostName1-1-2>
              <ModuleDisableState1-2>ENABLED</ModuleDisableState1-2>
              <ModuleCPUCriticalThreshold2>0</ModuleCPUCriticalThreshold2>
              <ModuleCPUMajorThreshold2>0</ModuleCPUMajorThreshold2>
              <ModuleCPUMinorThreshold2>0</ModuleCPUMinorThreshold2>
              <ModuleCPUMinorClearThreshold2>0</ModuleCPUMinorClearThreshold2>
              <ModuleDiskCriticalThreshold2>90</ModuleDiskCriticalThreshold2>
              <ModuleDiskMajorThreshold2>80</ModuleDiskMajorThreshold2>
              <ModuleDiskMinorThreshold2>70</ModuleDiskMinorThreshold2>
              <ModuleMemCriticalThreshold2>90</ModuleMemCriticalThreshold2>
              <ModuleMemMajorThreshold2>0</ModuleMemMajorThreshold2>
              <ModuleMemMinorThreshold2>0</ModuleMemMinorThreshold2>
              <ModuleSwapCriticalThreshold2>90</ModuleSwapCriticalThreshold2>
              <ModuleSwapMajorThreshold2>80</ModuleSwapMajorThreshold2>
              <ModuleSwapMinorThreshold2>70</ModuleSwapMinorThreshold2>
              <ModuleDiskMonitorFileSystem1-2>/</ModuleDiskMonitorFileSystem1-2>
              <ModuleDBRootCount1-2>unassigned</ModuleDBRootCount1-2>
              <ModuleDBRootID1-1-2>unassigned</ModuleDBRootID1-1-2>
              <ModuleType3>pm</ModuleType3>
              <ModuleDesc3>Performance Module</ModuleDesc3>
              <RunType3>SIMPLEX</RunType3>
              <ModuleCount3>1</ModuleCount3>
              <ModuleIPAddr1-1-3>10.0.0.166</ModuleIPAddr1-1-3>
              <ModuleHostName1-1-3>cs-test-mdb-cs-node-0.cs-test-mdb-clust</ModuleHostName1-1-3>
              <ModuleDisableState1-3>ENABLED</ModuleDisableState1-3>
              <ModuleCPUCriticalThreshold3>0</ModuleCPUCriticalThreshold3>
              <ModuleCPUMajorThreshold3>0</ModuleCPUMajorThreshold3>
              <ModuleCPUMinorThreshold3>0</ModuleCPUMinorThreshold3>
              <ModuleCPUMinorClearThreshold3>0</ModuleCPUMinorClearThreshold3>
              <ModuleDiskCriticalThreshold3>90</ModuleDiskCriticalThreshold3>
              <ModuleDiskMajorThreshold3>80</ModuleDiskMajorThreshold3>
              <ModuleDiskMinorThreshold3>70</ModuleDiskMinorThreshold3>
              <ModuleMemCriticalThreshold3>90</ModuleMemCriticalThreshold3>
              <ModuleMemMajorThreshold3>0</ModuleMemMajorThreshold3>
              <ModuleMemMinorThreshold3>0</ModuleMemMinorThreshold3>
              <ModuleSwapCriticalThreshold3>90</ModuleSwapCriticalThreshold3>
              <ModuleSwapMajorThreshold3>80</ModuleSwapMajorThreshold3>
              <ModuleSwapMinorThreshold3>70</ModuleSwapMinorThreshold3>
              <ModuleDiskMonitorFileSystem1-3>/</ModuleDiskMonitorFileSystem1-3>
              <ModuleDBRootCount1-3>1</ModuleDBRootCount1-3>
              <ModuleDBRootID1-1-3>1</ModuleDBRootID1-1-3>
          </SystemModuleConfig>
          <SystemExtDeviceConfig>
              <Count>0</Count>
              <Name1>unassigned</Name1>
              <IPAddr1>0.0.0.0</IPAddr1>
              <DisableState1>ENABLED</DisableState1>
          </SystemExtDeviceConfig>
          <SessionManager>
              <MaxConcurrentTransactions>1000</MaxConcurrentTransactions>
              <TxnIDFile>/var/lib/columnstore/data1/systemFiles/dbrm/SMTxnID</TxnIDFile>
          </SessionManager>
          <VersionBuffer>
              <VersionBufferFileSize>1GB</VersionBufferFileSize>
          </VersionBuffer>
          <OIDManager>
              <OIDBitmapFile>/var/lib/columnstore/data1/systemFiles/dbrm/oidbitmap</OIDBitmapFile>
              <FirstOID>3000</FirstOID>
          </OIDManager>
          <WriteEngine>
              <BulkRoot>/var/lib/columnstore/data/bulk</BulkRoot>
              <BulkRollbackDir>/var/lib/columnstore/data1/systemFiles/bulkRollback</BulkRollbackDir>
              <MaxFileSystemDiskUsagePct>98</MaxFileSystemDiskUsagePct>
              <CompressedPaddingBlocks>1</CompressedPaddingBlocks>
          </WriteEngine>
          <DBRM_Controller>
              <NumWorkers>1</NumWorkers>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8616</Port>
          </DBRM_Controller>
          <DBRM_Worker1>
              <IPAddr>cs-test-mdb-cs-node-0.cs-test-mdb-clust</IPAddr>
              <Port>8700</Port>
              <Module>pm1</Module>
          </DBRM_Worker1>
          <DBRM_Worker2>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker2>
          <DBRM_Worker3>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker3>
          <DBRM_Worker4>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker4>
          <DBRM_Worker5>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker5>
          <DBRM_Worker6>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker6>
          <DBRM_Worker7>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker7>
          <DBRM_Worker8>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker8>
          <DBRM_Worker9>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker9>
          <DBRM_Worker10>
              <IPAddr>0.0.0.0</IPAddr>
              <Port>8700</Port>
              <Module>unassigned</Module>
          </DBRM_Worker10>
          <DBBC>
              <NumCaches>1</NumCaches>
              <IOMTracing>0</IOMTracing>
              <BRPTracing>0</BRPTracing>
              <ReportFrequency>65536</ReportFrequency>
              <MaxOpenFiles>2K</MaxOpenFiles>
              <DecreaseOpenFilesCount>200</DecreaseOpenFilesCount>
              <FDCacheTrace>0</FDCacheTrace>
              <NumBlocksPct>50</NumBlocksPct>
          </DBBC>
          <Installation>
              <SystemStartupOffline>n</SystemStartupOffline>
              <InitialInstallFlag>y</InitialInstallFlag>
              <SingleServerInstall>y</SingleServerInstall>
              <ServerTypeInstall>2</ServerTypeInstall>
              <PMwithUM>n</PMwithUM>
              <MySQLRep>n</MySQLRep>
              <DBRootStorageType>StorageManager</DBRootStorageType>
              <UMStorageType>internal</UMStorageType>
              <EEPackageType>binary</EEPackageType>
              <SystemLogConfigFile>/etc/rsyslog.d/columnstore.conf</SystemLogConfigFile>
              <Cloud>unassigned</Cloud>
              <AmazonVPCNextPrivateIP>autoassign</AmazonVPCNextPrivateIP>
              <UMInstanceType>unassigned</UMInstanceType>
              <UMSecurityGroup>unassigned</UMSecurityGroup>
              <UMVolumeSize>unassigned</UMVolumeSize>
              <UMVolumeType>gp2</UMVolumeType>
              <UMVolumeIOPS>unassigned</UMVolumeIOPS>
              <PMInstanceType>unassigned</PMInstanceType>
              <PMVolumeSize>unassigned</PMVolumeSize>
              <PMVolumeType>gp2</PMVolumeType>
              <PMVolumeIOPS>unassigned</PMVolumeIOPS>
              <AmazonPMFailover>y</AmazonPMFailover>
              <AmazonAutoTagging>y</AmazonAutoTagging>
              <AmazonElasticIPCount>0</AmazonElasticIPCount>
              <AmazonElasticIPAddr1>unassigned</AmazonElasticIPAddr1>
              <AmazonElasticModule1>unassigned</AmazonElasticModule1>
              <DataRedundancyConfig>n</DataRedundancyConfig>
              <DataRedundancyCopies>0</DataRedundancyCopies>
              <DataRedundancyStorageType>unassigned</DataRedundancyStorageType>
              <CoreFileFlag>n</CoreFileFlag>
              <MySQLPort>3306</MySQLPort>
              <AmazonDeviceName>/dev/xvd</AmazonDeviceName>
              <LockFileDirectory>/var/lock/subsys</LockFileDirectory>
              <ProfileFile>/etc/profile.d/columnstoreAlias.sh</ProfileFile>
              <DataRedundancyNetworkType/>
          </Installation>
          <ExtentMap>
              <FilesPerColumnPartition>4</FilesPerColumnPartition>
              <ExtentsPerSegmentFile>2</ExtentsPerSegmentFile>
              <BRM_UID>0x0</BRM_UID>
          </ExtentMap>
          <HashJoin>
              <MaxBuckets>128</MaxBuckets>
              <MaxElems>128K</MaxElems>
              <FifoSizeLargeSide>64</FifoSizeLargeSide>
              <PmMaxMemorySmallSide>1G</PmMaxMemorySmallSide>
              <TotalUmMemory>25%</TotalUmMemory>
              <TotalPmUmMemory>100%</TotalPmUmMemory>
              <CPUniqueLimit>100</CPUniqueLimit>
              <AllowDiskBasedJoin>N</AllowDiskBasedJoin>
              <TempFileCompression>Y</TempFileCompression>
          </HashJoin>
          <JobList>
              <FlushInterval>16K</FlushInterval>
              <FifoSize>16</FifoSize>
              <RequestSize>1</RequestSize>
              <ThreadPoolSize>100</ThreadPoolSize>
          </JobList>
          <TupleWSDL>
              <MaxSize>1M</MaxSize>
              <InitialCapacity>1M</InitialCapacity>
              <MaxBuckets>512</MaxBuckets>
          </TupleWSDL>
          <RowAggregation>
              </RowAggregation>
          <CrossEngineSupport>
              <Host>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Host>
              <Port>3306</Port>
              <User>cej</User>
              <Password>C0lumnStore!</Password>
              <TLSCA/>
              <TLSClientCert/>
              <TLSClientKey/>
          </CrossEngineSupport>
          <QueryStats>
              <Enabled>N</Enabled>
          </QueryStats>
          <UserPriority>
              <Enabled>N</Enabled>
          </UserPriority>
          <NetworkCompression>
              <Enabled>Y</Enabled>
          </NetworkCompression>
          <QueryTele>
              <Host>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Host>
              <Port>0</Port>
          </QueryTele>
          <StorageManager>
              <MaxSockets>30</MaxSockets>
              <Enabled>Y</Enabled>
          </StorageManager>
          <DataRedundancyConfig>
              <DBRoot1PMs/>
          </DataRedundancyConfig>
          <ConfigRevision>1</ConfigRevision>
          <ClusterManager>cs-test-mdb-cs-node-0.cs-test-mdb-clust</ClusterManager>
          <ClusterName>MyCluster</ClusterName>
          <NextNodeId>2</NextNodeId>
          <NextDBRootId>2</NextDBRootId>
          <DesiredNodes>
              <Node>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Node>
          </DesiredNodes>
          <ActiveNodes>
              <Node>cs-test-mdb-cs-node-0.cs-test-mdb-clust</Node>
          </ActiveNodes>
          <InactiveNodes/>
          <PrimaryNode>cs-test-mdb-cs-node-0.cs-test-mdb-clust</PrimaryNode>
          <SharedStorage>yes</SharedStorage>
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            jens.rowekamp Jens Röwekamp (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.