Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-5106

mcsRebuild EM - support multi-node + S3

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.4.1
    • Component/s: advanced tools
    • Labels:
      None
    • Sprint:
      2021-17

      Description

      `mcsRebuildEM` supports 2 types of storage `shared local storage` and `S3`.
      `shared local storage`.

      Shared local storage use case.
      1. setup a cluster.
      2. login into mcs1 node.
      docker exec -it mcs1 bash
      3. create a database, table and insert a row.

      [root@mcs1 /]# mysql
      Welcome to the MariaDB monitor.  Commands end with ; or \g.
      Your MariaDB connection id is 17
      Server version: 10.6.8-4-MariaDB-log MariaDB Server
       
      Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
       
      Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
       
      MariaDB [(none)]> create database temp;
      Query OK, 1 row affected (0.000 sec)
       
      MariaDB [(none)]> use temp;
      Database changed
      MariaDB [temp]> create table t1 (a int, b varchar(200)) engine=columnstore;
      Query OK, 0 rows affected (0.599 sec)
       
      MariaDB [temp]> insert into t1 values (1, "abcdf");
      Query OK, 1 row affected (0.151 sec)
       
      MariaDB [temp]> select * from t1;
      +------+-------+
      | a    | b     |
      +------+-------+
      |    1 | abcdf |
      +------+-------+
      1 row in set (0.033 sec)
       
      MariaDB [temp]> exit
      Bye
      

      4. Shutdown cluster.

      [root@mcs1 /]# mcsShutdown 
      {
        "timestamp": "2022-06-13 09:17:17.040974",
        "mcs1": {
          "timestamp": "2022-06-13 09:17:26.433336"
        },
        "mcs2": {
          "timestamp": "2022-06-13 09:17:29.753189"
        },
        "mcs3": {
          "timestamp": "2022-06-13 09:17:33.071669"
        }
      }
      

      5. Remove BRM_saves_em file. (Backup file of extentmap)

      [root@mcs1 /]# rm /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_em 
      rm: remove regular file '/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_em'? y
      

      6. Restore extent map from data files.

      [root@mcs1 /]# mcsRebuildEM -v
      The launch of mcsRebuildEM tool must be sanctioned by MariaDB support. 
      Do you want to continue Y/N? y
      Initialize system extents from the initial state
      Collect extents for the DBRoot /var/lib/columnstore/data1
      Collect extents for the DBRoot /var/lib/columnstore/data2
      Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/188.dir/000.dir/FILE000.cdf
      fileName2Oid:  [OID: 3004, partition: 0, segment: 0] 
      Searching for HWM... 
      Block count: 256
      HWM is: 0
      FileId is collected [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]
      Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/187.dir/000.dir/FILE000.cdf
      fileName2Oid:  [OID: 3003, partition: 0, segment: 0] 
      Searching for HWM... 
      Block count: 128
      HWM is: 0
      FileId is collected [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]
      Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/189.dir/000.dir/FILE000.cdf
      fileName2Oid:  [OID: 3005, partition: 0, segment: 0] 
      Searching for HWM... 
      Block count: 256
      HWM is: 0
      FileId is collected [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]
      Collect extents for the DBRoot /var/lib/columnstore/data3
      Build extent map with size 3
      Extent is created, allocated size 4096 actual LBID 234496
      For [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]
      Setting a HWM for [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]
      Extent is created, allocated size 8192 actual LBID 238592
      For [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]
      Setting a HWM for [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]
      Extent is created, allocated size 8192 actual LBID 246784
      For [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]
      Setting a HWM for [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]
      Completed.
      

      7. Start cluster.

      [root@mcs1 /]# mcsStart 
      {
        "timestamp": "2022-06-13 09:22:42.826742"
      }
      

      8. Check CS does work.

      [root@mcs1 /]# mysql 
      Welcome to the MariaDB monitor.  Commands end with ; or \g.
      Your MariaDB connection id is 19
      Server version: 10.6.8-4-MariaDB-log MariaDB Server
       
      Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
       
      Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
       
      MariaDB [(none)]> use temp;
      Reading table information for completion of table and column names
      You can turn off this feature to get a quicker startup with -A
       
      Database changed
      MariaDB [temp]> select * from t1;
      +------+-------+
      | a    | b     |
      +------+-------+
      |    1 | abcdf |
      +------+-------+
      1 row in set (0.082 sec)
       
      MariaDB [temp]> exit
      Bye
      

      S3 use case.
      1, 2, 3, 4 steps are the same as for `shared local storage`.
      5. Start `StorageManager` (access to S3)

      [root@mcs1 ~]# StorageManager 
      StorageManager[2634]: Using the config file found at /etc/columnstore/storagemanager.cnf
      StorageManager[2634]: Cache/cache_size = 4294967296
      StorageManager[2634]: S3Storage: S3 connectivity & permissions are OK
      StorageManager[2634]: max_concurrent_downloads = 20
      StorageManager[2634]: max_concurrent_downloads = 21
      StorageManager[2634]: max_concurrent_uploads = 20
      StorageManager[2634]: max_concurrent_uploads = 21
      StorageManager[2634]: StorageManager started.
      StorageManager[2634]: SessionManager waiting for sockets.
      StorageManager[2633]: StorageManager main process has started
      

      5. Remove `BRM_saves_em`, you can do it by hand on S3 bucket or by running tool, it will ask you to delete this file if this file exists.
      6. Run `mcsRebuildEm`

      [root@mcs1 ~]# mcsRebuildEM -v
      The launch of mcsRebuildEM tool must be sanctioned by MariaDB support. 
      Do you want to continue Y/N? y
      /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_em file exists. 
      Do you want to delete this file Y/N? y
      StorageManager[2634]: Ownership: taking ownership of data1
      Initialize system extents from the initial state
      Collect extents for the DBRoot /var/lib/columnstore/data1
      Collect extents for the DBRoot /var/lib/columnstore/data2
      StorageManager[2634]: Ownership: taking ownership of data2
      Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/188.dir/000.dir/FILE000.cdf
      fileName2Oid:  [OID: 3004, partition: 0, segment: 0] 
      Searching for HWM... 
      Block count: 256
      HWM is: 0
      FileId is collected [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]
      Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/187.dir/000.dir/FILE000.cdf
      fileName2Oid:  [OID: 3003, partition: 0, segment: 0] 
      Searching for HWM... 
      Block count: 128
      HWM is: 0
      FileId is collected [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]
      Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/189.dir/000.dir/FILE000.cdf
      fileName2Oid:  [OID: 3005, partition: 0, segment: 0] 
      Searching for HWM... 
      Block count: 256
      HWM is: 0
      FileId is collected [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]
      Collect extents for the DBRoot /var/lib/columnstore/data3
      Build extent map with size 3
      Extent is created, allocated size 4096 actual LBID 234496
      For [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]
      Setting a HWM for [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]
      Extent is created, allocated size 8192 actual LBID 238592
      For [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]
      Setting a HWM for [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]
      Extent is created, allocated size 8192 actual LBID 246784
      For [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]
      Setting a HWM for [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]
      Completed.
      

      7. Shutdown `StorageManager`.

      [root@mcs1 ~]# ps -aux 
      USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
      root           1  0.0  0.0   4376   840 ?        Ss   09:35   0:00 /usr/bin/tini -- docker-entrypoint.sh start-se
      root           7  0.0  0.0  14852  3276 ?        S    09:35   0:00 /bin/bash /usr/bin/docker-entrypoint.sh start-
      root          12  0.0  0.0 197260  2368 ?        Ssl  09:35   0:00 rsyslogd
      root          14  0.0  0.0  14852  3248 ?        S    09:35   0:00 bash /usr/bin/start-services
      root         930  0.4  0.1 1503624 59888 ?       Sl   09:36   0:02 /usr/share/columnstore/cmapi/python/bin/python
      root        1014  0.0  0.0  15116  3472 ?        S    09:36   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/li
      mysql       1178  0.1  0.3 1692432 108224 ?      Sl   09:36   0:00 /usr/sbin/mariadbd --basedir=/usr --datadir=/v
      root        1226  0.0  0.0 111688  5496 ?        Sl   09:36   0:00 monit -I
      root        2410  0.0  0.0  15116  3764 pts/0    Ss   09:38   0:00 bash
      root        2634  0.1  0.0 2863456 28436 pts/0   Sl   09:42   0:00 StorageManager
      root        2720  0.0  0.0  50544  3788 pts/0    R+   09:45   0:00 ps -aux
      [root@mcs1 ~]# kill -s 15 2634
      [root@mcs1 ~]# StorageManager[2634]: SessionManager Caught Signal 15
      StorageManager[2634]: Shutdown StorageManager...
      StorageManager[2634]: Ownership: releasing ownership of data1
      StorageManager[2634]: Ownership: releasing ownership of data2
      StorageManager[2634]: StorageManager Shutdown Complete.
      

      8. Start cluster.

      [root@mcs1 ~]# mcsStart 
      {
        "timestamp": "2022-06-13 09:47:28.612874"
      }
      

      9. Check that CS works.

      [root@mcs1 ~]# mysql
      Welcome to the MariaDB monitor.  Commands end with ; or \g.
      Your MariaDB connection id is 19
      Server version: 10.6.8-4-MariaDB-log MariaDB Server
       
      Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
       
      Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
       
      MariaDB [(none)]> use temp;
      Reading table information for completion of table and column names
      You can turn off this feature to get a quicker startup with -A
       
      Database changed
      MariaDB [temp]> select * from t1;
      +------+-------+
      | a    | b     |
      +------+-------+
      |    1 | fsddf |
      +------+-------+
      1 row in set (0.071 sec)
      
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dleeyh Daniel Lee
              Reporter:
              denis0x0D Denis Khalikov
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.