[MCOL-5106] mcsRebuild EM - support multi-node + S3 - Jira

XML

Word

Printable

Details

Type: New Feature
Status: Closed (View Workflow)
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 6.4.1
Component/s: None
Labels:
None

Sprint:
2021-17

Description

`mcsRebuildEM` supports 2 types of storage `shared local storage` and `S3`.
`shared local storage`.

Shared local storage use case.
1. setup a cluster.
2. login into mcs1 node.
docker exec -it mcs1 bash
3. create a database, table and insert a row.

[root@mcs1 /]# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \g.

Your MariaDB connection id is 17

Server version: 10.6.8-4-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database temp;

Query OK, 1 row affected (0.000 sec)

MariaDB [(none)]> use temp;

Database changed

MariaDB [temp]> create table t1 (a int, b varchar(200)) engine=columnstore;

Query OK, 0 rows affected (0.599 sec)

MariaDB [temp]> insert into t1 values (1, "abcdf");

Query OK, 1 row affected (0.151 sec)

MariaDB [temp]> select * from t1;

+------+-------+

| a    | b     |

+------+-------+

|    1 | abcdf |

+------+-------+

1 row in set (0.033 sec)

MariaDB [temp]> exit

Bye

4. Shutdown cluster.

[root@mcs1 /]# mcsShutdown

  "timestamp": "2022-06-13 09:17:17.040974",

  "mcs1": {

    "timestamp": "2022-06-13 09:17:26.433336"

},

  "mcs2": {

    "timestamp": "2022-06-13 09:17:29.753189"

},

  "mcs3": {

    "timestamp": "2022-06-13 09:17:33.071669"

5. Remove BRM_saves_em file. (Backup file of extentmap)

[root@mcs1 /]# rm /var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_em

rm: remove regular file '/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_em'? y

6. Restore extent map from data files.

[root@mcs1 /]# mcsRebuildEM -v

The launch of mcsRebuildEM tool must be sanctioned by MariaDB support.

Do you want to continue Y/N? y

Initialize system extents from the initial state

Collect extents for the DBRoot /var/lib/columnstore/data1

Collect extents for the DBRoot /var/lib/columnstore/data2

Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/188.dir/000.dir/FILE000.cdf

fileName2Oid:  [OID: 3004, partition: 0, segment: 0]

Searching for HWM...

Block count: 256

HWM is: 0

FileId is collected [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]

Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/187.dir/000.dir/FILE000.cdf

fileName2Oid:  [OID: 3003, partition: 0, segment: 0]

Searching for HWM...

Block count: 128

HWM is: 0

FileId is collected [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]

Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/189.dir/000.dir/FILE000.cdf

fileName2Oid:  [OID: 3005, partition: 0, segment: 0]

Searching for HWM...

Block count: 256

HWM is: 0

FileId is collected [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]

Collect extents for the DBRoot /var/lib/columnstore/data3

Build extent map with size 3

Extent is created, allocated size 4096 actual LBID 234496

For [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]

Setting a HWM for [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]

Extent is created, allocated size 8192 actual LBID 238592

For [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]

Setting a HWM for [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]

Extent is created, allocated size 8192 actual LBID 246784

For [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]

Setting a HWM for [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]

Completed.

7. Start cluster.

[root@mcs1 /]# mcsStart

  "timestamp": "2022-06-13 09:22:42.826742"

8. Check CS does work.

[root@mcs1 /]# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \g.

Your MariaDB connection id is 19

Server version: 10.6.8-4-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use temp;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

MariaDB [temp]> select * from t1;

+------+-------+

| a    | b     |

+------+-------+

|    1 | abcdf |

+------+-------+

1 row in set (0.082 sec)

MariaDB [temp]> exit

Bye

S3 use case.
1, 2, 3, 4 steps are the same as for `shared local storage`.
5. Start `StorageManager` (access to S3)

[root@mcs1 ~]# StorageManager

StorageManager[2634]: Using the config file found at /etc/columnstore/storagemanager.cnf

StorageManager[2634]: Cache/cache_size = 4294967296

StorageManager[2634]: S3Storage: S3 connectivity & permissions are OK

StorageManager[2634]: max_concurrent_downloads = 20

StorageManager[2634]: max_concurrent_downloads = 21

StorageManager[2634]: max_concurrent_uploads = 20

StorageManager[2634]: max_concurrent_uploads = 21

StorageManager[2634]: StorageManager started.

StorageManager[2634]: SessionManager waiting for sockets.

StorageManager[2633]: StorageManager main process has started

5. Remove `BRM_saves_em`, you can do it by hand on S3 bucket or by running tool, it will ask you to delete this file if this file exists.
6. Run `mcsRebuildEm`

[root@mcs1 ~]# mcsRebuildEM -v

The launch of mcsRebuildEM tool must be sanctioned by MariaDB support.

Do you want to continue Y/N? y

/var/lib/columnstore/data1/systemFiles/dbrm/BRM_saves_em file exists.

Do you want to delete this file Y/N? y

StorageManager[2634]: Ownership: taking ownership of data1

Initialize system extents from the initial state

Collect extents for the DBRoot /var/lib/columnstore/data1

Collect extents for the DBRoot /var/lib/columnstore/data2

StorageManager[2634]: Ownership: taking ownership of data2

Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/188.dir/000.dir/FILE000.cdf

fileName2Oid:  [OID: 3004, partition: 0, segment: 0]

Searching for HWM...

Block count: 256

HWM is: 0

FileId is collected [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]

Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/187.dir/000.dir/FILE000.cdf

fileName2Oid:  [OID: 3003, partition: 0, segment: 0]

Searching for HWM...

Block count: 128

HWM is: 0

FileId is collected [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]

Processing file: /var/lib/columnstore/data2/000.dir/000.dir/011.dir/189.dir/000.dir/FILE000.cdf

fileName2Oid:  [OID: 3005, partition: 0, segment: 0]

Searching for HWM...

Block count: 256

HWM is: 0

FileId is collected [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]

Collect extents for the DBRoot /var/lib/columnstore/data3

Build extent map with size 3

Extent is created, allocated size 4096 actual LBID 234496

For [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]

Setting a HWM for [OID: 3003, partition: 0, segment: 0, col width: 4, lbid:234496, hwm: 0, isDict: 0]

Extent is created, allocated size 8192 actual LBID 238592

For [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]

Setting a HWM for [OID: 3004, partition: 0, segment: 0, col width: 8, lbid:238592, hwm: 0, isDict: 0]

Extent is created, allocated size 8192 actual LBID 246784

For [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]

Setting a HWM for [OID: 3005, partition: 0, segment: 0, col width: 8, lbid:246784, hwm: 0, isDict: 1]

Completed.

7. Shutdown `StorageManager`.

[root@mcs1 ~]# ps -aux

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND

root           1  0.0  0.0   4376   840 ?        Ss   09:35   0:00 /usr/bin/tini -- docker-entrypoint.sh start-se

root           7  0.0  0.0  14852  3276 ?        S    09:35   0:00 /bin/bash /usr/bin/docker-entrypoint.sh start-

root          12  0.0  0.0 197260  2368 ?        Ssl  09:35   0:00 rsyslogd

root          14  0.0  0.0  14852  3248 ?        S    09:35   0:00 bash /usr/bin/start-services

root         930  0.4  0.1 1503624 59888 ?       Sl   09:36   0:02 /usr/share/columnstore/cmapi/python/bin/python

root        1014  0.0  0.0  15116  3472 ?        S    09:36   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/li

mysql       1178  0.1  0.3 1692432 108224 ?      Sl   09:36   0:00 /usr/sbin/mariadbd --basedir=/usr --datadir=/v

root        1226  0.0  0.0 111688  5496 ?        Sl   09:36   0:00 monit -I

root        2410  0.0  0.0  15116  3764 pts/0    Ss   09:38   0:00 bash

root        2634  0.1  0.0 2863456 28436 pts/0   Sl   09:42   0:00 StorageManager

root        2720  0.0  0.0  50544  3788 pts/0    R+   09:45   0:00 ps -aux

[root@mcs1 ~]# kill -s 15 2634

[root@mcs1 ~]# StorageManager[2634]: SessionManager Caught Signal 15

StorageManager[2634]: Shutdown StorageManager...

StorageManager[2634]: Ownership: releasing ownership of data1

StorageManager[2634]: Ownership: releasing ownership of data2

StorageManager[2634]: StorageManager Shutdown Complete.

8. Start cluster.

[root@mcs1 ~]# mcsStart

  "timestamp": "2022-06-13 09:47:28.612874"

9. Check that CS works.

[root@mcs1 ~]# mysql

Welcome to the MariaDB monitor.  Commands end with ; or \g.

Your MariaDB connection id is 19

Server version: 10.6.8-4-MariaDB-log MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use temp;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

MariaDB [temp]> select * from t1;

+------+-------+

| a    | b     |

+------+-------+

|    1 | fsddf |

+------+-------+

1 row in set (0.071 sec)

Attachments

Issue Links

relates to

MCOL-5131 mcsRebuildEM - add support to calculate HWM for system cat. files.

Closed

Activity

People

Assignee:: Daniel Lee (Inactive)

Reporter:: Denis Khalikov

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 2022-05-27 17:11

Updated:: 2024-01-10 14:31

Resolved:: 2022-06-28 14:29

Git Integration

Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.