Details
-
New Feature
-
Status: Stalled (View Workflow)
-
Critical
-
Resolution: Unresolved
-
1.5.3
-
None
-
2021-3, 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9, 2021-10, 2021-11, 2021-12, 2021-16, 2021-17, 2022-22, 2022-23, 2023-4, 2023-5, 2023-6, 2023-7, 2023-8, 2023-10, 2023-11, 2023-12
Description
When using storagemanager/S3 with Columnstore we want to avoid requiring additional shared storage: NFS or GlusterFS.
There are three purposes for the shared storage:
- store S3 meta, that is a list of object files parts of the original MCS file
- store journal, that is a text file contains <offset, byte array> pairs that alters contents of S3 object files copies stored locally.
- dbroot ownernership mechanism
There is a write-up on some S3 implementation details in MCS.
The suggested solution is to leverage an existing distributed KeyValueStorage to fulfill the purposes mentioned earlier. Distributed KVS chosen is FoundationDB.
Attachments
Issue Links
1.
|
Deliver FoundationDB with CMAPI | Open | Unassigned | |
2.
|
Start/Stop for FoundationDB on MCS startup | Open | Unassigned | |
3.
|
Initial configuration and future configuration sync with MCS topology | Open | Unassigned | |
4.
|
FoundationDB monitoring and inspection | Open | Unassigned | |
5.
|
Move ownership mechanism to FoundationDB | Open | Unassigned | |
6.
|
Move S3 metadata into FoundationDB | Open | Unassigned | |
7.
|
Aggregate S3 metadata to reduce number of keys in FoundationDB | Open | Unassigned | |
8.
|
Develop methods to move S3 metadata from files into FoundationDB for existing installations | Open | Unassigned | |
9.
|
Develop seamless upgrade strategy for existing installations | Open | Unassigned | |
10.
|
Move journal from files into FoundationDB | Open | Unassigned | |
11.
|
Develop checks to enfoce journal flush before MCS cluster upgrade | Open | Unassigned | |
12.
|
Develop tools to access S3 metadata from MariaDB client conneciton | Open | Unassigned | |
13.
|
Generate metadata from S3 data. | Open | Denis Khalikov | |
14.
|
Test the feature/fix issues | Open | Unassigned |