[MCOL-5402] save_brm must save oidbitmap as a part of extent map to improve snapshot backups Created: 2023-01-27  Updated: 2023-12-15

Status: Open
Project: MariaDB ColumnStore
Component/s: ?
Affects Version/s: 22.08.7
Fix Version/s: Icebox

Type: New Feature Priority: Major
Reporter: Roman Assignee: Max Mether
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-5403 Design a consistent snapshot backup p... Open

 Description   

There are number of inefficiencies that complicate S3 installation backups. One of them is the fact that save_brm utility called on cluster shutdown doesn't save oidbitmap file.
Here is a simple failure scenario that demostrates this:
1 Install MCS on S3
2 Remove all meta files in /var/lib/columnstore/storagemanager/metadata/data1/systemFiles/dbrm
3 Call mcs cluster stop or save_brm directly
In the end there will data saved on either cache or S3 + .meta files that describes EM files on S3. See the output taken after step 3.

 
[root@mcs1 dbrm]# pwd
/var/lib/columnstore/storagemanager/metadata/data1/systemFiles/dbrm
[root@mcs1 dbrm]# ls -la
total 32
drwxr-xr-x 2 root root 4096 Jan 27 03:14 .
drwxr-xr-x 3 root root 4096 Jan 27 02:04 ..
-rw-r--r-- 1 root root  247 Jan 27 03:14 BRM_saves_current.meta
-rw-r--r-- 1 root root  246 Jan 27 03:14 BRM_saves_em.meta
-rw-r--r-- 1 root root   63 Jan 27 03:14 BRM_saves_journal.meta
-rw-r--r-- 1 root root  244 Jan 27 03:14 BRM_saves_vbbm.meta
-rw-r--r-- 1 root root  241 Jan 27 03:14 BRM_saves_vss.meta
-rw-r--r-- 1 root root  237 Jan 27 03:14 SMTxnID.meta

MCS complains on its next startup about non-empty EM and lack of oidbitmap file. The suggested solution is to save oidbitmap to save oidbitmap.meta file.


Generated at Thu Feb 08 02:57:37 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.