rebuildEM utility must support compressed segment files (MCOL-4566)

[MCOL-4624] Implement proper calculation for HWM from segment file. Created: 2021-03-20  Updated: 2023-10-26  Resolved: 2021-03-23

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.1

Type: Sub-Task Priority: Minor
Reporter: Denis Khalikov Assignee: Denis Khalikov
Resolution: Done Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-4566 rebuildEM utility must support compre... Closed
Sprint: 2021-4, 2021-5, 2021-6, 2021-8, 2021-9, 2021-10, 2021-11, 2021-12, 2021-13

 Description   

To proper restore individual extent we have to calculate HWM from the segment file.
At the current version, segment file has a field `blockCount` which specifies an amount of preallocated blocks inside segment file, but HWM specifies the the last filled block - 1. So based on block count, we can make a binary search for range (0, blockCount) and find the first block with the empty value. Since those file are compressed we have to use header ptrs part.
Note: The HWM is important part, adding this will solve the problem `cpiinfo` as well, the bulk is using `batch` insertion, so it aligns rows to block size (even if we insert 1 row it will increment HWM by one).



 Comments   
Comment by Denis Khalikov [ 2021-03-23 ]

This is implemented, algorithm is simple, starting from the last block in segment file, and searching for the first non empty row in block.
We need to start from the last block, because bulk inserts to file starting from the new block, so it could be a empty block.

Generated at Thu Feb 08 02:51:45 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.