Future improvement possibilities for storage-manager (MCOL-3449)

[MCOL-3989] implement size reporting Created: 2020-05-08  Updated: 2023-10-25  Resolved: 2023-10-25

Status: Closed
Project: MariaDB ColumnStore
Component/s: Storage Manager
Affects Version/s: None
Fix Version/s: Icebox

Type: Sub-Task Priority: Minor
Reporter: Patrick LeBlanc (Inactive) Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None


 Description   

This has fallen through the cracks. I do not believe we ever got around to implementing size reporting for SM. IIRC, the way it is done by other IDBDataFiles when a table is compressed, is to load all of the data, decompress all of the data, and sum up the size of the fully decompressed data. That's awful for tables on a local disk, and would be even more awful to implement it that way for S3-backed tables.

Ideally, for this ticket we would redesign that for SM and non-SM tables. Maybe we can return a size estimate based on extentmap data instead. If it needs to be a precise value (not sure how important that is), maybe we add an extra field to EM data for this. Alternatively, we can store the uncompressed size of a compressed file in the file header to reduce the amount of data to load and avoid extra steps. Many easy options for this one.


Generated at Thu Feb 08 02:46:57 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.