LZ4 compression for on-disk columnar data (MCOL-987)

[MCOL-4654] LZ4 compression support for `CompressedInetStreamSocket` and `JoinPartition` Created: 2021-03-31  Updated: 2021-04-05  Resolved: 2021-04-05

Status: Closed
Project: MariaDB ColumnStore
Component/s: writeengine
Affects Version/s: 6.1.1
Fix Version/s: 6.1.1

Type: Sub-Task Priority: Major
Reporter: Denis Khalikov Assignee: Denis Khalikov
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

 Description   

LZ4 compression support for `CompressedInetStreamSocket` and `JoinPartition`.
1. `getUncompressedSize`: currently `CompressedInetStreamSocket` and `JoinPartition` require a function `getUncompressedSize`, so we can allocate appropriate size for uncompressed buffer. LZ4 does not have by default this function, reading the `snappy` documentation I found out that the cost of this function is O(1) so it probably stores the input size in the compressed block, so I think we can do the same way for those two specific parts.

2. Currently `CompressedInetStreamSocket` and `JoinPartition` use config to enable/disable compression, for sockets and for temporal files. Probably we should add a one more field in config to specify the compression type.



 Comments   
Comment by Denis Khalikov [ 2021-04-05 ]

Closed because of `subtask` requirements.

Generated at Thu Feb 08 02:51:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.