Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-987 LZ4 compression for on-disk columnar data
  3. MCOL-4654

LZ4 compression support for `CompressedInetStreamSocket` and `JoinPartition`

    XMLWordPrintable

    Details

    • Type: Sub-Task
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.1.1
    • Fix Version/s: 6.1.1
    • Component/s: writeengine
    • Labels:
      None
    • Sprint:
      2021-5, 2021-6, 2021-7, 2021-8, 2021-9

      Description

      LZ4 compression support for `CompressedInetStreamSocket` and `JoinPartition`.
      1. `getUncompressedSize`: currently `CompressedInetStreamSocket` and `JoinPartition` require a function `getUncompressedSize`, so we can allocate appropriate size for uncompressed buffer. LZ4 does not have by default this function, reading the `snappy` documentation I found out that the cost of this function is O(1) so it probably stores the input size in the compressed block, so I think we can do the same way for those two specific parts.

      2. Currently `CompressedInetStreamSocket` and `JoinPartition` use config to enable/disable compression, for sockets and for temporal files. Probably we should add a one more field in config to specify the compression type.

        Attachments

          Activity

            People

            Assignee:
            denis0x0D Denis Khalikov
            Reporter:
            denis0x0D Denis Khalikov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.