Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-987 LZ4 compression for on-disk columnar data
  3. MCOL-4654

LZ4 compression support for `CompressedInetStreamSocket` and `JoinPartition`

    XMLWordPrintable

Details

    • Sub-Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 6.1.1
    • 6.1.1
    • writeengine
    • None
    • 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

    Description

      LZ4 compression support for `CompressedInetStreamSocket` and `JoinPartition`.
      1. `getUncompressedSize`: currently `CompressedInetStreamSocket` and `JoinPartition` require a function `getUncompressedSize`, so we can allocate appropriate size for uncompressed buffer. LZ4 does not have by default this function, reading the `snappy` documentation I found out that the cost of this function is O(1) so it probably stores the input size in the compressed block, so I think we can do the same way for those two specific parts.

      2. Currently `CompressedInetStreamSocket` and `JoinPartition` use config to enable/disable compression, for sockets and for temporal files. Probably we should add a one more field in config to specify the compression type.

      Attachments

        Activity

          People

            denis0x0D Denis Khalikov
            denis0x0D Denis Khalikov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.