Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-987

LZ4 compression for on-disk columnar data

    XMLWordPrintable

Details

    • New Feature
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 6.1.1
    • 6.1.1
    • writeengine
    • None

    Description

      According to different comparisions, e.g. here LZ4 might have:

      • better compression rate
      • better decompression speed
      • almost the same compression speed
        compared with Snappy. MCS uses Snappy by default for both columnar files and CompressedInetStreamSocket(TCP socket) implementation.

      The chunk size is an important parameter used to define how much worth of data is compressed in one go to store in the compressed columnar file. As of now it is set to 4MB that might be less apropriate for LZ4 so one should compare different compressed chunk size values.
      In the end MCS must have another compression method that is controlled via the session variable columnstore_compression_type. There will be no separate knob to control compression used by CompressedInetStreamSocket.

      If LZ4 performs as well as expected(faster decompression, better compression, compression speed parity) it will become our default.

      Attachments

        Activity

          People

            dleeyh Daniel Lee (Inactive)
            LinuxJedi Andrew Hutchings (Inactive)
            Votes:
            4 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.