Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-5952

ByteStream improvements

    XMLWordPrintable

Details

    • New Feature
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      At the moment, ByteStream uses a growing buffer for serialized data. This can lead to increased memory consumption under certain conditions. For example, when dumping RGData to disk, we create a ByteStream object that consumes at least as much memory as the RGData itself. Moreover, if compression is enabled, we allocate another buffer for the compressed data.

      It would be nice to have one of (or both):

      • support for streaming: caller provides read/write callbacks to provide more data to deserialize or process already serialized part
      • zero-copy processing for large portions of data, such as RGData::rowData.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              alexey.antipovsky Aleksei Antipovskii
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.