Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-3417 Profile / optimize SM
  3. MCOL-3418

optimize around cpimport's inefficiencies during uncompressed import

    XMLWordPrintable

Details

    • Sub-Task
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • ?
    • None

    Description

      Importing part table from a 500gb dbt3 dataset. When the target table is compressed, observed behavior is expected and good (19k calls to write()). When uncompressed, the opposite (750k calls to write()).

      Small writes make the overhead parts in SM dominate. In particular, what rises to the top of the profile are functions related to json parsing.

      The first thing I will do is implement a parsed-json cache, then reevaluate.

      Attachments

        Activity

          People

            pleblanc Patrick LeBlanc (Inactive)
            pleblanc Patrick LeBlanc (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.