Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4912

MCS bulk insertion is slow

    XMLWordPrintable

    Details

    • Sprint:
      2021-14, 2021-15, 2021-16, 2021-17

      Description

      The client faced bulk insertion operation(and presumably INSERT operations) slowdown. The system has no obvious hardware bottlenecks and despite the fact that the storage is remote (FCoE) there is no evidence FCoE contributes a lot to the overall timings of bulk ingestion operations.
      The table is about 80 columns(mostly dicts) and bulk ingestion of 82 records takes 18 seconds where 12 are spent in preprocessing phase. Preprocessing phase involves BRM communication and so-called HWM chunk backup(a backup of the most recent compressed chunk of a segment/dict files).
      cpimport strace file showed that BRM communcations socket operations and mutex-es involved contributes a lot to the overall timings.
      The immediate workaround is to horizontally scale BRM, namely EM whilst the permanent solution is to introduce lookup structure to speed up EM operations.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              drrtuy Roman
              Reporter:
              drrtuy Roman
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.