Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-388

Latency after cpimport job has been completed

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • 1.0.4
    • Icebox
    • cpimport
    • None

    Description

      Build used: 1.0.4-1 beta

      The issue was first identified by Stephane Varoqui working onsite with Believe, migrating to ColumnStore from InfiniDB. On a 1UM-4PM system, after cpimport returned the job has been completed, it took 16 seconds before returning back to the OS. I also confirmed similar latency on a 1UM-2PM stack, executing cpimport from the UM.

      time /usr/local/mariadb/columnstore/bin/cpimport tpch1c nation nation.tbl
      2016-11-04 10:39:45 (17377) INFO : Running distributed import (mode 1) on all PMs...
      2016-11-04 10:39:46 (17377) INFO : For table tpch1c.nation: 25 rows processed and 25 rows inserted.
      2016-11-04 10:39:46 (17377) INFO : Bulk load completed, total run time : 1.12012 seconds

      real 0m5.196s
      user 0m0.152s
      sys 0m0.040s

      I verified by examining the log files in each PM that the PM did complete the job and reported a successful status back to the UM quickly. The time being printing the last line and actually return to the OS is 4 seconds on my tests. Stephane reported that as number of Pos increases, this latency multiplies.

      This issue affects customer who has large number of small files to import, as this latency is many times over the actually loading time.

      Attachments

        Issue Links

          Activity

            People

              dthompson David Thompson (Inactive)
              dleeyh Daniel Lee (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.