Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-1526

cpimport of a file into a wide column table very slow

    XMLWordPrintable

Details

    • New Feature
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Won't Do
    • 1.1.5
    • Icebox
    • cpimport
    • None
    • Single Node UM, PM configuration

    Description

      When a table with more than a few hundred columns is loaded using cpimport the load times are large. Email -

      This user has to load hundreds of CSV files into their CS tables and are claiming that using LOAD DATA INFILE or cpimport is an order of magnitude slower than loading the same data onto other storage engines, for example MyISAM. I'm wondering if you can help explain if this is an expected behavior or there is something that can be done to improve the load times into CS.
      Their tests are being run on a single server installation in AWS EC2 (UM and PM running on the same machine).

      The attached files include a definition of 2 tables (one CS, and the other one MyISAM), the output of SHOW VARIABLES and 2 CSV files to test. They are comparing the time reported by the server at the end of LOAD DATA INFILE.

      Attachments

        Activity

          People

            toddstoffel Todd Stoffel (Inactive)
            rprakash Ravi Prakash (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.