[MCOL-1526] cpimport of a file into a wide column table very slow Created: 2018-07-04  Updated: 2022-11-05  Resolved: 2022-11-05

Status: Closed
Project: MariaDB ColumnStore
Component/s: cpimport
Affects Version/s: 1.1.5
Fix Version/s: Icebox

Type: New Feature Priority: Major
Reporter: Ravi Prakash (Inactive) Assignee: Todd Stoffel (Inactive)
Resolution: Won't Do Votes: 0
Labels: None
Environment:

Single Node UM, PM configuration


Attachments: File bwtech.tgz    

 Description   

When a table with more than a few hundred columns is loaded using cpimport the load times are large. Email -

This user has to load hundreds of CSV files into their CS tables and are claiming that using LOAD DATA INFILE or cpimport is an order of magnitude slower than loading the same data onto other storage engines, for example MyISAM. I'm wondering if you can help explain if this is an expected behavior or there is something that can be done to improve the load times into CS.
Their tests are being run on a single server installation in AWS EC2 (UM and PM running on the same machine).

The attached files include a definition of 2 tables (one CS, and the other one MyISAM), the output of SHOW VARIABLES and 2 CSV files to test. They are comparing the time reported by the server at the end of LOAD DATA INFILE.



 Comments   
Comment by Todd Stoffel (Inactive) [ 2022-11-05 ]

This item is being closed because it was well passed the expiration date with no activity. If you suspect this was done in error please create a new ticket.

Generated at Thu Feb 08 02:29:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.