[MCOL-5269] LOAD DATA INFILE poor performance Created: 2022-10-18 Updated: 2023-11-17 Resolved: 2023-11-10 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | cpimport |
| Affects Version/s: | 6.4.4-dompe |
| Fix Version/s: | 23.10 |
| Type: | Bug | Priority: | Critical |
| Reporter: | Bryan Bancroft (Inactive) | Assignee: | Gagan Goel (Inactive) |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Environment: |
56cores, 528gb, 5 node cluster |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
mariadb documentation https://mariadb.com/docs/reference/col6/cli/cpimport/ ,says "If the LOAD DATA [ LOCAL ] INFILE or INSERT INTO .. SELECT FROM .. statements are used with a ColumnStore table, MariaDB Enterprise ColumnStore 6 transforms the operation into a bulk load that uses cpimport by default" The performance between LOAD DATA INFILE and cpimport by loading from the same file, there's a huge difference. Below is the log. root@multi-unmanaged-db-p09:~# cpimport -s ',' -E '"' test tab1 /snfs1/temp/.db_backups/multi/test.csv MariaDB [test]> LOAD DATA INFILE |
| Comments |
| Comment by Bryan Bancroft (Inactive) [ 2022-10-26 ] |
|
https://jira.mariadb.org/browse/MCOL-4853 works around this issue |
| Comment by Chris Calender (Inactive) [ 2023-01-18 ] |
|
regarding above comment, see the last comment in |