[MCOL-571] CS provides incorrect path in logs to bad and err files when using load data infile Created: 2017-02-15  Updated: 2021-02-19  Resolved: 2021-02-19

Status: Closed
Project: MariaDB ColumnStore
Component/s: cpimport
Affects Version/s: None
Fix Version/s: 5.5.2

Type: Bug Priority: Major
Reporter: Justin Swanhart (Inactive) Assignee: Roman
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-4483 Consolidate Log Location Closed
Epic Link: ColumnStore Compatibility Improvements
Sprint: 2021-1, 2021-2, 2021-3

 Description   

I am trying to load a dump created by "mysqldump -T ..." and CS is throwing error -1 during the load process. I am using the LOAD DATA INFILE command, no using cpimport directly.
MariaDB [(none)]> load data infile '/tmp/user.txt' into table quser fields terminated by "\t" enclosed by '"';
ERROR 1030 (HY000): Got error -1 "Internal error < 0 (Not system error)" from storage engine Columnstore

Looking at the logs I see:
{{2017-02-15 13:58:58 (3379) INFO : Number of rows with errors = 1. Row numbers with error reasons are listed in file /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.err
2017-02-15 13:58:58 (3379) INFO : Number of rows with errors = 1. Exact error rows are listed in file /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.bad
2017-02-15 13:58:58 (3379) ERR : Actual error row count(1) exceeds the max error rows(0) allowed for table quantum.quser [1451]
2017-02-15 13:58:58 (3379) CRIT : Bulkload Read (thread 0) Failed for Table quantum.quser. Terminating this job. [1451]
2017-02-15 13:58:58 (3379) INFO : Bulk load completed, total run time : 618.577 seconds
}}

However the files mentioned in the above error can not be found on the filesystem:
{{# ls /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.err
ls: cannot access '/usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.err': No such file or directory

root@qfarm-dev2:/usr/local/mariadb/columnstore# ls /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.bad
ls: cannot access '/usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.bad': No such file or directory}}



 Comments   
Comment by Justin Swanhart (Inactive) [ 2017-02-15 ]

The files appear to be created in a different location and they have an _1 suffix:
root@qfarm-dev2:/usr/local/mariadb# find . -name quser*
./columnstore/mysql/db/quser.tbl.Job_3626_3379.err_1
./columnstore/mysql/db/quantum/quser.frm
./columnstore/mysql/db/quser.tbl.Job_3626_22179.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_3379.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_22179.err_1

Comment by David Thompson (Inactive) [ 2017-02-16 ]

By default LDI is internally converted to a cpimport job and thus it uses the cpimport style file names and the files are created under the mysql/db directory.
This can be disabled but leads to slower performance due to the DML path being followed. https://mariadb.com/kb/en/mariadb/columnstore-batch-insert-mode/ documents how to disable this.

Also the LDI kb article (https://mariadb.com/kb/en/mariadb/columnstore-load-data-infile/) did not cover this so have added a section at the end to document where the bad and err files are written to.

Comment by Justin Swanhart (Inactive) [ 2017-02-16 ]

my point is the log files point to files that don't exist. You read the log file, it says go look HERE for output and that output doesn't exist.

This should be an open bug.

Comment by David Thompson (Inactive) [ 2017-02-17 ]

Good point, agreed.

Comment by Ben Thompson (Inactive) [ 2021-01-26 ]

While fixing MCOL-4483 and logging path issues. went ahead and removed the logging that shows up from cpimport.bin that gives a path to a temp log file that is sent back to cpimport. cpimport reports the correct path to the file.bad_[pmID] that is useful to users.

Comment by David Hall (Inactive) [ 2021-02-19 ]

Fixed as part of MCOL-4483

Generated at Thu Feb 08 02:22:05 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.