I am trying to load a dump created by "mysqldump -T ..." and CS is throwing error -1 during the load process. I am using the LOAD DATA INFILE command, no using cpimport directly.
MariaDB [(none)]> load data infile '/tmp/user.txt' into table quser fields terminated by "\t" enclosed by '"';
ERROR 1030 (HY000): Got error -1 "Internal error < 0 (Not system error)" from storage engine Columnstore
Looking at the logs I see:
{{2017-02-15 13:58:58 (3379) INFO : Number of rows with errors = 1. Row numbers with error reasons are listed in file /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.err
2017-02-15 13:58:58 (3379) INFO : Number of rows with errors = 1. Exact error rows are listed in file /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.bad
2017-02-15 13:58:58 (3379) ERR : Actual error row count(1) exceeds the max error rows(0) allowed for table quantum.quser [1451]
2017-02-15 13:58:58 (3379) CRIT : Bulkload Read (thread 0) Failed for Table quantum.quser. Terminating this job. [1451]
2017-02-15 13:58:58 (3379) INFO : Bulk load completed, total run time : 618.577 seconds
}}
However the files mentioned in the above error can not be found on the filesystem:
{{# ls /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.err
ls: cannot access '/usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.err': No such file or directory
root@qfarm-dev2:/usr/local/mariadb/columnstore# ls /usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.bad
ls: cannot access '/usr/local/mariadb/columnstore/bin/quser.tbl.Job_3626_3379.bad': No such file or directory}}
The files appear to be created in a different location and they have an _1 suffix:
root@qfarm-dev2:/usr/local/mariadb# find . -name quser*
./columnstore/mysql/db/quser.tbl.Job_3626_3379.err_1
./columnstore/mysql/db/quantum/quser.frm
./columnstore/mysql/db/quser.tbl.Job_3626_22179.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_3379.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_22179.err_1
Justin Swanhart (Inactive)
added a comment - The files appear to be created in a different location and they have an _1 suffix:
root@qfarm-dev2:/usr/local/mariadb# find . -name quser*
./columnstore/mysql/db/quser.tbl.Job_3626_3379.err_1
./columnstore/mysql/db/quantum/quser.frm
./columnstore/mysql/db/quser.tbl.Job_3626_22179.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_3379.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_22179.err_1
By default LDI is internally converted to a cpimport job and thus it uses the cpimport style file names and the files are created under the mysql/db directory.
This can be disabled but leads to slower performance due to the DML path being followed. https://mariadb.com/kb/en/mariadb/columnstore-batch-insert-mode/ documents how to disable this.
David Thompson (Inactive)
added a comment - - edited By default LDI is internally converted to a cpimport job and thus it uses the cpimport style file names and the files are created under the mysql/db directory.
This can be disabled but leads to slower performance due to the DML path being followed. https://mariadb.com/kb/en/mariadb/columnstore-batch-insert-mode/ documents how to disable this.
Also the LDI kb article ( https://mariadb.com/kb/en/mariadb/columnstore-load-data-infile/ ) did not cover this so have added a section at the end to document where the bad and err files are written to.
my point is the log files point to files that don't exist. You read the log file, it says go look HERE for output and that output doesn't exist.
This should be an open bug.
Justin Swanhart (Inactive)
added a comment - my point is the log files point to files that don't exist. You read the log file, it says go look HERE for output and that output doesn't exist.
This should be an open bug.
While fixing MCOL-4483 and logging path issues. went ahead and removed the logging that shows up from cpimport.bin that gives a path to a temp log file that is sent back to cpimport. cpimport reports the correct path to the file.bad_[pmID] that is useful to users.
Ben Thompson (Inactive)
added a comment - While fixing MCOL-4483 and logging path issues. went ahead and removed the logging that shows up from cpimport.bin that gives a path to a temp log file that is sent back to cpimport. cpimport reports the correct path to the file.bad_ [pmID] that is useful to users.
Bug
Major
The files appear to be created in a different location and they have an _1 suffix:
root@qfarm-dev2:/usr/local/mariadb# find . -name quser*
./columnstore/mysql/db/quser.tbl.Job_3626_3379.err_1
./columnstore/mysql/db/quantum/quser.frm
./columnstore/mysql/db/quser.tbl.Job_3626_22179.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_3379.bad_1
./columnstore/mysql/db/quser.tbl.Job_3626_22179.err_1