[MCOL-637] Getting errors when using cpimport mode 3 Created: 2017-03-24  Updated: 2020-08-25  Resolved: 2018-09-13

Status: Closed
Project: MariaDB ColumnStore
Component/s: cpimport
Affects Version/s: 1.0.6.1
Fix Version/s: Icebox

Type: Bug Priority: Minor
Reporter: Anders Karlsson Assignee: Andrew Hutchings (Inactive)
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

CentOS 6.4


 Description   

Loading a simple table gets an error, but only in mode 3:

CREATE TABLE test.a(c1 INTEGER) ENgine=ColumnStore;

Then a simple data file, a.dat:

100
 
200

Then run cpimport:
$ cpimport test a -m3 a.dat
And the result is:
Locale is : C

Using table OID 3399 as the default JOB ID
Input file(s) will be read from : /home2/anders/src/m17/csload
Job description file : /usr/local/Calpont/data/bulk/tmpjob/3399_D20170324_T171324_S118317_Job_3399.xml
Log file for this job: /usr/local/Calpont/data/bulk/log/Job_3399.log
2017-03-24 17:13:24 (23117) INFO : successfully loaded job file /usr/local/Calpont/data/bulk/tmpjob/3399_D20170324_T171324_S118317_Job_3399.xml
2017-03-24 17:13:24 (23117) INFO : Job file loaded, run time for this step : 0.378337 seconds
2017-03-24 17:13:24 (23117) INFO : PreProcessing check starts
2017-03-24 17:13:24 (23117) INFO : input data file /home2/anders/src/m17/csload/a.dat
2017-03-24 17:13:24 (23117) ERR : Backup error opening column file for OID-3400; DBRoot-1; partition-0; segment-0 [1052]
2017-03-24 17:13:24 (23117) CRIT : Error in pre-processing the job file for table test.a [1052]

Error in loading job data

 Comments   
Comment by David Thompson (Inactive) [ 2017-03-24 ]

can you run:
select * from information_schema.columnstore_files where object_id=3400 ( id from 2nd to last line) and check if this exists or not?

Not able to reproduce this locally so there must have been some series of steps before.

Comment by Anders Karlsson [ 2017-03-24 ] 

+-----------+------------+--------------+--------------------------------------------------------------------------------------------+-----------+----------------------+
 
| OBJECT_ID | SEGMENT_ID | PARTITION_ID | FILENAME                                                                                   | FILE_SIZE | COMPRESSED_DATA_SIZE |
 
+-----------+------------+--------------+--------------------------------------------------------------------------------------------+-----------+----------------------+
 
|      3400 |          0 |            0 | /home2/local/mariadb/columnstore/data1/000.dir/000.dir/013.dir/072.dir/000.dir/FILE000.cdf |   1056768 |                73728 |
 
+-----------+------------+--------------+--------------------------------------------------------------------------------------------+-----------+----------------------+

Comment by Anders Karlsson [ 2017-03-24 ]

$ ls -l /home2/local/mariadb/columnstore/data1/000.dir/000.dir/013.dir/072.dir/000.dir/FILE000.cdf
-rwxrwxrwx. 1 root root 1056768 Mar 24 17:13 /home2/local/mariadb/columnstore/data1/000.dir/000.dir/013.dir/072.dir/000.dir/FILE000.cdf

Comment by David Thompson (Inactive) [ 2017-03-24 ]

does the same cpimport command still fail now?

Comment by Andrew Hutchings (Inactive) [ 2018-09-13 ]

Is this a root or non-root install? Mode 3 writes to the data files directly so if this a root install the unprivileged user you are using cannot have access to the files.

Comment by Anders Karlsson [ 2018-09-13 ]

I tested it with 1.1.5 now, and it seems to work as expected. I I run cpimport as root (with sudo) it works out of the box. If I run it as a non-root users I get reasonable errors, i.e. oermission problems accessing this or that file / directory. Changing a few permission flags and it works as expected. So the error messages I get are reasonable and fixable, and after fixing them it works, so you can close this issue I guess.

Comment by Andrew Hutchings (Inactive) [ 2018-09-13 ]

I'll close it for now. We have longer term plans regarding permissions and ownership that would make these kind of issues go away.

Generated at Thu Feb 08 02:22:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.