[MCOL-720] can't import a file to exist table by cpimport Created: 2017-05-18  Updated: 2017-05-18  Resolved: 2017-05-18

Status: Closed
Project: MariaDB ColumnStore
Component/s: cpimport
Affects Version/s: 1.0.9
Fix Version/s: Icebox

Type: Bug Priority: Minor
Reporter: y-taka Assignee: Unassigned
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

Single Server (CentOS 7.3)

Attachments: Zip Archive columnstore_logs.zip    

 Description   

I setup MariaDB ColumnStore environment as "Single Server".

I get dump file from below site, and try to import it by cpimport.
However "'wiki_test.page_counts' does not exist in Columnstore." error happened
even I created database and table.

https://dumps.wikimedia.org/other/pagecounts-raw/2016/2016-05/

pm1# gunzip -r pagecounts-2016050*
pm1# ls -lh pagecounts-2016050*
-rwxr-xr-x. 1 root root 353M May 17 15:58 pagecounts-20160501-000000
...
pm1# head pagecounts-20160501-000000
aa Category:Language_user_templates 1 6020
aa File:Hitchhicking_on_the_road_132_-_Gasp%C3%A9sie_Canada.jpg 1 8520
...

pm1# mcsmysql
MariaDB [(none)]> create database wiki_test;
MariaDB [(none)]> use wiki_test;
MariaDB [wiki_test]> create table page_counts (domain_code varchar(30), page_title text, count_views int unsigned, total_response_size int unsigned);
MariaDB [wiki_test]> exit

pm1# cpimport wiki_test page_counts ./pagecounts-20160501-000000 -s "\s"
Locale is : C
Column delimiter : \

Unable to set default JobID; Error getting OID for table wiki_test.page_counts: IDB-2006: 'wiki_test.page_counts' does not exist in Columnstore.

 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-05-18 ]

Hi,

cpimport is for ColumnStore tables only, by default if you do not specify and engine MariaDB will create an InnoDB table. You need to do the following when creating the table:

create table page_counts (domain_code varchar(30), page_title varchar(4000), count_views int unsigned, total_response_size int unsigned) engine=columnstore;

Please note that the TEXT data type isn't supported in ColumnStore version 1.0. This is coming in 1.1.

I should also note that "\s" will not work as a column delimiter in cpimport. I believe those files just use a space delimited so you would need to do:

cpimport wiki_test page_counts ./pagecounts-20160501-000000 -s ' '

Comment by y-taka [ 2017-05-18 ]

Thanks,

As you said, I have to specify "engine=columnstore".
In addition, I found text data type is not supported in ColumnStore.

> I should also note that "\s" will not work as a column delimiter in cpimport.
> I believe those files just use a space delimited so you would need to do:

In fact, "\s" don't work well, but " " is also bad as below.
So I converted " " into "\t" by tr command, and succeed in importing at last.

cpimport wiki_test page_counts ./pagecounts-20160501-000000 -s " "
Locale is : C
Column delimiter : -

As a result, my problem is not a bug, I'm sorry.

Comment by Andrew Hutchings (Inactive) [ 2017-05-18 ]

No problem, I glad you got it to work.

Generated at Thu Feb 08 02:23:19 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.