[MCOL-281] LDI using cpimport pads char column values with spaces Created: 2016-09-01  Updated: 2016-09-13  Resolved: 2016-09-13

Status: Closed
Project: MariaDB ColumnStore
Component/s: DMLProc
Affects Version/s: 1.0.1, 1.0.2
Fix Version/s: 1.0.3

Type: Bug Priority: Major
Reporter: Daniel Lee (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-280 Beta issues Closed
Sprint: 1.0.3

 Description   

When LDI uses cpimport to load data in the background (infinidb_use_import_for_batchinsert=1, which is the default setting in my.cnf), the loaded data is incorrect.

After loading the dbt3.region table with 5 rows,

-------------------------------------------------------------------------------------------------------------------------------------------

r_regionkey r_name r_comment

-------------------------------------------------------------------------------------------------------------------------------------------

0 AFRICA lar deposits. blithely final packages cajole. regular waters are final requests. regular accounts are according to
1 AMERICA hs use ironic, even requests. s
2 ASIA ges. thinly even pinto beans ca
3 EUROPE ly final courts cajole furiously final excuse
4 MIDDLE EAST uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl

------------------------——————————————————————————————————————————————————————————

The following query returned no data:

select * from region where r_name = 'MIDDLE EAST';
Empty set (0.05 sec)

If should return:

------------------------------------------------------------------------------------------------------------------------------------

r_regionkey r_name r_comment

------------------------------------------------------------------------------------------------------------------------------------

4 MIDDLE EAST uickly special accounts cajole carefully blithely close requests. carefully final asymptotes haggle furiousl

------------------------------------------------------------------------------------------------------------------------------------
1 row in set (0.29 sec)

If I use the other methods to load data into the table, the query worked correctly.

1. Insert
2. LDI without using cpimport
3. cpimport

Further investigation revealed that LDI with cpimport loads data with space padded to char strings:

Good table

MariaDB [mytest]> select r_regionkey, r_name, hex(r_name) from region;
----------------------------------------------

r_regionkey r_name hex(r_name)

----------------------------------------------

0 AFRICA 414652494341
1 AMERICA 414D4552494341
2 ASIA 41534941
3 EUROPE 4555524F5045
4 MIDDLE EAST 4D4944444C452045415354

----------------------------------------------
5 rows in set (0.82 sec)

Bad table

MariaDB [mytest]> select r_regionkey, r_name, hex(r_name) from badregion;
--------------------------------------------------------------------------

r_regionkey r_name hex(r_name)

--------------------------------------------------------------------------

0 AFRICA 41465249434120202020202020202020202020202020202020
1 AMERICA 414D4552494341202020202020202020202020202020202020
2 ASIA 41534941202020202020202020202020202020202020202020
3 EUROPE 4555524F504520202020202020202020202020202020202020
4 MIDDLE EAST 4D4944444C4520454153542020202020202020202020202020

--------------------------------------------------------------------------
5 rows in set (1.07 sec)

This issue has caused much pain in my effort to get the DBT3 queries to work and to get DBT3 performance stats. One good thing came out from it is the it triggered the issue as described in MCOL-240, ExeMgr gets restarted if the subquery returned a NULL value.

 Comments   
Comment by David Hall (Inactive) [ 2016-09-07 ]

Just changed the way the thing looked for length.

Comment by Andrew Hutchings (Inactive) [ 2016-09-07 ]

Code review done. Fix looks good to me!

Comment by Daniel Lee (Inactive) [ 2016-09-13 ]

Build verified:

Name : mariadb-columnstore-platform
Version : 1.0.3
Release : 1
Architecture: x86_64
Install Date: Tue 13 Sep 2016 09:26:32 AM CDT

Generated at Thu Feb 08 02:19:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.