[MCOL-4672] MariaDB 10.5.8 Columnstore utf8mb4 support Created: 2021-04-08  Updated: 2021-08-30  Resolved: 2021-08-30

Status: Closed
Project: MariaDB ColumnStore
Component/s: ?, MDB Plugin
Affects Version/s: 5.5.1
Fix Version/s: 5.6.1

Type: New Feature Priority: Major
Reporter: andreas eschbacher Assignee: David Hall (Inactive)
Resolution: Fixed Votes: 0
Labels: utf8mb4
Environment:
version 10.5.8-MariaDB
version_comment MariaDB Server

Attachments: PNG File UTF8.png     PNG File utf8mb4_data.png     PNG File utf8mb4_data02.png    
Issue Links:
Blocks
blocks MCOL-1374 Support utf8mb4 Closed
is blocked by MCOL-4707 Columnstore Bugs 5.4.1 Open

 Description   

Hello,

we still have a problem with columnstore engine and utf8mb4
https://jira.mariadb.org/browse/MCOL-1374

Using the create table statements with no engine specified, it works (we can import Data)! Conclusion: columnstore engine does not seem to be compatible with utf8mb4!
How can I query the exact Columnstore Version because in MariaDB 10.5.8 is only the text MaraDB Server in parameter version_comment.

Columnstore_commit_hash source
Columnstore_version 5.4.1

Br Andreas

 Comments   
Comment by David Hall (Inactive) [ 2021-04-08 ]

It's a little unintuitive, but this will show the columnstore version:
SHOW STATUS LIKE 'columnstore%';

Comment by andreas eschbacher [ 2021-04-08 ]

Thank you David,
so our Version is

Columnstore_commit_hash source
Columnstore_version 5.4.1

We have Problems when importing Json Data with ETL Process via Pentaho (Mariadb-java-client.2.7.2) when we use utf8mb4.


how it should look like with:
(utf8)

OUR MAIN PROBLEM IS:
ColumnStore with BulkLoader when the DB is set to UTF8MB4 won't work.

Comment by andreas eschbacher [ 2021-04-09 ]

tested variations:
we tried to save a special ambulance emoji
1) on EBI01-Server utf8mb4 with bulk loader, STRICT_TRANS_TABLES and without columstore leads to ????.
2) on EBI01-Server with utf8mb4 with table output step, STRICT_TRANS_TABLES and without columstore leads to right icon.
3) on EBI03 (DB Version 10.5) with utf8mb4 using bulk loader, STRICT_TRANS_TABLES and without columnstore leads to ????.
So even using NO columnstore, utf8mb4 does not seem to be able to store the icon correctly using the bulk loader, independent of the MariaDB version.

Comment by andreas eschbacher [ 2021-04-12 ]

We found a workaround to be able to store the Ambulance-Emoji.

Our Base table is still utf8 our new Dimensions table is set to utf8mb4.
The Emoji in this Pentaho-ETL Step is not copied via a bulk-loader step.
This is works fine for us.

Comment by David Hall (Inactive) [ 2021-08-30 ]

The Columnstore Version is shown by
SELECT calgetversion();

Problems with load data infile and utf8mb4 have been correct in version 5.6.1.

Hope this helps.

Comment by David Hall (Inactive) [ 2021-08-30 ]

Fixed in 5.6.1

Generated at Thu Feb 08 02:52:07 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.