[MCOL-3323] Pentaho plugin strange behavior when load data into table Created: 2019-05-22  Updated: 2023-10-26  Resolved: 2023-10-25

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.2.3
Fix Version/s: Icebox

Type: Bug Priority: Critical
Reporter: Nicola Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None
Environment:

Vmware environment 6.7
all vm with Oracle Linux 7.6
MariadDB columnstore 1.2.3 :
3 PM
2 UM
1 Pentaho 8.2 with pentaho kettle adapter 1.2.3


Issue Links:
Blocks
is blocked by MDEV-21504 Collation: Create shared library for ... Closed

 Description   

Hi,
we've a lot job that load with plugin in mariadb columnstore, but one job in particular is creating problem.
There are jobs that load about 500k records into the Columnstore tables without problems, This job instead fails to enter even 100 records (with limit a 10 records the job go in success).
it seems that the plugin is not able to commit the inserted records, in fact after several minutes it goes into error.

This is the table where I can load about 500 thousand records at a time.
CREATE TABLE `c6_messaggio_in` (
`PROG_MESSAGGIO_IN_D` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
`PROG_TIPO_MSG_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`PROG_STATO_ELABORAZIONE_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`MESSAGGIO` TEXT NULL DEFAULT NULL,
`DATA_RICEZIONE` DATETIME NULL DEFAULT NULL,
`IP_ADDRESS` VARCHAR(50) NULL DEFAULT NULL,
`ID_MSG_APP` VARCHAR(50) NULL DEFAULT NULL,
`MSG_OPER` TEXT NULL DEFAULT NULL,
`COD_MITT` VARCHAR(1) NULL DEFAULT NULL,
`USER_INSERT` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
`DATE_INSERT` DATETIME NULL DEFAULT NULL,
`USER_MODIFY` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
`DATE_MODIFY` DATETIME NULL DEFAULT NULL,
`USER_DELETE` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
`DATE_DELETE` DATETIME NULL DEFAULT NULL,
`GATEWAY` VARCHAR(1) NULL DEFAULT 'N',
`CODEID` VARCHAR(50) NULL DEFAULT NULL,
`ID_MSG_APP_REQ` VARCHAR(50) NULL DEFAULT NULL
)
COLLATE='utf8mb4_general_ci'
ENGINE=Columnstore
;

This is the table where I can load about 10 records at a time but with 100 record it crash.
CREATE TABLE `c6_messaggio_out` (
`PROG_MESSAGGIO_OUT_ID` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
`PROG_TIPO_MSG_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`MESSAGGIO` TEXT NULL DEFAULT NULL,
`DATA_TRASMISSIONE` DATETIME NULL DEFAULT NULL,
`ID_MSG_APP` VARCHAR(50) NULL DEFAULT NULL,
`MSG_OPER` TEXT NULL DEFAULT NULL,
`COD_MITT` VARCHAR(1) NULL DEFAULT NULL,
`PROG_STATO_ELABORAZIONE_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`USER_INSERT` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`DATE_INSERT` DATETIME NULL DEFAULT NULL,
`USER_MODIFY` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`DATE_MODIFY` DATETIME NULL DEFAULT NULL,
`USER_DELETE` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
`DATE_DELETE` DATETIME NULL DEFAULT NULL,
`CODEID` VARCHAR(50) NULL DEFAULT NULL,
`GATEWAY` VARCHAR(1) NULL DEFAULT 'N'
)
COLLATE='utf8mb4_general_ci'
ENGINE=Columnstore
;

I suspect that clob (text) fields can create difficulties when they are loaded by the plugin, but at the same time the c6_messaggio_in table which also has clob fields has no problems.

Sorry for my bad english.
Thanks,
Regards
Nicola Battista



 Comments   
Comment by Nicola [ 2019-06-19 ]

Hi,
Any news?

Thanks
Regards
Nicola Battista

Generated at Thu Feb 08 02:41:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.