Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-3323

Pentaho plugin strange behavior when load data into table

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Critical
    • Resolution: Won't Fix
    • 1.2.3
    • Icebox
    • None
    • None
    • Vmware environment 6.7
      all vm with Oracle Linux 7.6
      MariadDB columnstore 1.2.3 :
      3 PM
      2 UM
      1 Pentaho 8.2 with pentaho kettle adapter 1.2.3

    Description

      Hi,
      we've a lot job that load with plugin in mariadb columnstore, but one job in particular is creating problem.
      There are jobs that load about 500k records into the Columnstore tables without problems, This job instead fails to enter even 100 records (with limit a 10 records the job go in success).
      it seems that the plugin is not able to commit the inserted records, in fact after several minutes it goes into error.

      This is the table where I can load about 500 thousand records at a time.
      CREATE TABLE `c6_messaggio_in` (
      `PROG_MESSAGGIO_IN_D` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
      `PROG_TIPO_MSG_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `PROG_STATO_ELABORAZIONE_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `MESSAGGIO` TEXT NULL DEFAULT NULL,
      `DATA_RICEZIONE` DATETIME NULL DEFAULT NULL,
      `IP_ADDRESS` VARCHAR(50) NULL DEFAULT NULL,
      `ID_MSG_APP` VARCHAR(50) NULL DEFAULT NULL,
      `MSG_OPER` TEXT NULL DEFAULT NULL,
      `COD_MITT` VARCHAR(1) NULL DEFAULT NULL,
      `USER_INSERT` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
      `DATE_INSERT` DATETIME NULL DEFAULT NULL,
      `USER_MODIFY` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
      `DATE_MODIFY` DATETIME NULL DEFAULT NULL,
      `USER_DELETE` SMALLINT(5) UNSIGNED NULL DEFAULT NULL,
      `DATE_DELETE` DATETIME NULL DEFAULT NULL,
      `GATEWAY` VARCHAR(1) NULL DEFAULT 'N',
      `CODEID` VARCHAR(50) NULL DEFAULT NULL,
      `ID_MSG_APP_REQ` VARCHAR(50) NULL DEFAULT NULL
      )
      COLLATE='utf8mb4_general_ci'
      ENGINE=Columnstore
      ;

      This is the table where I can load about 10 records at a time but with 100 record it crash.
      CREATE TABLE `c6_messaggio_out` (
      `PROG_MESSAGGIO_OUT_ID` BIGINT(20) UNSIGNED NULL DEFAULT NULL,
      `PROG_TIPO_MSG_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `MESSAGGIO` TEXT NULL DEFAULT NULL,
      `DATA_TRASMISSIONE` DATETIME NULL DEFAULT NULL,
      `ID_MSG_APP` VARCHAR(50) NULL DEFAULT NULL,
      `MSG_OPER` TEXT NULL DEFAULT NULL,
      `COD_MITT` VARCHAR(1) NULL DEFAULT NULL,
      `PROG_STATO_ELABORAZIONE_ID` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `USER_INSERT` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `DATE_INSERT` DATETIME NULL DEFAULT NULL,
      `USER_MODIFY` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `DATE_MODIFY` DATETIME NULL DEFAULT NULL,
      `USER_DELETE` TINYINT(3) UNSIGNED NULL DEFAULT NULL,
      `DATE_DELETE` DATETIME NULL DEFAULT NULL,
      `CODEID` VARCHAR(50) NULL DEFAULT NULL,
      `GATEWAY` VARCHAR(1) NULL DEFAULT 'N'
      )
      COLLATE='utf8mb4_general_ci'
      ENGINE=Columnstore
      ;

      I suspect that clob (text) fields can create difficulties when they are loaded by the plugin, but at the same time the c6_messaggio_in table which also has clob fields has no problems.

      Sorry for my bad english.
      Thanks,
      Regards
      Nicola Battista

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              nbattista89 Nicola
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.