Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-1335

Data load through PDi adapter appears corrupted in the database

    XMLWordPrintable

    Details

      Description

      1. Start bulk load of a large CSV file with pentaho bulk load adapter. File contains 121 million rows (9 GB in size).
      2. The load inserts data into table where all columns are of type = Varchar (256)

      CREATE TABLE F_TRANS_DELTA (
      trans_id       VARCHAR(256) NULL,
      customer_id    VARCHAR(256) NULL,
      merchant_id    VARCHAR(256) NULL,
      card_id        VARCHAR(256) NULL,
      trans_datetime VARCHAR(256) NULL,
      trans_type     VARCHAR(256) NULL,
      amount         VARCHAR(256) NULL,
      reversed       VARCHAR(256) NULL,
      response_code  VARCHAR(256) NULL,
      is_first_topup VARCHAR(256) NULL
      ) ENGINE=Columnstore DEFAULT CHARSET=utf8;
      

      3. The pentaho job finished successfully and the data seems to be inserted into the database.
      4. The following queries can be executed succesfully:

      select * from f_trans_delta limit 10;
      select count(*) from f_trans_delta;
      

      5. Run query:

      select * from f_trans_delta where merchant_id='12558';
      

      Result:
      1. Error in mcsmysql client:
      ERROR 1815 (HY000): Internal error: IDB-2035: An internal error occurred. Check the error log file & contact support.
      2. Error in messages log file on PM1:
      Apr 12 13:12:04 mariadb-59f24c1f-215-1 Calpont[801]: 04.760635 |0|0|0| E 00 CAL0000: /data/buildbot/bb-worker/nightly-rpm-centos7/build/mariadb-columnstore-server/mariadb-columnstore-engine/primitives/primproc/dictstep.cpp@353: assertion 'pt[primMsg->NVALS].offsetIndex != 0' failed
      Apr 12 13:12:04 mariadb-59f24c1f-215-1 PrimProc[801]: 04.761228 |0|0|0| W 28 CAL0000: IDB-2035: An internal error occurred. Check the error log file & contact support.

      Expected:
      Successful query execution.

        Attachments

          Activity

            People

            Assignee:
            alexey.vorovich alexey vorovich
            Reporter:
            elena.kotsinova Elena Kotsinova
            Votes:
            7 Vote for this issue
            Watchers:
            12 Start watching this issue

              Dates

              Created:
              Updated:

                Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.