Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-774

cpimport outputs value truncation warning when read buffer (-b) is set to 1

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.10, 1.1.0
    • Fix Version/s: 1.4.1
    • Component/s: None
    • Labels:
      None

      Description

      Build tested: Github source

      I did the following imports of the 1g orders table.

      The first run was fine. The 2nd run reported truncation warnings when I changed the -b value from 4 to 1. I changed the -r value (readers) and still got the same warnings.

      There are indeed truncated values loaded to the columns.

      [root@localhost bin]# ./cpimport -r 4 -b 4 mytest orders /data/qa/source/dbt3/1g/orders.tbl
      Locale is : C
      number of read threads : 4

      Using table OID 3004 as the default JOB ID
      Input file(s) will be read from : /usr/local/mariadb/columnstore/bin
      Job description file : /usr/local/mariadb/columnstore/data/bulk/tmpjob/3004_D20170615_T152858_S014732_Job_3004.xml
      Log file for this job: /usr/local/mariadb/columnstore/data/bulk/log/Job_3004.log
      2017-06-15 15:28:58 (16773) INFO : successfully loaded job file /usr/local/mariadb/columnstore/data/bulk/tmpjob/3004_D20170615_T152858_S014732_Job_3004.xml
      2017-06-15 15:28:58 (16773) INFO : Job file loaded, run time for this step : 0.0493131 seconds
      2017-06-15 15:28:58 (16773) INFO : PreProcessing check starts
      2017-06-15 15:28:58 (16773) INFO : input data file /data/qa/source/dbt3/1g/orders.tbl
      2017-06-15 15:28:58 (16773) INFO : PreProcessing check completed
      2017-06-15 15:28:58 (16773) INFO : preProcess completed, run time for this step : 0.071727 seconds
      2017-06-15 15:28:58 (16773) INFO : No of Read Threads Spawned = 4
      2017-06-15 15:28:58 (16773) INFO : No of Parse Threads Spawned = 3
      2017-06-15 15:29:01 (16773) INFO : For table mytest.orders: 1500000 rows processed and 1500000 rows inserted.
      2017-06-15 15:29:02 (16773) INFO : Bulk load completed, total run time : 4.13393 seconds

      [root@localhost bin]# ./cpimport -r 4 -b 1 mytest orders /data/qa/source/dbt3/1g/orders.tbl
      Locale is : C
      number of read threads : 4

      Using table OID 3004 as the default JOB ID
      Input file(s) will be read from : /usr/local/mariadb/columnstore/bin
      Job description file : /usr/local/mariadb/columnstore/data/bulk/tmpjob/3004_D20170615_T153019_S007770_Job_3004.xml
      Log file for this job: /usr/local/mariadb/columnstore/data/bulk/log/Job_3004.log
      2017-06-15 15:30:19 (17127) INFO : successfully loaded job file /usr/local/mariadb/columnstore/data/bulk/tmpjob/3004_D20170615_T153019_S007770_Job_3004.xml
      2017-06-15 15:30:19 (17127) INFO : Job file loaded, run time for this step : 0.0194149 seconds
      2017-06-15 15:30:19 (17127) INFO : PreProcessing check starts
      2017-06-15 15:30:19 (17127) INFO : input data file /data/qa/source/dbt3/1g/orders.tbl
      2017-06-15 15:30:19 (17127) INFO : PreProcessing check completed
      2017-06-15 15:30:19 (17127) INFO : preProcess completed, run time for this step : 0.078114 seconds
      2017-06-15 15:30:19 (17127) INFO : No of Read Threads Spawned = 4
      2017-06-15 15:30:19 (17127) INFO : No of Parse Threads Spawned = 3
      2017-06-15 15:30:23 (17127) INFO : For table mytest.orders: 1500000 rows processed and 1500000 rows inserted.
      2017-06-15 15:30:23 (17127) WARN : Column mytest.orders.o_orderstatus; Number of character strings truncated: 142
      2017-06-15 15:30:23 (17127) WARN : Column mytest.orders.o_orderdate; Number of invalid dates replaced with zero value : 142
      2017-06-15 15:30:23 (17127) INFO : Bulk load completed, total run time : 4.59954 seconds

      [root@localhost bin]# ./cpimport -r 1 -b 1 mytest orders /data/qa/source/dbt3/1g/orders.tbl
      Locale is : C
      number of read threads : 1

      Using table OID 3004 as the default JOB ID
      Input file(s) will be read from : /usr/local/mariadb/columnstore/bin
      Job description file : /usr/local/mariadb/columnstore/data/bulk/tmpjob/3004_D20170615_T215908_S792849_Job_3004.xml
      Log file for this job: /usr/local/mariadb/columnstore/data/bulk/log/Job_3004.log
      2017-06-15 21:59:08 (758) INFO : successfully loaded job file /usr/local/mariadb/columnstore/data/bulk/tmpjob/3004_D20170615_T215908_S792849_Job_3004.xml
      2017-06-15 21:59:08 (758) INFO : Job file loaded, run time for this step : 0.020762 seconds
      2017-06-15 21:59:08 (758) INFO : PreProcessing check starts
      2017-06-15 21:59:08 (758) INFO : input data file /data/qa/source/dbt3/1g/orders.tbl
      2017-06-15 21:59:08 (758) INFO : PreProcessing check completed
      2017-06-15 21:59:08 (758) INFO : preProcess completed, run time for this step : 0.090452 seconds
      2017-06-15 21:59:08 (758) INFO : No of Read Threads Spawned = 1
      2017-06-15 21:59:08 (758) INFO : No of Parse Threads Spawned = 3
      2017-06-15 21:59:11 (758) INFO : For table mytest.orders: 1500000 rows processed and 1500000 rows inserted.
      2017-06-15 21:59:11 (758) WARN : Column mytest.orders.o_orderstatus; Number of character strings truncated: 142
      2017-06-15 21:59:11 (758) WARN : Column mytest.orders.o_orderdate; Number of invalid dates replaced with zero value : 142
      2017-06-15 21:59:11 (758) INFO : Bulk load completed, total run time : 3.1271 seconds

        Attachments

          Activity

            People

            Assignee:
            dleeyh Daniel Lee
            Reporter:
            dleeyh Daniel Lee
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: