[MCOL-1322] data corruption when setting NUMERIC columns to NULL through the python and C++ API Created: 2018-04-04  Updated: 2023-10-26  Resolved: 2018-10-03

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.1.3
Fix Version/s: 1.1.7, 1.2.0

Type: Bug Priority: Major
Reporter: Dimitris Theodorou Assignee: Jens Röwekamp (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by MCOL-1570 Wrong data when inserting NULL into D... Closed
Sprint: 2018-18

 Description   

Using the bulkInsert.setNull() function on a NUMERIC column results in data corruption. The columns and the ones that follow are affected.

A test case that reproduces this:

The table

CREATE TABLE `test_null_decimal3` (
 
  `id` varchar(128) NOT NULL,
 
  `num1` decimal(8,6) DEFAULT NULL,
 
  `num2` decimal(9,6) DEFAULT NULL,
 
  `b` tinyint(1) DEFAULT NULL
 
) ENGINE=Columnstore DEFAULT CHARSET=utf8;

Python code usage

import pymcsapi
 
driver = pymcsapi.ColumnStoreDriver(columnstore_config_path)
 
bulk_insert = driver.createBulkInsert(db, table, 0, 0)
 
 
 
# INSERT INTO table VALUES ('text', NULL, NULL, 4)
 
bulk_insert.setColumn(0, 'text')
 
bulk_insert.setNull(1)
 
bulk_insert.setNull(2)
 
bulk_insert.setColumn(3, 4)
 
 
 
bulk_insert.writeRow()
 
bulk_insert.commit()

The commit finishes successfully with no error counts. Querying the table gives this result:

MariaDB [db]> select * from test_null_decimal3;
 
+------+----------+----------+------+
 
| id   | num1     | num2     | b    |
 
+------+----------+----------+------+
 
| text | 0.000000 | 8.388608 |    0 |
 
+------+----------+----------+------+
 
1 row in set (0.10 sec)

You can see the values for num2 and b are wrong. I've had corruption errors on the table after this:

2018-04-04 17:27:02 (27472) ERR  : writeToFile: error initializing to-be-compressed buffer for OID 5625;  Error in uncompressing data.  [1652]
 
2018-04-04 17:27:02 (27472) ERR  : writeToFileExtentCheck: write token extent failed:  Error in uncompressing data.  [1652]
 
2018-04-04 17:27:02 (27472) ERR  : finishParsing: flush error with column ;  Error in uncompressing data.  [1652]
 
2018-04-04 17:27:02 (27472) ERR  : setParseComplete completion error; Failed to load table: t;  Error in uncompressing data.  [1652]

Doing the same insert through INSERT or cpimport gives the correct results. I'm running the 1.1.3 release

 Comments   
Comment by Dimitris Theodorou [ 2018-04-04 ]

May be related to https://jira.mariadb.org/browse/MCOL-1295

Comment by Jens Röwekamp (Inactive) [ 2018-06-28 ]

Confirmed bug with mcsapi 1.1.5 where MCOL-1295 was fixed.

Used provided Python example and translated code to C++.
The same buggy behaviour was shown in both languages.

#include <libmcsapi/mcsapi.h>
 
#include <string>
 
 
 
int main(int argc, char* argv[])
 
{
 
    mcsapi::ColumnStoreDriver* driver = new mcsapi::ColumnStoreDriver();
 
    mcsapi::ColumnStoreBulkInsert* bulk = driver->createBulkInsert("test", "mcol_1322", 0, 0);
 
    std::string t = "text";
 
    bulk->setColumn(0, t);
 
    bulk->setNull(1);
 
    bulk->setNull(2);
 
    bulk->setColumn(3,4);
 
    bulk->writeRow();
 
    bulk->commit();
 
    delete bulk;
 
    delete driver;
 
}

Comment by Andrew Hutchings (Inactive) [ 2018-10-02 ]

For QA: The dataconvert-null test in the API test suite has been expanded to cover this.

Comment by Jens Röwekamp (Inactive) [ 2018-10-03 ]

test suite successfully executed on Debian 9, CentOS 7, Windows 10

Generated at Thu Feb 08 02:27:52 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.