[MCOL-129] INSERTIONS are lost, no ERROR reported for out-of-range values in STRICT mode Created: 2016-06-12  Updated: 2016-11-23  Resolved: 2016-11-23

Status: Closed
Project: MariaDB ColumnStore
Component/s: DMLProc
Affects Version/s: None
Fix Version/s: 1.0.5

Type: Bug Priority: Major
Reporter: Justin Swanhart (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: relnote

Sprint: 2016-21, 2016-22, 2016-23

 Description   

insertion of -128 works for MyISAM

mysql> create table t2(c1 tinyint) ENGINE=MYISAM;
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t2 values(-128);
Query OK, 1 row affected (0.01 sec)

mysql> select * from t2;
------

c1

------

-128

------
1 row in set (0.00 sec)

mysql> drop table t2;
Query OK, 0 rows affected (0.00 sec)

value is "conformed" to smallest possible value on ColumnStore:

mysql> create table t2(c1 tinyint) engine=columnstore;
Query OK, 0 rows affected (0.39 sec)

mysql> insert into t2 values(-128);
Query OK, 1 row affected, 1 warning (0.17 sec)

mysql> show warnings;
-----------------------------------------------------------------

Level Code Message

-----------------------------------------------------------------

Warning 9999 CAL0001: IDB-2025: Data truncated for column 'c1'

-----------------------------------------------------------------
1 row in set (0.00 sec)

Note above error does not have properly error code or error message, it should be

error=1264, error message=Out of range value for column 'c1' at row 1

mysql> select * from t2;
------

c1

------

-126

------
1 row in set (0.06 sec)

STRICT_MODE should result in ERROR and no row insertion

mysql> set sql_mode=STRICT_ALL_TABLES;
Query OK, 0 rows affected (0.00 sec)

Columnstore says one row was inserted, but raises warning:

mysql> insert into t2 values(-128);
Query OK, 1 row affected, 1 warning (0.06 sec)
mysql> show warnings;
---------------------------------------------------------------

Level Code Message

---------------------------------------------------------------

Error 9999 CAL0001: IDB-2025: Data truncated for column 'c1'

---------------------------------------------------------------
1 row in set (0.00 sec)

Again, note above error does not have properly error code or error message, it should be

error=1264, error message=Out of range value for column 'c1' at row 1

INSERTION FAILED BUT QUERY RETURNED 1 ROW INSERTED!
mysql> select * from t2;
------

c1

------

-126

------
1 row in set (0.01 sec)



 Comments   
Comment by Justin Swanhart (Inactive) [ 2016-06-20 ]

THERE SHOULD BE TWO ROWS IN THE COLUMNSTORE TABLE.

COLUMNSTORE LOSES INSERTIONS!

Comment by Justin Swanhart (Inactive) [ 2016-06-20 ]

The problem happens only on the "reserved" values for columnstore:

MariaDB [test]> set sql_mode=strict_all_tables;

Query OK, 0 rows affected (0.00 sec)

MariaDB [test]> create table lost(c1 tinyint) engine=columnstore;
Query OK, 0 rows affected (0.91 sec)

CORRECT behavior

MariaDB [test]> insert into lost values(10000000);
ERROR 1264 (22003): Out of range value for column 'c1' at row 1

BUGGY behavior

MariaDB [test]> insert into lost values(-128);
Query OK, 1 row affected, 1 warning (0.21 sec)

MariaDB [test]> select * from lost;
Empty set (0.25 sec)

Comment by Justin Swanhart (Inactive) [ 2016-06-20 ]

There is no way to work around this issue - STRICT MODE is the recommended mode to prevent invalid data from entering the database. Lost insertions reported as success are not acceptable and can not be worked around because in non-strict mode the value will be changed to a -126 WHICH IS NOT ACCEPTABLE.

A bug with severe loss of functionality and no workaround is a BLOCKER.

Comment by Justin Swanhart (Inactive) [ 2016-06-20 ]

This is a data loss issue with no workaround. It is not 'minor'.

So that we are clear, please provide the criteria for determining what severity a bug is. Nobody is supposed to apply a personal decision for severity of bugs, there is (historically) a clear guideline regarding what constitutes what severity.

In MySQL and MariaDB, a bug that results in data loss (any major loss of functionality) and has no workaround (as this does not) is the highest severity.

This is a DATA LOSS issue with NO WORKAROUND.

Comment by Justin Swanhart (Inactive) [ 2016-06-20 ]

I am a SENIOR SOFTWARE ENGINEER. I am qualified to determine the severity level of a bug that I report. There is clearly a failure of process here somewhere.

Comment by David Thompson (Inactive) [ 2016-10-03 ]

There are documented limitations with datatype ranges where certain values are reserved and not available to a user. This is a separate issue / enhancement and should be prioritized accordingly.

The real issue here is that with sql_mode STRICT_ALL_TABLES, when you try to insert the reserved value there is no error and the row is not inserted. This also happens with multiple columns where the other columns are valid values but only say c1 is -128.

In the case of an update, the update goes through with the value saturation happening and a warning being issued.

The behavior is the same in infinidb so this is a legacy issue rather than an issue with the porting to mariadb server 10.x. There is a workaround however which is to not use the STRICT_ALL_TABLES mode.

Ideally the system should correctly report the error.

Comment by Daniel Lee (Inactive) [ 2016-11-22 ]

There is no comments on what changes was made to address this ticket, except what's in the ticket history. Is the ticket ready for testing?

Comment by Daniel Lee (Inactive) [ 2016-11-22 ]

Build tested: 1.0.5-1

mcsadmin> getsoft
getsoftwareinfo Tue Nov 22 10:13:49 2016

Name : mariadb-columnstore-platform
Version : 1.0.5
Release : 1
Architecture: x86_64
Install Date: Tue 22 Nov 2016 10:10:12 AM CST
Group : Applications/Databases
Size : 9990112
License : Copyright (c) 2016 MariaDB Corporation Ab., all rights reserved; redistributable under the terms of the GPL, see the file COPYING for details.
Signature : (none)
Source RPM : mariadb-columnstore-platform-1.0.5-1.src.rpm
Build Date : Mon 21 Nov 2016 07:29:37 PM CST

With sql_mode=STRICT_ALL_TABLES, the row was not inserted as expected, but the error msg is the same as the warning in non STRICT mode. "IDB-2025: Data truncated for column 'c1' " seem to indicate that the value was truncated and inserted.

Reopening for clarification and/or better error msg.

*MariaDB [mytest]> create table t2(c1 tinyint) engine=columnstore;
Query OK, 0 rows affected (0.54 sec)

MariaDB [mytest]> insert into t2 values(-128);
Query OK, 1 row affected, 1 warning (0.12 sec)

MariaDB [mytest]> show warnings;
-----------------------------------------------------------------

Level Code Message

-----------------------------------------------------------------

Warning 1264 CAL0001: IDB-2025: Data truncated for column 'c1'

-----------------------------------------------------------------
1 row in set (0.00 sec)

MariaDB [mytest]> select * from t2;
------

c1

------

-126

------
1 row in set (0.19 sec)

MariaDB [mytest]> set sql_mode=STRICT_ALL_TABLES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [mytest]> insert into t2 values(-128);
ERROR 1264 (22003): CAL0001: IDB-2025: Data truncated for column 'c1'
MariaDB [mytest]> select * from t2;
------

c1

------

-126

------
1 row in set (0.01 sec)

Comment by Andrew Hutchings (Inactive) [ 2016-11-23 ]

Assuming that is one sequence you have put there then that is correct behaviour after the fix. The second insert should fail (and does) so the only data you see is from your insert before you changed the sql_mode.

The new behaviour is to error on truncation for STRICT_ALL_TABLES or STRICT_TRANS_TABLES and do not apply the insert, this is in-line with other engines. Previous behaviour was to emit a warning and apply the insert anyway, just as it was without the sql_mode.

Comment by Daniel Lee (Inactive) [ 2016-11-23 ]

Build verified: 1.0.5-1

mcsadmin> getsoft
getsoftwareinfo Wed Nov 23 14:43:49 2016

Name : mariadb-columnstore-platform
Version : 1.0.5
Release : 1
Architecture: x86_64
Install Date: Wed 23 Nov 2016 02:34:05 PM CST
Group : Applications/Databases
Size : 9990112
License : Copyright (c) 2016 MariaDB Corporation Ab., all rights reserved; redistributable under the terms of the GPL, see the file COPYING for details.
Signature : (none)
Source RPM : mariadb-columnstore-platform-1.0.5-1.src.rpm
Build Date : Wed 23 Nov 2016 08:55:49 AM CST

Gotcha.

Also verified with the latest build, since the last build had and issue.

Generated at Thu Feb 08 02:18:43 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.