[MDEV-7650] COLUMN_CHECK() does not always detect data truncation Created: 2015-03-01  Updated: 2016-04-19  Resolved: 2016-04-19

Status: Closed
Project: MariaDB Server
Component/s: Dynamic Columns
Affects Version/s: 10.0.16
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Tom Assignee: Oleksandr Byelkin
Resolution: Not a Bug Votes: 0
Labels: verified
Environment:

Server version: 10.0.16-MariaDB Homebrew
Darwin fsb 13.4.0 Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64



 Description   

Bug is fully described in the following test script:

select @@max_allowed_packet;
# 1073741824
 
select column_check(column_create('a', 1, 'b', repeat('1234567890', 6554)));
# 1
 
set @txt = (select repeat('1234567890', 6554));
select column_check(column_create('a', 1, 'txt', @txt));
# 1
 
select column_check(column_create('a', 1, 'css', @txt));
# 1
 
create database if not exists dyncol_test;
use dyncol_test;
drop table if exists t;
create table t (a int, b varchar(255), dcols blob);
 
insert into t (a, b, dcols) values (1, 'two', column_create('txt', @txt));
select column_check(dcols),
    substring(column_get(dcols, 'txt' as char), 1, 40),
    substring(column_json(dcols), 1, 40) from t;
# 1   1234567890123456789012345678901234567890    {"txt":"12345678901234567890123456789012
 
delete from t;
insert into t (a, b, dcols) values (1, 'two', column_create('one', 1, 'txt', @txt));
select column_check(dcols),
    substring(column_get(dcols, 'txt' as char), 1, 40),
    substring(column_json(dcols), 1, 40) from t;
# 1   1234567890123456789012345678901234567890    {"one":1,"txt":"123456789012345678901234
 
delete from t;
insert into t (a, b, dcols) values (1, 'two', column_create('css', @txt));
select column_check(dcols),
    substring(column_get(dcols, 'css' as char), 1, 40),
    substring(column_json(dcols), 1, 40) from t;
# 1   1234567890123456789012345678901234567890    {"css":"12345678901234567890123456789012
 
delete from t;
insert into t (a, b, dcols) values (1, 'two', column_create('one', 1, 'css', @txt));
select column_check(dcols),
    substring(column_get(dcols, 'css' as char), 1, 40),
    substring(column_json(dcols), 1, 40) from t;
# ERROR 1919 (HY000) at line 31: Encountered illegal format of dynamic column string
 
# Looks like the conditions for the error are:
# 1. dynamic column blob size greater than 64kB
# 2. dynamic column blob is saved to a table
# 3. dynamic column blob contains more than one dynamic columns
# 4. the dynamic column with the long value is named 'css' and not 'txt' !!!



 Comments   
Comment by Elena Stepanova [ 2015-03-01 ]

Thanks for the report and the test case.

Comment by Tom [ 2015-03-01 ]

Dynamic columns are packed into the blob in alphabetical order.

create database if not exists dyncol_test;
use dyncol_test;
drop table if exists t;
create table t (id int, dcols blob);
 
set @txt = repeat('1234567890', 6551);
insert into t (id, dcols) values 
    (1, column_create('aaa', 1, 'aab', @txt)),
    (2, column_create('aac', 1, 'aab', @txt)),
    (3, column_create('aaa', 1, 'aab', concat(@txt, '1234567890'))),
    (4, column_create('aac', 1, 'aab', concat(@txt, '1234567890')));
 
select id, column_check(dcols) from t;
>>>
id	column_check(dcols)
1	1
2	1
3	1
4	0

Comment by Elena Stepanova [ 2015-03-01 ]

Right. Long ago there was a discussion about consequences of data truncation in dynamic columns (which happens here, all INSERTs produce the warning); but I can't find or remember what the outcome of that discussion was, so I leave it to Sanja to comment, he will remember for sure.

Comment by Sergei Golubchik [ 2015-03-01 ]

A clearer test case:

MariaDB [test]> select column_check(left(column_create('one', 1, 'txt', @txt), 65535));
+-----------------------------------------------------------------+
| column_check(left(column_create('one', 1, 'txt', @txt), 65535)) |
+-----------------------------------------------------------------+
|                                                               1 |
+-----------------------------------------------------------------+
1 row in set (0.00 sec)
 
MariaDB [test]> select column_check(left(column_create('one', 1, 'css', @txt), 65535));
+-----------------------------------------------------------------+
| column_check(left(column_create('one', 1, 'css', @txt), 65535)) |
+-----------------------------------------------------------------+
|                                                               0 |
+-----------------------------------------------------------------+
1 row in set (0.00 sec)

Comment by Tom [ 2015-03-01 ]

I did not know that blobs were restricted to such a small size.

This was therefore my foolish error. Caught out again by the crazy selection of MySQL types.

I do not know how to delete or close this issue.

Comment by Sergei Golubchik [ 2015-03-02 ]

While we can close it, I'd suggest to keep it as a COLUMN_CHECK bug. I'll change the subject to better reflect what this bug is about.

Comment by Oleksandr Byelkin [ 2016-04-19 ]

It is not a bug. Dynamic columns goal was to keep everything compact but not check integrity.

Length of the string is not stored, it detected by beginning of other dynamic field or by end of whole string where dynamic fields stored. So if last field happened to be string you can chop from string as much as string length without destroying internal data integrity.

If one need to check integrity other means should be used (like SHA1 or control sum or even signature).

Generated at Thu Feb 08 07:21:14 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.