[MCOL-457] null bit header error when the number of NULL columns is divisible by 8 and the last column is a NOT NULL Created: 2016-12-08  Updated: 2016-12-12  Resolved: 2016-12-12

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.0.5
Fix Version/s: 1.0.6

Type: Bug Priority: Major
Reporter: David Thompson (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2016-24

 Description   

It appears that, at least for this case, if the first column in a column store table is not null then it errors out with:
ERROR 1815 (HY000): Internal error: Null bit header is wrong size
On an insert into select statement. A workaround is to remove the not null constraint on the first column.

To reproduce:

CREATE TABLE t1_cs (
 
   c1 varchar(7) NOT NULL,
 
   c2 varchar(3) NOT NULL,
 
   c3 varchar(6) NOT NULL,
 
   c4 varchar(18) DEFAULT NULL,
 
   c5 varchar(9) DEFAULT NULL,
 
   c6 varchar(10) DEFAULT NULL,
 
   n1 decimal(11,2) DEFAULT NULL,
 
   n2 decimal(13,2) DEFAULT NULL,
 
   c7 varchar(5) DEFAULT NULL,
 
   d1 date DEFAULT NULL,
 
   d2 datetime DEFAULT NULL,
 
   d3   datetime NOT NULL,
 
   d4 datetime NOT NULL
 
) ENGINE=columnstore;
 
 
 
CREATE TABLE t1_in (
 
   c1 varchar(7) NOT NULL,
 
   c2 varchar(3) NOT NULL,
 
   c3 varchar(6) NOT NULL,
 
   c4 varchar(18) DEFAULT NULL,
 
   c5 varchar(9) DEFAULT NULL,
 
   c6 varchar(10) DEFAULT NULL,
 
   n1 decimal(11,2) DEFAULT NULL,
 
   n2 decimal(13,2) DEFAULT NULL,
 
   c7 varchar(5) DEFAULT NULL,
 
   d1 date DEFAULT NULL,
 
   d2 datetime DEFAULT NULL,
 
   d3   datetime NOT NULL,
 
   d4 datetime NOT NULL
 
) ENGINE=InnoDB;
 
 
 
 
 
insert into t1_in values (
 
'ABC',
 
'ABC',
 
'ABC',
 
'ABC',
 
'ABC',
 
'ABC',
 
1.2,
 
1.2,
 
'ABC',
 
now(),
 
now(),
 
now(),
 
now()
 
);
 
 
 
insert into t1_cs select * from t1_in;
 
ERROR 1815 (HY000): Internal error: Null bit header is wrong size
 
 
 
drop table t1_cs;
 
 
 
CREATE TABLE t1_cs (
 
   c1 varchar(7),
 
   c2 varchar(3) NOT NULL,
 
   c3 varchar(6) NOT NULL,
 
   c4 varchar(18) DEFAULT NULL,
 
   c5 varchar(9) DEFAULT NULL,
 
   c6 varchar(10) DEFAULT NULL,
 
   n1 decimal(11,2) DEFAULT NULL,
 
   n2 decimal(13,2) DEFAULT NULL,
 
   c7 varchar(5) DEFAULT NULL,
 
   d1 date DEFAULT NULL,
 
   d2 datetime DEFAULT NULL,
 
   d3   datetime NOT NULL,
 
   d4 datetime NOT NULL
 
) ENGINE=columnstore;
 
 
 
insert into t1_cs select * from t1_in;
 
ERROR 1815 (HY000): Internal error: PM2 : Bulkload Parse (thread 2) Failed for Table test.t1_cs during parsing.  Terminating this job.
 
 
 
exit
 
 
 
mcsmysql test
 
insert into t1_cs select * from t1_in;
 
--works



 Comments   
Comment by David Thompson (Inactive) [ 2016-12-09 ]

Another workaround is to remove not null from the last 2 columns instead. In the original test case there is a cpimport err and bad file in the db directory which alludes to this, the data file while binary was missing the last 2 date values.

Comment by Andrew Hutchings (Inactive) [ 2016-12-09 ]

OK, this is a NULL bitmap problem when there are exactly 8 columns that could be NULL in the table and there is a NOT NULL column after the last NULLable column. I know where this will be broken in the code...

Comment by Andrew Hutchings (Inactive) [ 2016-12-09 ]

The description in this Jira is a good test case.

Longer explanation of fix:
The DML processor loops over every column to see if it is part of the NULL column bitmask received from MariaDB. Once we have looped over 8 NULLable bits we need the next byte so the bitmask pointer is moved to the next byte. After we have read through the bits for 8 NULL columns in a table it is possible there are only NOT NULL columns left, but the pointer has been moved. On the next loop regardless of whether it is a NULL or NOT NULL column we check to see if the pointer has been moved too far. It has but it doesn't matter because the pointer won't be used to actually read. We generate an error where there is really no problem.
This patch moves the pointer check to only run when we hit a column that could be NULL.

Comment by Daniel Lee (Inactive) [ 2016-12-12 ]

Build verified: 1.0.601

mcsadmin> getsoft
getsoftwareinfo Mon Dec 12 19:15:55 2016

Name : mariadb-columnstore-platform
Version : 1.0.6
Release : 1
Architecture: x86_64
Install Date: Mon 12 Dec 2016 06:34:30 PM UTC
Group : Applications/Databases
Size : 10016671
License : Copyright (c) 2016 MariaDB Corporation Ab., all rights reserved; redistributable under the terms of the GPL, see the file COPYING for details.
Signature : (none)
Source RPM : mariadb-columnstore-platform-1.0.6-1.src.rpm
Build Date : Mon 12 Dec 2016 04:45:55 PM UTC

Verified with provided scenario. The error msg no longer being outputted.

Generated at Thu Feb 08 02:21:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.