|
https://github.com/mariadb-SergeyZefirov/mariadb-columnstore-engine/tree/MCOL-4621-unsigned-join-failure-under-centos8 - relevant branch.
Right now there's nothing more than test to invstigate problems.
Results will follow shortly.
|
|
First of all, running the unsigned_joins test:
First run on clean columnstore after recompile and install:
# ./mtr --force --suite=../storage/columnstore/columnstore/mtr/basic --extern socket=/var/lib/mysql/mysql.sock unsigned_joins
|
perl: warning: Setting locale failed.
|
perl: warning: Please check that your locale settings:
|
LANGUAGE = (unset),
|
LC_ALL = (unset),
|
LANG = "ru_RU.UTF-8"
|
are supported and installed on your system.
|
perl: warning: Falling back to the standard locale ("C").
|
Logging: ./mtr --force --suite=../storage/columnstore/columnstore/mtr/basic --extern socket=/var/lib/mysql/mysql.sock unsigned_joins
|
vardir: /data/mdb-server/mysql-test/var
|
Checking supported features...
|
MariaDB Version 10.6.0
|
- skipping SSL, mysqld not compiled with SSL
|
- binaries are debug compiled
|
Collecting tests...
|
- from '../storage/columnstore/columnstore/mtr/basic'
|
|
==============================================================================
|
|
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
|
worker[1] mysql-test-run: WARNING: running this script as _root_ will cause some tests to be skipped
|
../storage/columnstore/columnstore/mtr/basic.unsigned_joins [ fail ]
|
Test ended at 2021-03-22 10:13:45
|
|
--- ../storage/columnstore/columnstore/mtr/basic/r/unsigned_joins.result 2021-03-22 08:46:28.134564539 +0000
|
+++ ../storage/columnstore/columnstore/mtr/basic/r/unsigned_joins.reject 2021-03-22 10:13:45.325562142 +0000
|
@@ -45,10 +45,10 @@
|
50
|
select count(*) as count11 from customer join orders on orders.o_custkey = customer.u_custkey;
|
count11
|
-1500
|
+0
|
select count(*) as count12 from customer left join orders on orders.o_custkey = customer.u_custkey;
|
count12
|
-1550
|
+150
|
select count(*) as count13 from customer right join orders on orders.o_custkey = customer.u_custkey;
|
count13
|
1500
|
@@ -69,7 +69,7 @@
|
1500
|
select count(*) as count14 from customer where exists (select u_custkey from orders where orders.u_custkey = customer.c_custkey);
|
count14
|
-100
|
+0
|
select count(*) as count15 from customer where not exists (select u_custkey from orders where orders.u_custkey = customer.c_custkey);
|
count15
|
50
|
|
mysqltest: Result length mismatch
|
...
|
After that, results may start to differ (in case below, it was fourth run):
TEST RESULT TIME (ms) or COMMENT
|
--------------------------------------------------------------------------
|
|
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
|
worker[1] mysql-test-run: WARNING: running this script as _root_ will cause some tests to be skipped
|
../storage/columnstore/columnstore/mtr/basic.unsigned_joins [ fail ]
|
Test ended at 2021-03-22 10:16:17
|
|
--- ../storage/columnstore/columnstore/mtr/basic/r/unsigned_joins.result 2021-03-22 08:46:28.134564539 +0000
|
+++ ../storage/columnstore/columnstore/mtr/basic/r/unsigned_joins.reject 2021-03-22 10:16:17.657570964 +0000
|
@@ -69,7 +69,7 @@
|
1500
|
select count(*) as count14 from customer where exists (select u_custkey from orders where orders.u_custkey = customer.c_custkey);
|
count14
|
-100
|
+0
|
select count(*) as count15 from customer where not exists (select u_custkey from orders where orders.u_custkey = customer.c_custkey);
|
count15
|
50
|
|
mysqltest: Result length mismatch
|
Sometimes results displayed are same as first result above.
|
|
Output of my 1.test:
create table customer (
|
c_custkey int,
|
c_name varchar (25),
|
c_address varchar (40),
|
c_nationkey int,
|
c_phone char (15),
|
c_acctbal decimal(12,2),
|
c_mktsegment char (10),
|
c_comment varchar (117)
|
) engine=columnstore;
|
create table orders (
|
o_orderkey int,
|
o_custkey int,
|
o_orderstatus char (1),
|
o_totalprice decimal(12,2),
|
o_orderdate date,
|
o_orderpriority char (15),
|
o_clerk char (15),
|
o_shippriority int,
|
o_comment varchar (79)
|
) engine=columnstore;
|
LOAD DATA LOCAL infile './suite/columnstore/std_data/1m_customer.tbl' INTO TABLE customer FIELDS TERMINATED BY '|';
|
SELECT * FROM information_schema.columnstore_extents;
|
OBJECT_ID OBJECT_TYPE LOGICAL_BLOCK_START LOGICAL_BLOCK_END MIN_VALUE MAX_VALUE WIDTH DBROOT PARTITION_ID SEGMENT_ID BLOCK_OFFSET MAX_BLOCKS HIGH_WATER_MARK STATE STATUS DATA_SIZE
|
3001 Column 234496 238591 NULL 139751256492248 4 1 0 0 0 4096 1 Valid Available 16384
|
3002 Column 238592 246783 NULL NULL 8 1 0 0 0 8192 2 Valid Available 24576
|
3003 Column 254976 263167 NULL NULL 8 1 0 0 0 8192 2 Valid Available 24576
|
3004 Column 271360 275455 NULL 139751256492248 4 1 0 0 0 4096 1 Valid Available 16384
|
3005 Column 275456 283647 NULL NULL 8 1 0 0 0 8192 2 Valid Available 24576
|
3006 Column 291840 300031 NULL 139751256492248 8 1 0 0 0 8192 2 Valid Available 24576
|
3007 Column 300032 308223 NULL NULL 8 1 0 0 0 8192 2 Valid Available 24576
|
3008 Column 316416 324607 NULL NULL 8 1 0 0 0 8192 2 Valid Available 24576
|
3009 Dictionary 246784 254975 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
3010 Dictionary 263168 271359 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
3011 Dictionary 283648 291839 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
3012 Dictionary 308224 316415 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
3013 Dictionary 324608 332799 NULL NULL 8192 1 0 0 0 8192 1 Invalid Available 16384
|
3015 Column 332800 336895 NULL NULL 4 1 0 0 0 4096 0 Valid Available 0
|
3016 Column 336896 340991 NULL NULL 4 1 0 0 0 4096 0 Valid Available 0
|
3017 Column 340992 342015 NULL NULL 1 1 0 0 0 1024 0 Valid Available 0
|
3018 Column 342016 350207 NULL NULL 8 1 0 0 0 8192 0 Valid Available 0
|
3019 Column 350208 354303 NULL NULL 4 1 0 0 0 4096 0 Valid Available 0
|
3020 Column 354304 362495 NULL NULL 8 1 0 0 0 8192 0 Valid Available 0
|
3021 Column 370688 378879 NULL NULL 8 1 0 0 0 8192 0 Valid Available 0
|
3022 Column 387072 391167 NULL NULL 4 1 0 0 0 4096 0 Valid Available 0
|
3023 Column 391168 399359 NULL NULL 8 1 0 0 0 8192 0 Valid Available 0
|
3024 Dictionary 362496 370687 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
3025 Dictionary 378880 387071 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
3026 Dictionary 399360 407551 NULL NULL 8192 1 0 0 0 8192 0 Invalid Available 0
|
What is interesting there is the first row of query from columnstore_extents virtual table.
As you can see, the extent is valid, its minimum value is NULL (which can be NULL or maximum integer value, BTW) and maximum value is 139751256492248. The hex representation of that value is 0x00007f1a600008d8 - clearly a pointer.
It looks like we have some buffer overrun or use of freed memory.
|
|
I test with MariaDB 10.6 and columnstore is minimal change over current develop.
|
|
My current findings.
In short, I observe incorrect width in the data passed to
int SlaveDBRMNode::bulkSetHWMAndCP(const vector<BulkSetHWMArg>& hwmArgs,
|
const std::vector<CPInfo>& setCPDataArgs,
|
const std::vector<CPInfoMerge>& mergeCPDataArgs,
|
VER_t transID, bool firstNode) throw()
|
The argument of interest if mergeCPDataArgs.
CPImport utility calculates ranges for columns it wrote and pass it through some textual representation, one extent information per line. The parsing process of that representation reads lines and parses them into vector of CPInfoMerge structures, it is in bool WEBrmUpdater::prepareCasualPartitionInfo() function.
Please note that at the moment the CPInfoMerge parsing process does not fill up colWidth field and also does not parse wide integer bounds.
Then that vector goes into socket as a chunk of memory (through (de)serializeInlineVector functions).
On a slave DBRM node it then used to fill up a map of int64_t->CPMaxMinMerge. That fill up process copies some parts of CPInfoMerge structure and max and min range bounds are copied conditionally - when colWidth of CPInfoMerge is less or equal to 8, int64_t ranges are copied, otherwise int128_t ranges are copied.
The logs, after deserialization and right at extentmap merge:
|0| I 01 CAL0000: deserialized merges:
|
|0| I 01 CAL0000: lbid 234496, min 1, max 150, width 1543496016
|
|0| I 01 CAL0000: lbid 271360, min 0, max 24, width 1543496016
|
|0| I 01 CAL0000: lbid 291840, min -98696, max 998338, width 1543496016
|
|0| D 30 CAL0000: ExtentMap::mergeExtentsMaxMin()
|
|0| D 30 CAL0000: FirstLBID=271360 min=140668734016898 max=140737014133536 seq=-1 typ: 6 new: 0
|
|0| D 30 CAL0000: FirstLBID=234496 min=140668734016898 max=140737014133536 seq=-1 typ: 6 new: 0
|
|0| D 30 CAL0000: FirstLBID=291840 min=140668734016898 max=140737014133536 seq=-1 typ: 4 new: 0
|
As you can see, column widths are huge and int64_t ranges are off charts.
Then I made a change in parsing process and got logs like this:
|0| I 01 CAL0000: deserialized merges:
|
|0| I 01 CAL0000: lbid 234496, min 1, max 150, width -1726623408
|
|0| I 01 CAL0000: lbid 271360, min 0, max 24, width -1726623408
|
|0| I 01 CAL0000: lbid 291840, min -98696, max 998338, width -1726623408
|
|0| D 30 CAL0000: ExtentMap::mergeExtentsMaxMin()
|
|0| D 30 CAL0000: FirstLBID=271360 min=0 max=24 seq=-1 typ: 6 new: 0
|
|0| D 30 CAL0000: FirstLBID=234496 min=1 max=150 seq=-1 typ: 6 new: 0
|
|0| D 30 CAL0000: FirstLBID=291840 min=-98696 max=998338 seq=-1 typ: 4 new: 0
|
The columns widths again are not correct but now they are less than 8 and int64_t ranges are copied properly.
This change that does not solve problem at the core provided a "solution" - test now "passes" at my work machine.
I will work with drrtuy on a proper solution.
|
|
The solution we ageed upon is to add column width to BRM report file format. Column width is available at cpimport and thus can be reported back.
|
|
I am about to test the solution.
|
|
I see correct column width in all logs and, as a consequence, ranges are correct and test passes.
|
|
https://github.com/mariadb-corporation/mariadb-columnstore-engine/pull/1822 - relevant PR.
|
|
PR is closed but there are still outstanding issue here: should we add support for wide decimals here?
Right now it appears that some work from MCOL-641 has to be done for unsigned_wide_decimal_joins.test to work and right now it is not quite possible to have wide decimals test working.
|
|
I probably messed a bit with process.
|
How to observe presence or absence of defect
The manifestation of a defect can be most clearly seen in the columnstore ranges after use of cpimport. The ranges may be incorrect after "SELECT min_value, max_value FROM information_schema.columnstore_extents". An example of that is in this comment.
As we are dealing with the use of uninitialized data, ranges most probable will not correspond to actual data. Instead of 1..150, the range may be 193336678..12876 - first, some randomly looking values in both bounds and, second, minimum value is bigger than maximum (an empty value set range)
Risks
cpimport marks ranges of extents it added as valid after successful run. The values, though, may be incorrect in the presence of defect.
| Condition |
Effect on queries |
| Range denotes empty value set |
Extent is eliminated on every query; results in omitted data, as if this part of data does not exists |
| Range denotes non empty set of values outside of actual data |
Extent may not be eliminated in the case of <= or >= filter condition with constant value inside the range; most probbly the result will be as with empty set range |
| Range denotes non-empty set partially intersecting actual values range |
some filters may work, some will not; the end result as if part of data is not present in table |
| Range denotes non-empty set that completely includes set of values of actual data |
this will result in excessive I/O |
I cannot state probability of all cases - these will depend on actual data. The probability of some random range denoting empty set is 0.5, so that situation should be observed most often.
For range to be random, the width column should be negative. It is also roughly half of values.
|