[MCOL-1021] Dictionary deduplication cache is not working correctly Created: 2017-11-08  Updated: 2017-11-10  Resolved: 2017-11-10

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: 1.1.2
Fix Version/s: 1.1.2

Type: Bug Priority: Critical
Reporter: Andrew Hutchings (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2017-22

 Description   

The dictionary de-duplication doesn't appear to be working. As using a test by dleeyh

 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-11-09 ]

Dictionary signatures were getting destroyed before being used for the de-duplication cache insertion. Effectively making the cache useless. This fix restores the signature pointer/length before it is inserted into the cache.

It also fixes a case inherited from InfiniDB where if the cache is full then it is no long read and de-duplication no longer happens for current entries.

For QA: cpimport of lineitem should now see l_shipinstruct and l_shipmode only use 1 block insert of more than one extent.

Comment by Daniel Lee (Inactive) [ 2017-11-10 ]

Build verified: 1.1.2 GitHub source

/root/columnstore/mariadb-columnstore-server
commit ed21e674cfd70db421957d0212ff7fc1835d06d5
Author: david hill <david.hill@mariadb.com>
Date: Mon Nov 6 08:41:53 2017 -0600

Update README.md

updated version

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit cf7eecf8bb52604d23792cb1ac522b9ae50169f5
Merge: 8822856 21ca6ca
Author: David.Hall <david.hall@mariadb.com>
Date: Thu Nov 9 18:00:02 2017 -0600

Merge pull request #314 from mariadb-corporation/MCOL-1021

MCOL-1021 Fix dictionary de-duplication cache

Loaded 10g dbt3 database and verified l_shipstruct and l_shipmode dictionary OIDs.

MariaDB [information_schema]> select * from columnstore_extents where object_id in (3109, 3110);
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

OBJECT_ID OBJECT_TYPE LOGICAL_BLOCK_START LOGICAL_BLOCK_END MIN_VALUE MAX_VALUE WIDTH DBROOT PARTITION_ID SEGMENT_ID BLOCK_OFFSET MAX_BLOCKS HIGH_WATER_MARK STATE STATUS DATA_SIZE

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

3109 Dictionary 877568 885759 NULL NULL 127 1 0 0 0 8192 0 Invalid Available 0
3109 Dictionary 1363968 1372159 NULL NULL 127 1 0 1 0 8192 0 Invalid Available 0
3109 Dictionary 1505280 1513471 NULL NULL 127 1 0 2 0 8192 0 Invalid Available 0
3109 Dictionary 1654784 1662975 NULL NULL 127 1 0 3 0 8192 0 Invalid Available 0
3110 Dictionary 893952 902143 NULL NULL 127 1 0 0 0 8192 0 Invalid Available 0
3110 Dictionary 1355776 1363967 NULL NULL 127 1 0 1 0 8192 0 Invalid Available 0
3110 Dictionary 1497088 1505279 NULL NULL 127 1 0 2 0 8192 0 Invalid Available 0
3110 Dictionary 1638400 1646591 NULL NULL 127 1 0 3 0 8192 0 Invalid Available 0

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
8 rows in set (0.01 sec)

Generated at Thu Feb 08 02:25:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.