[MCOL-1016] information_schema.columnstore_extents data_size calculation incorrect Created: 2017-11-07  Updated: 2017-11-08  Resolved: 2017-11-08

Status: Closed
Project: MariaDB ColumnStore
Component/s: MDB Plugin
Affects Version/s: 1.0.11, 1.1.1
Fix Version/s: 1.0.12, 1.1.2

Type: Bug Priority: Major
Reporter: Andrew Hutchings (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2017-22

 Description   

The data_size column is calculated using (HWM+1-BLOCK_OFFSET)*BLOCK_SIZE. But for multi-segment files the HWM for the lower segments is 0. This means that the lower segment shows a size of 1 block and the last segment shows a size of one extent segment.

This should probably show 0 for the lower segments and the entire usage for the upper segment if possible.



 Comments   
Comment by Andrew Hutchings (Inactive) [ 2017-11-07 ]

Pull requests for 1.0 and 1.1

For QA:

information_schema.columnstore_extents will now show data_size as 0 if HWM is 0 and the last segment in an extent file will contain the data_size (it will have a BLOCK_OFFSET > 0). In addition the compression_ratio calculation has been fixed.

For a 10 column 80M row insert it has gone from calculating the compression ratio from 181% to 70% which is much more accurate (below 100% is compressed, above means the data files are bigger than the data). Also the total_data_size has gone from 2.25GB to 3.87GB.

Comment by Daniel Lee (Inactive) [ 2017-11-08 ]

Builds verified: 1.0.12 and 1.1.2 Github source

1.0.12

/root/columnstore/mariadb-columnstore-server
commit a42eb6d1e74e44c9e8fd9bb8290e6ce7dbf909f5
Merge: 2965fc8 6a14ced
Author: David.Hall <david.hall@mariadb.com>
Date: Tue Oct 3 10:12:33 2017 -0500

Merge pull request #69 from mariadb-corporation/MCOL-940

MCOL-940

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 024c96afedb59aca0b45e3adc996d55af799b0a2
Merge: 28d26c8 929d0d3
Author: David.Hall <david.hall@mariadb.com>
Date: Tue Nov 7 14:13:26 2017 -0600

Merge pull request #310 from mariadb-corporation/MCOL-1016

MCOL-1016 Fix I_S calculations

1.1.2

/root/columnstore/mariadb-columnstore-server
commit ed21e674cfd70db421957d0212ff7fc1835d06d5
Author: david hill <david.hill@mariadb.com>
Date: Mon Nov 6 08:41:53 2017 -0600

Update README.md

updated version

/root/columnstore/mariadb-columnstore-server/mariadb-columnstore-engine
commit 0db37ad7df6567e05971a1222c9ef7423742869f
Author: david hill <david.hill@mariadb.com>
Date: Tue Nov 7 18:19:36 2017 -0600

added new script to cmake

I noticed the case where there is only one block of data for the column (HWM=0), the data size will also show 0. Confirmed with developer that this is the expected behavior.

Also discovered another issue in 1.1.12 (MCOL-1021)

Generated at Thu Feb 08 02:25:33 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.