[MCOL-4685] Eliminate some irrelevant settings (uncompressed data and extents per file) Created: 2021-04-20  Updated: 2021-06-16  Resolved: 2021-06-16

Status: Closed
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.1

Type: Task Priority: Minor
Reporter: Gregory Dorman (Inactive) Assignee: Daniel Lee (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MCOL-4566 rebuildEM utility must support compre... Closed
Sprint: 2021-7, 2021-8, 2021-9

 Description   

1. remove the option to declare uncompressed columns (set columnstore_compression_type = 0).
2. ignore [COMMENT '[compression=0] option at table or column level (no error messages, just disregard);
3. remove the option to set more than 2 extents per file (ExtentsPreSegmentFile)



 Comments   
Comment by Denis Khalikov [ 2021-06-07 ]

Changes:

1. Removes the option to declare uncompressed columns (set columnstore_compression_type = 0).
2. Ignores [COMMENT '[compression=0] option at table or column level (no error messages, just disregard).
3. Removes the option to set more than 2 extents per file (ExtentsPreSegmentFile).
4. Updates rebuildEM tool to support up to 10 dictionary extent per dictionary segment file.
5. Adds check for `DBRootStorageType` for rebuildEM tool.
6. Renamed rebuildEM to mcsRebuildEM.

Testing instructions (denis0x0D, please verify and complete):
1. try set columnstore_compression_type = 0. Should return no messages, but set compression to SNAPPY regardless.
2. set compreesion=0 in COMMENT, but check the resulting data, should be compressed still.
3. see what happens if you try to set ExtentsPreSegmentFile to 1 or something bigger than 2. Pay attntion to upgrade in addition to fresh install.
4. denis0x0D - please define the test
5. Try the tool for both disk-resident and S3. See what happens on each.
6. check the name.

Comment by Daniel Lee (Inactive) [ 2021-06-15 ]

Is the test case for in the last comment meant for MCOL-4566?

Comment by Daniel Lee (Inactive) [ 2021-06-15 ]

Build tested: 6.1.1 ( Drone #2586 )

So far, tested columnstore_compression_type.

When setting columnstore_compression_type in MariaDB session or server.cnf, this is the new behavior:

6.1.1
set to 0, "show variables" shows UNUSED, table created using compression 2
set to 1, "show variables" shows SNAPPY, table created using compression 2
set to 2, "show variables" shows SNAPPY, table created using compression 2

Why do we introduced the new value UNUSED?
What does UNUSED mean?

This new value of UNUSED is ambiguous and creates confusion. It should be set to SNAPPY instead when columnstore_compression_type is set to 0.

5.5.2 behavior
set to 0, "show variables" shows NO_COMPRESSION, table created using compression 0
set to 1, "show variables" shows SNAPPY, table created using compression 2
set to 2, "show variables" shows SNAPPY, table created using compression 2

Comment by Daniel Lee (Inactive) [ 2021-06-15 ]

I did not get an error when setting compression to 0

MariaDB [mytest]> set columnstore_compression_type=0;
Query OK, 0 rows affected (0.000 sec)
MariaDB [mytest]> show variables like "%compression%";
+------------------------------------------+------------------+
| Variable_name                            | Value            |
+------------------------------------------+------------------+
| column_compression_threshold             | 100              |
| column_compression_zlib_level            | 6                |
| column_compression_zlib_strategy         | DEFAULT_STRATEGY |
| column_compression_zlib_wrap             | OFF              |
| columnstore_compression_type             | UNUSED           |
| innodb_compression_algorithm             | zlib             |
| innodb_compression_default               | OFF              |
| innodb_compression_failure_threshold_pct | 5                |
| innodb_compression_level                 | 6                |
| innodb_compression_pad_pct_max           | 50               |
+------------------------------------------+------------------+
10 rows in set (0.003 sec)

Comment by Denis Khalikov [ 2021-06-15 ]

dleeyh right, currently no error for compression_type=0, it sets just default compression.
gdorman ok, I'll change it to SNAPPY

Comment by Daniel Lee (Inactive) [ 2021-06-15 ]

Build tested: 6.1.1 ( Drone #2586 )

Verified:
1. When columnstore_compression_type is set to 0, type 2 (SNAPPY) is used to create compressed tables
2. Setting compression to 0 at both table and column level also ended type 2 SNAPPY compression
3. Uncompressed tables created in 5.5.2 are readable in 6.1.1

4. ExtentsPerSegmentFile has been removed from Columnstore.xml file
5. Adding ExtentsPerSegmentFile to Columnstore.xml and setting it to 4 has no effect on data files allocation

Comment by Daniel Lee (Inactive) [ 2021-06-16 ]

Build verified: 6.1.1 ( Drone #2599 )

columnstore_compression_type shows SNAPPY when setting to 0.

Generated at Thu Feb 08 02:52:12 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.