[MDEV-11371] Big column compressed Created: 2016-11-29  Updated: 2021-08-17  Due: 2016-12-22  Resolved: 2017-08-31

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Fix Version/s: 10.3.2

Type: Task Priority: Major
Reporter: Sergey Vojtovich Assignee: Sergey Vojtovich
Resolution: Fixed Votes: 0
Labels: contribution, foundation

Issue Links:
Blocks
is blocked by MDEV-13540 Server crashes in copy or Assertion `... Closed
is blocked by MDEV-13541 Server crashes in next_breadth_first_... Closed
Problem/Incident
causes MDEV-13857 Use the 10.2 libmariadb in 10.3 Closed
causes MDEV-24797 Column Compression - ERROR 1265 (0100... Closed
Relates
relates to MDEV-13378 Mysqld crashes with COMPRESSED data t... Closed
relates to MDEV-11381 AliSQL: [Feature] Issue#30 SUPPORT BI... Closed
relates to MDEV-13359 Enable online ALTER TABLE for compres... Closed
relates to MDEV-13539 Latest revision of bb-10.3-svoj (big ... Closed
relates to MDEV-13795 ALTER TABLE…DROP PRIMARY KEY, ADD PRI... Closed
Sub-Tasks:
Key
Summary
Type
Status
Assignee
MDEV-13342 Testing for MDEV-11371 (Big column co... Technical task Closed Alice Sherepa  
MDEV-13760 Document column compression introduce... Technical task Closed Ian Gilfillan  
Sprint: 10.3.1-2

 Description   

Storage engine independent support for column compression.

TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB, TINYTEXT, TEXT, MEDIUMTEXT, LONGTEXT,
VARCHAR and VARBINARY columns can be compressed.

New COMPRESSED column attribute added:
COMPRESSED[=<compression_method>]

The only supported method currently is zlib. It is not possible to create index over compressed column.
CSV storage engine stores compressed field data uncompressed on disk.
Binary log stores compressed field data compressed on disk.

System variables added:
column_compression_threshold - Minimum column data length eligible for compression.
column_compression_zlib_level - zlib compression level (1 gives best speed, 9 gives best compression).
column_compression_zlib_strategy - The strategy parameter is used to tune the compression algorithm. Use the value DEFAULT_STRATEGY for normal data, FILTERED for data produced by a filter (or predictor), HUFFMAN_ONLY to force Huffman encoding only (no string match), or RLE to limit match distances to one (run-length encoding). Filtered data consists mostly of small values with a somewhat random distribution. In this case, the compression algorithm is tuned to compress them better. The effect of FILTERED is to force more Huffman coding and less string matching; it is somewhat intermediate between DEFAULT_STRATEGY and HUFFMAN_ONLY. RLE is designed to be almost as fast as HUFFMAN_ONLY, but give better compression for PNG image data. The strategy parameter only affects the compression ratio but not the correctness of the compressed output even if it is not set appropriately. FIXED prevents the use of dynamic Huffman codes, allowing for a simpler decoder for special applications.
column_compression_zlib_wrap - Generate zlib header and trailer and compute adler32 check value. It can be used with storage engines that don't provide data integrity verification to detect data corruption.

Status variables added:
Column_compressions - incremented every time field data is compressed.
Column_decompressions - incremented every time field data is decompressed.



 Comments   
Comment by Sergey Vojtovich [ 2017-08-31 ]

Pushed https://github.com/MariaDB/server/commit/fdc47792354c820aa4a8542d7c00d434424a63fb

Generated at Thu Feb 08 07:49:26 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.