[MDEV-23497] make ROW_FORMAT=COMPRESSED read-only by default Created: 2020-08-17  Updated: 2022-02-14  Resolved: 2020-11-11

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Fix Version/s: 10.6.0

Type: Task Priority: Major
Reporter: Sergei Golubchik Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
causes MDEV-24503 Assertion `m_prebuilt->trx == thd_to_... Closed
causes MDEV-25100 Read-only table with ROW_FORMAT=COMPR... Closed
causes MDEV-27736 Allow seamless upgrade despite ROW_FO... Closed
Relates
relates to MDEV-25099 Inconsistent error upon updating read... Open
relates to MDEV-12152 KEY_BLOCK_SIZE strangeness in ALTER T... Open
relates to MDEV-22367 Remove write support for ROW_FORMAT=C... Closed
relates to MDEV-26400 ALTER TABLE does not remove KEY_BLOCK... Stalled

 Description   

introduce a new innodb command line option that will make compressed tables writable. by default they should be read-only. if the write support is enabled, it should print a warning on startup.

This will allow users to prepare for MDEV-22367



 Comments   
Comment by Marko Mäkelä [ 2020-11-10 ]

I think that it was easier to complete this than to provide an estimate. Most of the time was spent on adding SET GLOBAL innodb_read_only_compressed=OFF around tests that cover the ROW_FORMAT=COMPRESSED format. We must keep those tests until we remove the write support.

Comment by Marko Mäkelä [ 2020-11-11 ]

To allow ROW_FORMAT=COMPRESSED tables to be created or written, we introduce an option:

SET GLOBAL innodb_read_only_compressed=OFF;

That parameter may be deprecated and ignored in the next major version (along with the code to support writing ROW_FORMAT=COMPRESSED tables in InnoDB.

Comment by Nuno [ 2021-01-01 ]

@Marko - will we at least be provided with a way to easily convert ONLINE these tables to the alternative? (PAGE_COMPRESSED=1)

I have large compressed tables on my 24-hour online write-intensive website.

This change seems to mean to me that I won't be able to upgrade my MariaDB version to 10.7+, once the COMPRESSED feature is completely removed...

Also, as discussed in MDEV-22839, PAGE_COMPRESSED doesn't really seem to work well for me. It even increased the non-sparse size of the data & indexes, when I tested...

I'm really not happy with this change.

> "Nowadays, with fast solid-state storage being a commodity,"

I would be careful in saying this.
Not everyone can afford AWS or similar (may save in storage, but doesn't save in data transfer out),
and many are still on Dedicated bare metal servers, where there's no choice to simply "increase" storage.

Comment by Olaf Buitelaar [ 2021-02-14 ]

@Nuno I agree with you. I've also TB's of table's using ROW_FORMAT=COMPRESSED and run on-prem. i'm a bit concerned indeed this feature will be dropped.

Comment by Marko Mäkelä [ 2021-03-10 ]

I implemented online table rebuild in MySQL 5.6.8, and the feature is present in MariaDB 10.0 already. You should be able to just execute something like the following:

ALTER TABLE compressed_table ROW_FORMAT=DYNAMIC page_compressed=1;

The table will be rebuilt, while even allowing concurrent writes. And concurrent writes will be allowed while the table is being rebuilt, as long as

SET GLOBAL innodb_read_only_compressed=OFF;

is in effect.

Comment by Olaf Buitelaar [ 2021-03-10 ]

Thanks for the suggestion, and i'm sure it would work. But the tables are extremely large 15+TB (however partitioned), this would mean we would require that space double during the rebuild. Also the tables are quite write intensive, so IO would double during the rebuild time. Last in the past i had some page corruption errors, which i was not able to recover. Probably rebuilding the table would let those errors surface again. Also during testing it seemed the compression ratio with

ROW_FORMAT=COMPRESSED

vs

ROW_FORMAT=DYNAMIC page_compressed=1

was much better
Altogether it seems it's going to be quite a difficult upgrade.
Having said that, i understand your motivation to simplify things in InnoDB. And i'm glad you do so.

Comment by Marko Mäkelä [ 2021-03-18 ]

olafbuitelaar, thank you for your feedback. ALTER TABLE should only read the clustered index, not secondary indexes. In fact, rebuilding a table would fix corrupted secondary indexes (I hope that MDEV-22373 is fixed now, but it is too early to tell).

I hope that page_compressed together with compressed columns (MDEV-11371) would make an acceptable replacement of ROW_FORMAT=COMPRESSED. One piece that is missing is something that allows us to avoid the problematic use of sparse files in page_compressed. I think that we must provide an option to write page_compressed tables as normal non-sparse files and let the a smart SSD compress away the sequences of zero bytes, as explained in https://mariadb.org/fest2020/ssd/.

Comment by Rick James [ 2021-08-02 ]

Are there any creditable benchmarks on the CPU, I/O, and disk space for ROW_FORMAT=COMPRESSED? I have long been skeptical about the feature. I will continue to steer people away from using that setting.

Comment by Marko Mäkelä [ 2021-08-30 ]

Based on https://github.com/nextcloud/server/issues/25436 NextCloud appears to be using ROW_FORMAT=COMPRESSED for InnoDB tables.

Comment by Marko Mäkelä [ 2021-08-30 ]

rjasdfiii, I remember that back in 2006 or 2007, when I had completed the implementation of ROW_FORMAT=COMPRESSED for the InnoDB Plugin for MySQL 5.0 based on some rough ideas of Heikki Tuuri, some performance tests were run, and the results were not great. I think that we really have to compare all performance aspects to page_compressed, as suggested in MDEV-22839.

Comment by Marko Mäkelä [ 2022-02-14 ]

MDEV-27736 in MariaDB Server 10.6.6 essentially reverted this change, by changing the default value of innodb_read_only_compressed to OFF. The plan to remove support for InnoDB ROW_FORMAT=COMPRESSED tables (MDEV-22367) has been cancelled.

Generated at Thu Feb 08 09:22:51 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.