[MDEV-26029] Sparse files are inefficient on thinly provisioned storage Created: 2021-06-28  Updated: 2021-07-20  Resolved: 2021-06-29

Status: Closed
Project: MariaDB Server
Component/s: Storage Engine - InnoDB
Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6
Fix Version/s: 10.6.3

Type: Bug Priority: Major
Reporter: Marko Mäkelä Assignee: Marko Mäkelä
Resolution: Fixed Votes: 0
Labels: performance

Issue Links:
Blocks
blocks MDEV-11068 Review which innodb_compression_algor... Closed
is blocked by MDEV-26042 Atomic write capability is not detect... Closed
Relates
relates to MDEV-26192 Sparse files are being created on thi... Closed

 Description   

The MariaDB implementation of page_compressed tables for InnoDB uses sparse files. In the worst case, in the data file, every data page will consist of some data followed by a hole. This may be extremely inefficient in some file systems.

If the underlying storage device is thinly provisioned (can compress data on the fly), it would be good to write regular files (with sequences of NUL bytes at the end of each page_compressed block) and let the storage device take care of compressing the data.

 Comments   
Comment by Marko Mäkelä [ 2021-07-20 ]

The detection of sparse files currently requires read access to the block device. It would be nice to have it exposed via the Linux /sys file system, similar to the SSD detection (MDEV-17380 reads files /sys/block/*/queue/rotational). If read access is not granted, one will see the following kind of a message on startup:

mariadb-10.6.3

 
Unable to determine if thin provisioning is used: open("/dev/sfdv0n1"): Permission denied

In such case, thin provisioning will be assumed to be disabled, that is, sparse files will be written for page_compressed tables, as was the case before this improvement.

Generated at Thu Feb 08 09:42:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.