Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-26029

Sparse files are inefficient on thinly provisioned storage

Details

    Description

      The MariaDB implementation of page_compressed tables for InnoDB uses sparse files. In the worst case, in the data file, every data page will consist of some data followed by a hole. This may be extremely inefficient in some file systems.

      If the underlying storage device is thinly provisioned (can compress data on the fly), it would be good to write regular files (with sequences of NUL bytes at the end of each page_compressed block) and let the storage device take care of compressing the data.

      Attachments

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. MDEV-11068 Review which innodb_compression_algorithm to support in binary packages

          • Major - Major loss of function.
          • Closed
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26042 Atomic write capability is not detected correctly

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-26192 Sparse files are being created on thinly provisioned storage

          • Major - Major loss of function.
          • Closed

          Activity

            marko Marko Mäkelä added a comment -

            The detection of sparse files currently requires read access to the block device. It would be nice to have it exposed via the Linux /sys file system, similar to the SSD detection (MDEV-17380 reads files /sys/block/*/queue/rotational). If read access is not granted, one will see the following kind of a message on startup:

            mariadb-10.6.3

            		 
            Unable to determine if thin provisioning is used: open("/dev/sfdv0n1"): Permission denied

            In such case, thin provisioning will be assumed to be disabled, that is, sparse files will be written for page_compressed tables, as was the case before this improvement.

            marko Marko Mäkelä added a comment - The detection of sparse files currently requires read access to the block device. It would be nice to have it exposed via the Linux /sys file system, similar to the SSD detection ( MDEV-17380 reads files /sys/block/*/queue/rotational ). If read access is not granted, one will see the following kind of a message on startup: mariadb-10.6.3 Unable to determine if thin provisioning is used: open("/dev/sfdv0n1"): Permission denied In such case, thin provisioning will be assumed to be disabled, that is, sparse files will be written for page_compressed tables, as was the case before this improvement.

            People

              marko Marko Mäkelä
              marko Marko Mäkelä
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.