Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15578

MyRocks: support zstandard compression where the distro allows it

Details

    Description

      Modern distros include ZStandard compression library.
      This task is to have MyRocks compiled with that compression library, and have it as a dependency.
      The goal is only to support it when the OS has a package for ZStandard.

      Fedora 24 has these packages:

      zstd.x86_64 : Zstd compression library
      libzstd.i686 : Zstd shared library
      libzstd.x86_64 : Zstd shared library
      libzstd-devel.i686 : Header files for Zstd library
      

      Ubuntu Xenial 16.04.3 LTS has

      libzstd-dev - fast lossless compression algorithm -- development files
      libzstd0 - fast lossless compression algorithm
      zstd - fast lossless compression algorithm -- CLI tool
      libzstd1 - fast lossless compression algorithm
      libzstd1-dev - fast lossless compression algorithm -- development files
      

      Debian Stretch has it: https://packages.debian.org/stretch/libzstd1
      CentOS 7 doesn't have it.

      Attachments

        Issue Links

          Activity

            There might be a problem with fixing this in 10.2 as changing package dependencies in a stable version may create problems. In this case, let's compile/link/depend on ZStandard in 10.3.

            psergei Sergei Petrunia added a comment - There might be a problem with fixing this in 10.2 as changing package dependencies in a stable version may create problems. In this case, let's compile/link/depend on ZStandard in 10.3.
            danblack Daniel Black added a comment -

            Centos/RHEL 7 - there is an EPEL package for libzstd-devel: https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/l/

            danblack Daniel Black added a comment - Centos/RHEL 7 - there is an EPEL package for libzstd-devel: https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/l/
            mg MG added a comment -

            Lack of zstd support in MyRocks for MariaDB was called out in a recent blog post and I have also wished to use this library for InnoDB page compression, so I voted on this issue.

            mg MG added a comment - Lack of zstd support in MyRocks for MariaDB was called out in a recent blog post and I have also wished to use this library for InnoDB page compression, so I voted on this issue.

            Please add zstd support!

            chmichael Charalampos Michael added a comment - Please add zstd support!
            mg MG added a comment -

            I see that in the changelog for 10233 and 10324 this was added, and the linked commit mentions also that 10414 and 1055 corrected the makefile problem.

            Quick test seems to show that zstd works in 10.4.14:

            MariaDB [test2]> select   @@version, SP.sst_name, SP.compression_algo from   information_schema.rocksdb_sst_props SP,   information_schema.rocksdb_ddl D,   information_schema.rocksdb_index_file_map IFM where   D.table_schema='test2' and D.table_name='myrockst1' and   D.index_number= IFM.index_number and   IFM.sst_name=SP.sst_name;
            +---------------------+------------+------------------+
            | @@version           | sst_name   | compression_algo |
            +---------------------+------------+------------------+
            | 10.4.14-MariaDB-log | 000096.sst | ZSTD             |
            +---------------------+------------+------------------+
            1 row in set (0.001 sec)
            

            Sadly this fix being for storage/rocksdb/build_rocksdb.cmake and the libzstd dependency coming with MariaDB-rocksdb-engine means that innodb_compression_algorithm was unaffected.

            I suspect that the folks at MariaDB can now close this bug.

            mg MG added a comment - I see that in the changelog for 10233 and 10324 this was added, and the linked commit mentions also that 10414 and 1055 corrected the makefile problem. Quick test seems to show that zstd works in 10.4.14: MariaDB [test2]> select @ @version , SP.sst_name, SP.compression_algo from information_schema.rocksdb_sst_props SP, information_schema.rocksdb_ddl D, information_schema.rocksdb_index_file_map IFM where D.table_schema= 'test2' and D.table_name= 'myrockst1' and D.index_number= IFM.index_number and IFM.sst_name=SP.sst_name; +---------------------+------------+------------------+ | @ @version | sst_name | compression_algo | +---------------------+------------+------------------+ | 10.4 . 14 -MariaDB-log | 000096 .sst | ZSTD | +---------------------+------------+------------------+ 1 row in set ( 0.001 sec) Sadly this fix being for storage/rocksdb/build_rocksdb.cmake and the libzstd dependency coming with MariaDB-rocksdb-engine means that innodb_compression_algorithm was unaffected. I suspect that the folks at MariaDB can now close this bug.
            danblack Daniel Black added a comment -

            Thanks mg for confirming

            MDEV-12933 is there to unify the provision of compression across engines

            danblack Daniel Black added a comment - Thanks mg for confirming MDEV-12933 is there to unify the provision of compression across engines

            People

              cvicentiu Vicențiu Ciorbaru
              psergei Sergei Petrunia
              Votes:
              3 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.