Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-27619

Errors while compiling RocksDB on macOS with Xcode 9.4.1 (zstd headers related)

    XMLWordPrintable

Details

    Description

      I've got the following compilation error while building current 10.7 code from GitHub:

      ...
      [ 75%] Building CXX object storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/db/arena_wrapped_db_iter.cc.o
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/arena_wrapped_db_iter.cc:10:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/arena_wrapped_db_iter.h:13:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/db_impl/db_impl.h:22:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/column_family.h:17:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/memtable_list.h:17:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/memtable.h:19:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/range_tombstone_fragmenter.h:15:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/pinned_iterators_manager.h:12:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/table/internal_iterator.h:14:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/table/format.h:22:
      In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/options/cf_options.h:14:
      /Users/Valerii/git/server/storage/rocksdb/rocksdb/util/compression.h:48:10: fatal error:
            'zstd.h' file not found
      #include <zstd.h>
               ^~~~~~~~
      1 error generated.
      make[2]: *** [storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/db/arena_wrapped_db_iter.cc.o] Error 1
      make[1]: *** [storage/rocksdb/CMakeFiles/rocksdblib.dir/all] Error 2
      make: *** [all] Error 2
      

      It was configured like this:

      cmake .. -DCMAKE_INSTALL_PREFIX=/Users/Valerii/dbs/maria10.7 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DWITH_EMBEDDED_SERVER=OFF -DPLUGIN_TOKUDB=NO -DWITH_SSL=/opt/local/libexec/openssl11 -DENABLE_DTRACE=1
      

      Looks like cmake found zstd where it really is located, at least I see these:

      Yuliyas-Air:rocksdb Valerii$ grep -ni zstd ~/git/server/buildtmp/CMakeCache.txt
      1126://Build RocksDB  with ZSTD compression. Possible values are 'ON',
      1128:WITH_ROCKSDB_ZSTD:STRING=AUTO
      1187:ZSTD_INCLUDE_DIRS:PATH=/opt/local/include
      1190:ZSTD_LIBRARIES:FILEPATH=/opt/local/lib/libzstd.dylib
      1415:rocksdblib_LIB_DEPENDS:STATIC=general;/opt/local/lib/libzstd.dylib;general;zlib;
      2880:PC_CURL_STATIC_LDFLAGS:INTERNAL=-L/opt/local/lib;-lcurl;-lidn2;-lpsl;-lssl;-lcrypto;-lssl;-lcrypto;-lzstd;-lz
      2883:PC_CURL_STATIC_LIBRARIES:INTERNAL=curl;idn2;psl;ssl;crypto;ssl;crypto;zstd;z
      3132://ADVANCED property for variable: ZSTD_INCLUDE_DIRS
      3133:ZSTD_INCLUDE_DIRS-ADVANCED:INTERNAL=1
      3134://ADVANCED property for variable: ZSTD_LIBRARIES
      3135:ZSTD_LIBRARIES-ADVANCED:INTERNAL=1
      3137:ZSTD_VALID:INTERNAL=1
      3214:__msg1_rocksdb_ZSTD:INTERNAL=5a3b48f2d55f361582620bdf0ab2061e
      3456:prefix_result:INTERNAL=AsynchDNS;HSTS;HTTPS-proxy;IDN;IPv6;Largefile;NTLM;NTLM_WB;PSL;SSL;TLS-SRP;UnixSockets;alt-svc;libz;zstd
      Yuliyas-Air:rocksdb Valerii$
      

      We have these:

      Yuliyas-Air:rocksdb Valerii$ ls -l /opt/local/include/z*.h
      -rw-r--r--  1 root  admin   16262 Sep 27  2017 /opt/local/include/zconf.h
      -rw-r--r--  1 root  wheel   25631 Dec 22 13:26 /opt/local/include/zdict.h
      -rw-r--r--  1 root  admin   96239 Sep 27  2017 /opt/local/include/zlib.h
      -rw-r--r--  1 root  wheel  148639 Dec 22 13:26 /opt/local/include/zstd.h
      -rw-r--r--  1 root  wheel    3817 Dec 22 13:26 /opt/local/include/zstd_errors.h
      

      The following very lame patch allowed to proceed and end up with 10.7 built and running normally:

      Yuliyas-Air:server Valerii$ cd storage/rocksdb/rocksdb
      Yuliyas-Air:rocksdb Valerii$ git diff
      diff --git a/util/compression.h b/util/compression.h
      index 8169841ba..3b25bda2a 100644
      --- a/util/compression.h
      +++ b/util/compression.h
      @@ -45,9 +45,11 @@
       #endif
       
       #if defined(ZSTD)
      -#include <zstd.h>
      +#include "/opt/local/include/zstd.h"
      +// #include <zstd.h>
       #if ZSTD_VERSION_NUMBER >= 10103  // v1.1.3+
      -#include <zdict.h>
      +#include  "/opt/local/include/zdict.h"
      +//#include <zdict.h>
       #endif  // ZSTD_VERSION_NUMBER >= 10103
       namespace ROCKSDB_NAMESPACE {
       // Need this for the context allocation override
      

      Attachments

        Activity

          People

            shulga Dmitry Shulga
            valerii Valerii Kravchuk
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.