[MDEV-27619] Errors while compiling RocksDB on macOS with Xcode 9.4.1 (zstd headers related) Created: 2022-01-25  Updated: 2023-11-28

Status: Open
Project: MariaDB Server
Component/s: Compiling, Storage Engine - RocksDB
Affects Version/s: 10.7.2, 10.11.3
Fix Version/s: 10.11, 11.0

Type: Bug Priority: Major
Reporter: Valerii Kravchuk Assignee: Dmitry Shulga
Resolution: Unresolved Votes: 0
Labels: macos
Environment:

macOS 10.13.6



 Description   

I've got the following compilation error while building current 10.7 code from GitHub:

...
[ 75%] Building CXX object storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/db/arena_wrapped_db_iter.cc.o
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/arena_wrapped_db_iter.cc:10:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/arena_wrapped_db_iter.h:13:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/db_impl/db_impl.h:22:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/column_family.h:17:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/memtable_list.h:17:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/memtable.h:19:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/range_tombstone_fragmenter.h:15:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/db/pinned_iterators_manager.h:12:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/table/internal_iterator.h:14:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/table/format.h:22:
In file included from /Users/Valerii/git/server/storage/rocksdb/rocksdb/options/cf_options.h:14:
/Users/Valerii/git/server/storage/rocksdb/rocksdb/util/compression.h:48:10: fatal error:
      'zstd.h' file not found
#include <zstd.h>
         ^~~~~~~~
1 error generated.
make[2]: *** [storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/db/arena_wrapped_db_iter.cc.o] Error 1
make[1]: *** [storage/rocksdb/CMakeFiles/rocksdblib.dir/all] Error 2
make: *** [all] Error 2

It was configured like this:

cmake .. -DCMAKE_INSTALL_PREFIX=/Users/Valerii/dbs/maria10.7 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DWITH_EMBEDDED_SERVER=OFF -DPLUGIN_TOKUDB=NO -DWITH_SSL=/opt/local/libexec/openssl11 -DENABLE_DTRACE=1

Looks like cmake found zstd where it really is located, at least I see these:

Yuliyas-Air:rocksdb Valerii$ grep -ni zstd ~/git/server/buildtmp/CMakeCache.txt
1126://Build RocksDB  with ZSTD compression. Possible values are 'ON',
1128:WITH_ROCKSDB_ZSTD:STRING=AUTO
1187:ZSTD_INCLUDE_DIRS:PATH=/opt/local/include
1190:ZSTD_LIBRARIES:FILEPATH=/opt/local/lib/libzstd.dylib
1415:rocksdblib_LIB_DEPENDS:STATIC=general;/opt/local/lib/libzstd.dylib;general;zlib;
2880:PC_CURL_STATIC_LDFLAGS:INTERNAL=-L/opt/local/lib;-lcurl;-lidn2;-lpsl;-lssl;-lcrypto;-lssl;-lcrypto;-lzstd;-lz
2883:PC_CURL_STATIC_LIBRARIES:INTERNAL=curl;idn2;psl;ssl;crypto;ssl;crypto;zstd;z
3132://ADVANCED property for variable: ZSTD_INCLUDE_DIRS
3133:ZSTD_INCLUDE_DIRS-ADVANCED:INTERNAL=1
3134://ADVANCED property for variable: ZSTD_LIBRARIES
3135:ZSTD_LIBRARIES-ADVANCED:INTERNAL=1
3137:ZSTD_VALID:INTERNAL=1
3214:__msg1_rocksdb_ZSTD:INTERNAL=5a3b48f2d55f361582620bdf0ab2061e
3456:prefix_result:INTERNAL=AsynchDNS;HSTS;HTTPS-proxy;IDN;IPv6;Largefile;NTLM;NTLM_WB;PSL;SSL;TLS-SRP;UnixSockets;alt-svc;libz;zstd
Yuliyas-Air:rocksdb Valerii$

We have these:

Yuliyas-Air:rocksdb Valerii$ ls -l /opt/local/include/z*.h
-rw-r--r--  1 root  admin   16262 Sep 27  2017 /opt/local/include/zconf.h
-rw-r--r--  1 root  wheel   25631 Dec 22 13:26 /opt/local/include/zdict.h
-rw-r--r--  1 root  admin   96239 Sep 27  2017 /opt/local/include/zlib.h
-rw-r--r--  1 root  wheel  148639 Dec 22 13:26 /opt/local/include/zstd.h
-rw-r--r--  1 root  wheel    3817 Dec 22 13:26 /opt/local/include/zstd_errors.h

The following very lame patch allowed to proceed and end up with 10.7 built and running normally:

Yuliyas-Air:server Valerii$ cd storage/rocksdb/rocksdb
Yuliyas-Air:rocksdb Valerii$ git diff
diff --git a/util/compression.h b/util/compression.h
index 8169841ba..3b25bda2a 100644
--- a/util/compression.h
+++ b/util/compression.h
@@ -45,9 +45,11 @@
 #endif
 
 #if defined(ZSTD)
-#include <zstd.h>
+#include "/opt/local/include/zstd.h"
+// #include <zstd.h>
 #if ZSTD_VERSION_NUMBER >= 10103  // v1.1.3+
-#include <zdict.h>
+#include  "/opt/local/include/zdict.h"
+//#include <zdict.h>
 #endif  // ZSTD_VERSION_NUMBER >= 10103
 namespace ROCKSDB_NAMESPACE {
 // Need this for the context allocation override


Generated at Thu Feb 08 09:54:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.