Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
10.3(EOL)
-
None
-
macOS 10.13.4
--
clang -v
Apple LLVM version 9.1.0 (clang-902.0.39.1)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
--
uname -a
Darwin blackbox 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar 5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64macOS 10.13.4 -- clang -v Apple LLVM version 9.1.0 (clang-902.0.39.1) Target: x86_64-apple-darwin17.5.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin -- uname -a Darwin blackbox 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar 5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
Description
This is a master issue used to track all the build issues on latest macOS 10.13.4, which fails in multiple storage engines. Steps for building followed from MariaDB.org
Attaching individual errors bellow:
Inno
/Users/shinnok/Work/mariadb/server/storage/innobase/include/sync0policy.h:53:4: error: cannot initialize a member subobject of type 'ulint'
|
(aka 'unsigned long') with an rvalue of type 'os_thread_id_t' (aka '_opaque_pthread_t *')
|
m_thread_id(os_thread_id_t(ULINT_UNDEFINED))
|
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/innobase/include/sync0policy.h:79:4: error: cannot initialize a parameter of type 'int64' (aka 'long long') with
|
an rvalue of type 'os_thread_id_t' (aka '_opaque_pthread_t *')
|
my_atomic_storelint(&m_thread_id, os_thread_get_curr_id());
|
RocksDB
[ 79%] Building CXX object storage/rocksdb/CMakeFiles/mysql_ldb.dir/tools/mysql_ldb.cc.o
|
[ 79%] Linking CXX executable mysql_ldb
|
Undefined symbols for architecture x86_64:
|
"__db_flush_", referenced from:
|
myrocks::rdb_get_global_perf_counters(myrocks::Rdb_perf_counters*) in librocksdb_aux_lib.a(rdb_perf_context.cc.o)
|
Was missing lib to dbug lib.
TokuDB
TokuDB especially is in a bad shape on macOS, maybe it didn't receive much attention in that regard.
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/ft-ops.cc:830:17: error: format specifies type 'long' but the argument has type 'int64_t'
|
(aka 'long long') [-Werror,-Wformat]
|
blocknum.b);
|
^~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/ft-ops.cc:840:17: error: format specifies type 'long' but the argument has type 'int64_t'
|
(aka 'long long') [-Werror,-Wformat]
|
blocknum.b,
|
^~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/serialize/ft_node-serialize.cc:1177:13: error: format specifies type 'long' but the argument
|
has type 'int64_t' (aka 'long long') [-Werror,-Wformat]
|
blocknum.b,
|
^~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/serialize/ft_node-serialize.cc:1204:13: error: format specifies type 'long' but the argument
|
has type 'int64_t' (aka 'long long') [-Werror,-Wformat]
|
node->blocknum.b,
|
^~~~~~~~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/serialize/ft_node-serialize.cc:1260:13: error: format specifies type 'long' but the argument
|
has type 'int64_t' (aka 'long long') [-Werror,-Wformat]
|
node->blocknum.b,
|
^~~~~~~~~~~~~~~~
|
...
|
[ 84%] Building CXX object storage/tokudb/PerconaFT/ft/CMakeFiles/ft_static.dir/serialize/ft-serialize.cc.o
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/serialize/ft-serialize.cc:736:13: error: format specifies type 'unsigned long' but the
|
argument has type 'uint64_t' (aka 'unsigned long long') [-Werror,-Wformat]
|
dump_state_of_toku_deserialize_ft_from();
|
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/serialize/ft-serialize.cc:666:13: note: expanded from macro
|
'dump_state_of_toku_deserialize_ft_from'
|
max_acceptable_lsn.lsn, \
|
^~~~~~~~~~~~~~~~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/serialize/ft-serialize.cc:736:13: error: format specifies type 'unsigned long' but the
|
argument has type 'uint64_t' (aka 'unsigned long long') [-Werror,-Wformat]
|
dump_state_of_toku_deserialize_ft_from();
|
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
...
|
Needs proper %ll format specifier.
[ 84%] Building CXX object storage/tokudb/PerconaFT/ft/CMakeFiles/ft_static.dir/txn/txn.cc.o
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/txn/txn.cc:271:19: error: missing field '__opaque' initializer
|
[-Werror,-Wmissing-field-initializers]
|
.state_cond = ZERO_COND_INITIALIZER, // Not TOKU_COND_INITIALIZER
|
^
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/portability/toku_pthread.h:167:13: note: expanded from macro 'ZERO_COND_INITIALIZER'
|
{ 0 } \
|
^
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/txn/txn.cc:271:19: error: missing field 'psi_cond' initializer
|
[-Werror,-Wmissing-field-initializers]
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/portability/toku_pthread.h:168:5: note: expanded from macro 'ZERO_COND_INITIALIZER'
|
}
|
^
|
ZERO_COND_INITIALIZER is using wrong toku_cond_t struct initializer for first member of type pthread_cond_t and not considering the TOKU_PTHREAD_DEBUG case which has one extra struct member of type pfs_key_t.
[ 75%] Performing install step for 'build_lzma'
|
Making install in api
|
/bin/sh:maria/build-mariadb/storage/tokudb/PerconaFT/xz/src/build_lzma/build-aux/install-sh: Permission denied
|
This issue can be fixed by doing chmod +x for install-sh script. It is being discussed on the mailing list.
/Users/shinnok/Work/mariadb/server/storage/tokudb/tokudb_thread.h:378:13: error: use of undeclared identifier 'pthread_mutex_timedlock'
|
int r = pthread_mutex_timedlock(&_mutex, &waittime);
|
^
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/tokudb_thread.h:486:13: error: use of undeclared identifier 'pthread_mutex_timedlock'
|
int r = pthread_mutex_timedlock(&_mutex, &waittime);
|
^
|
pthread_mutex_timedlock() is not available in pthreads for Mac, as it's not part of the POSIX pthreads spec, I need a suggestion on how to proceed.The encompassing event_t::wait() method seems to be unused. Could be removed? Otherwise rewrite without timedlock.
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/util/omt.cc:946:13: error: taking address of packed member 'value' of class or structure 'toku::omt_internal::omt_node_templated<referenced_xid_tuple, false>' may result in an unaligned pointer value [-Werror,-Waddress-of-packed-member]
|
*out = &n->value;
|
^~~~~~~~
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/util/omt.cc:871:13: note: in instantiation of member function 'toku::omt<referenced_xid_tuple, referenced_xid_tuple *, false>::copyout' requested here
|
copyout(value, &n);
|
^
|
In file included from /Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/ft/txn/txn.h:46:
|
In file included from /Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/util/omt.h:759:
|
/Users/shinnok/Work/mariadb/server/storage/tokudb/PerconaFT/util/omt.cc:762:24: error: taking address of packed member 'value' of class or structure 'toku::omt_internal::omt_node_templated<referenced_xid_tuple, false>' may result in an unaligned pointer value [-Werror,-Waddress-of-packed-member]
|
int r = f(&n.value, idx_root, iterate_extra);
|
^~~~~~~
|
^
|
2 errors generated.
|
This is happening because omt_ classes are declared as packed and clang has -Waddress-of-packed-member when passing the address of a packed member, a legit concern on different architectures. The easiest way to get rid of the errors is to remove the packed attribute from said structs. Also reported MDEV-14223.
[ 87%] Linking CXX shared library libtokufractaltree.dylib
|
Undefined symbols for architecture x86_64:
|
"_debug_sync_C_callback_ptr", referenced from:
|
toku_debug_sync(tokutxn*, char const*) in ydb_row_lock.cc.o
|
|
|
....
|
Was missing link to dbug lib.
[ 88%] Linking CXX shared library libtokufractaltree.dylib
|
Undefined symbols for architecture x86_64:
|
"_opt_debug_sync_timeout", referenced from:
|
toku_debug_sync(tokutxn*, char const*) in ydb_row_lock.cc.o
|
ld: symbol(s) not found for architecture x86_64
|
Not sure what's happening here, I see that opt_debug_sync_timeout is declared as extern, however no immediate definition is available. It is used to inform the compiler via __builtin_expect branch prediction on line toku_debug_sync.h:66:
if (likely(!opt_debug_sync_timeout))
|
return;
|
Attachments
Issue Links
- relates to
-
-
- Closed
-
-
-
- Closed
-
Activity
Worth mentioning is that some, or all, of the fixes reported here are impairing the Travis macOS builds also. Builds were last done 16 days ago.
https://travis-ci.org/MariaDB/server/jobs/364437601
https://travis-ci.org/MariaDB/server/builds/364437590
I've split all fixes in PR#691 into smaller branch specific PRs based on when the issue first originated, to the best of my ability. Sadly, I think I'm out of the merge window this time, but still do proceed and review the fixes at:
https://github.com/MariaDB/server/pull/750
https://github.com/MariaDB/server/pull/752
https://github.com/MariaDB/server/pull/753
Merged the PR that relates to MyRocks: https://github.com/MariaDB/server/pull/753
Attaching a sample macOS build of 10.3 with all my fixes in, featuring 100% completion:
macOS 10.13:
http://buildbot.askmonty.org/buildbot/builders/macos-10-13/builds/73
Ubuntu 16.04:
https://buildbot.askmonty.org/buildbot/builders/kvm-fulltest/builds/13369/steps/compile/logs/stdio
http://buildbot.askmonty.org/buildbot/grid?category=main&branch=tm-10.3-macfixes
http://buildbot.askmonty.org/buildbot/grid?category=experimental&branch=tm-10.3-macfixes
psergey I see that you reverted the change in innobase, can I have the failure output? 
teodor, yes. Was just going to make a note about this: http://lists.askmonty.org/pipermail/commits/2018-May/012548.html
Unfortunately I don't have the output, and it seems difficult to get it out the buildbot. The complaints about failing build were from Marko (I assume x64 linux) and Wlad (re windows).
I think you can observe the compile failures by pushing the patch into a bb-* branch.
Oh, and now the buildbot page is finally loaded:
http://buildbot.askmonty.org/buildbot/builders/winx64-debug/builds/8580
(copied to https://gist.github.com/spetrunia/e0b4cea6e7769ec8ead486ceacd4137d as BB sometimes deletes the logs)
Hmm, that's interesting. Looks like MSVC complains that it can't reinterpret_cast from DWORD (32b unsigned) which is returned by GetCurrentThreadId() to ULINT size_t ULONG_PTR (64b unsigned on on x64) [1]. However, on macOS the complaint is that you cannot convert from opaque type returned by pthread to size_t (only fixable via reinterpret_cast)...
Maybe needs more consideration to not use cast after all. I would be curious about the Linux error too, not sure how to dig up the buildbot builds for that specific commit though. wlad Do you have that at hand in any way?
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa383751(v=vs.85).aspx
I only look at windows builds. Everyone else notices Linux failures. Since there was no noise about broken buildbot, and just marko complained, I'd assume it was 32bit.
Also, if there is no need to introduce casts on platforms other than macOS, maybe you can surround the cast with a nice #if _darwin_ or whatever macOS defines.
All main branches build properly with default CMake args now on macOS 10.13.5 with Apple LLVM version 9.1.0 (clang-902.0.39.2).
MDEV-14223seems to have been closed with fixed status, however it doesn't look like being so.