[MDEV-30831] Cannot compile AWS KMS Plugin for 10.6 Created: 2023-03-10  Updated: 2023-10-20

Status: Open
Project: MariaDB Server
Component/s: Plugin - AWS key management
Affects Version/s: 10.6.13
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Rich Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: None
Environment:

Ubuntu 22.04 LTS


 Description   

Trying to compile just the AWS KMS plugin (aws_key_management) on Ubuntu 22.04 w/ all dependencies installed.

I have followed all instructions: I have all dependencies ready in the environment using "apt build-dep mariadb-server-10.6" and have git-cloned the 10.6 branch of MariaDB server to a folder called "server" in the homedir.

The cmake step "cmake -DPLUGIN_AWS_KEY_MANAGEMENT=DYNAMIC -DAWS_SDK_EXTERNAL_PROJECT=1 -DNOT_FOR_DISTRIBUTION=1 ../server/" succeeds without errors, but the "make aws_key_management" fails with:

[ 48%] Building C object tests/CMakeFiles/aws-c-common-tests.dir/byte_buf_test.c.o
 
/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon/tests/byte_buf_test.c: In function 's_test_buffer_advance':
 
/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon/tests/byte_buf_test.c:236:35: error: 'arr' may be used uninitialized [-Werror=maybe-uninitialized]
 
  236 |     struct aws_byte_buf src_buf = aws_byte_buf_from_empty_array(arr, sizeof(arr));
 
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
In file included from /home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon/tests/byte_buf_test.c:16:
 
/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon/include/aws/common/byte_buf.h:507:36: note: by argument 1 of type 'const void *' to 'aws_byte_buf_from_empty_array' declared here
 
  507 | AWS_COMMON_API struct aws_byte_buf aws_byte_buf_from_empty_array(const void *bytes, size_t capacity);
 
      |                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon/tests/byte_buf_test.c:235:13: note: 'arr' declared here
 
  235 |     uint8_t arr[16];
 
      |             ^~~
 
cc1: all warnings being treated as errors
 
gmake[9]: *** [tests/CMakeFiles/aws-c-common-tests.dir/build.make:146: tests/CMakeFiles/aws-c-common-tests.dir/byte_buf_test.c.o] Error 1
 
gmake[9]: Leaving directory '/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon-build'
 
gmake[8]: *** [CMakeFiles/Makefile2:910: tests/CMakeFiles/aws-c-common-tests.dir/all] Error 2
 
gmake[8]: Leaving directory '/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon-build'
 
gmake[7]: Leaving directory '/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps/build/src/AwsCCommon-build'
 
gmake[7]: *** [Makefile:146: all] Error 2
 
gmake[6]: Leaving directory '/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps'
 
gmake[6]: *** [CMakeFiles/AwsCCommon.dir/build.make:86: build/src/AwsCCommon-stamp/AwsCCommon-build] Error 2
 
gmake[5]: *** [CMakeFiles/Makefile2:87: CMakeFiles/AwsCCommon.dir/all] Error 2
 
gmake[5]: Leaving directory '/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps'
 
gmake[4]: Leaving directory '/home/local/user/build-mariadb/extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-build/.deps'
 
gmake[4]: *** [Makefile:91: all] Error 2
 
CMake Error at CMakeLists.txt:224 (message):
 
  Failed to build third-party libraries.
 
 
 
 
 
-- Configuring incomplete, errors occurred!
 
make[3]: *** [extra/aws_sdk/CMakeFiles/aws_sdk_cpp.dir/build.make:93: extra/aws_sdk/aws_sdk_cpp-prefix/src/aws_sdk_cpp-stamp/aws_sdk_cpp-configure] Error 1
 
make[2]: *** [CMakeFiles/Makefile2:10717: extra/aws_sdk/CMakeFiles/aws_sdk_cpp.dir/all] Error 2
 
make[1]: *** [CMakeFiles/Makefile2:8237: plugin/aws_key_management/CMakeFiles/aws_key_management.dir/rule] Error 2
 
make: *** [Makefile:2825: aws_key_management] Error 2

We have tried changing to try a few other different versions of the AWS SDK by altering the GIT_TAG in "server/extra/aws_sdk/CMakelists.txt" in case the chosen "1.8.29" is bugged but we cannot find one that works - they either fail at the same step or at another.

Any other ideas?

 Comments   
Comment by bulepage [ 2023-05-01 ]

On 20.04 I successful compiled, but I didn't succeed on 22.04

Comment by Kevin M [ 2023-07-05 ]

I have the same issue and attempted with gcc 9 / 10 / and 22.04 default 11

Comment by Daniel Black [ 2023-07-06 ]

I tried, could work it out either.

Suggestion:

Rewrite it in rust as external.

current implementation as you see is very light on what it does https://github.com/MariaDB/server/blob/11.2/plugin/aws_key_management/aws_key_management_plugin.cc

I tried (gcc-13.1.1, culr-8.0.1:

  • 1.9.379,1.10.20,1.10.40 - CurlHttpClient.cpp:414:49: error: ‘CURLOPT_PUT’ is deprecated: since 7.12.1.
  • c {{src/AwsCCommon/source/posix/system_info.c:396:23: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
    void aws_backtrace_log() {}}}
  • 1.11.114, 1.10.57 - link errors everywhere 

    /usr/bin/ld: Aws.cpp:(.text+0xbec): undefined reference to `Aws::Crt::Io::TlsConnectionOptions::~TlsConnectionOptions()'
    		 
    /usr/bin/ld: Aws.cpp:(.text+0xcac): undefined reference to `Aws::Crt::Io::TlsContextOptions::~TlsContextOptions()'
    		 
    /usr/bin/ld: extra/aws_sdk/aws_sdk_cpp/lib/libaws-cpp-sdk-core.a(Aws.cpp.o): in function `std::_Sp_counted_ptr_inplace<Aws::Crt::Io::TlsConnectionOptions, std::allocator<Aws::Crt::Io::TlsConnectionOptions>, (__gnu_cxx::_Lock_policy)2>::_M_dispose()':
    		 
    Aws.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN3Aws3Crt2Io20TlsConnectionOptionsESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE10_M_disposeEv[_ZNSt23_Sp_counted_ptr_inplaceIN3Aws3Crt2Io20TlsConnectionOptionsESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE10_M_disposeEv]+0x5): undefined reference to `Aws::Crt::Io::TlsConnectionOptions::~TlsConnectionOptions()'
    		 
    /usr/bin/ld: extra/aws_sdk/aws_sdk_cpp/lib/libaws-cpp-sdk-core.a(Aws.cpp.o): in function `std::_Sp_counted_ptr_inplace<Aws::Crt::Io::ClientBootstrap, std::allocator<Aws::Crt::Io::ClientBootstrap>, (__gnu_cxx::_Lock_policy)2>::_M_dispose()':
    		 
    Aws.cpp:(.text._ZNSt23_Sp_counted_ptr_inplaceIN3Aws3Crt2Io15ClientBootstrapESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE10_M_disposeEv[_ZNSt23_Sp_counted_ptr_inplaceIN3Aws3Crt2Io15ClientBootstrapESaIS3_ELN9__gnu_cxx12_Lock_policyE2EE10_M_disposeEv]+0x5): undefined reference to `Aws::Crt::Io::ClientBootstrap::~ClientBootstrap()'
    		 
    /usr/bin/ld: extra/aws_sdk/aws_sdk_cpp/lib/libaws-cpp-sdk-core.a(HttpRequest.cpp.o): in function `Aws::Http::HttpRequest::ToCrtHttpRequest()':
    		 
    HttpRequest.cpp:(.text+0x426): undefined reference to `Aws::Crt::ApiAllocator()'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0x431): undefined reference to `Aws::Crt::Http::HttpRequest::HttpRequest(aws_allocator*)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0x4e8): undefined reference to `Aws::Crt::Http::HttpMessage::SetBody(std::shared_ptr<std::istream> const&)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0x575): undefined reference to `Aws::Crt::ByteCursorFromCString(char const*)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0x58e): undefined reference to `Aws::Crt::ByteCursorFromCString(char const*)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0x5a9): undefined reference to `Aws::Crt::Http::HttpMessage::AddHeader(aws_http_header const&)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0xb3e): undefined reference to `Aws::Crt::ByteCursorFromCString(char const*)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0xb49): undefined reference to `Aws::Crt::Http::HttpRequest::SetPath(aws_byte_cursor)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0xb6e): undefined reference to `Aws::Crt::ByteCursorFromCString(char const*)'
    		 
    /usr/bin/ld: HttpRequest.cpp:(.text+0xb79): undefined reference to `Aws::Crt::Http::HttpRequest::SetMethod(aws_byte_cursor)'

  • 1.9.200 - openssl-3.0 deprecated functions.

curl errors - replace CURLOPT_PUT -> CURLOPT_UPLOAD. Later version look like they should autodetect hwoever don't include curl. Once corrected link error appear.

https://github.com/aws/aws-sdk-cpp/tags?after=1.11.0

Comment by Vladislav Vaintroub [ 2023-10-20 ]

Yes, the external project uses a very old GIT TAG, 4 years old. Since we stopped compiling it in CI, I'd say no wonder it does not build

Generated at Thu Feb 08 10:19:13 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.