[#MDEV-10332] Server 10.2: Add support for OpenSSL 1.1

Comment by Brian Evans [ 2016-08-31 ]

OpenSSL 1.1.0 GA is now available and affects 10.1 as well

[  4%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o

cd

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql-abi_x86_64.amd64/mysys_ssl

&& /usr/bin/x86_64-pc-linux-gnu-g++  -DHAVE_CONFIG_H -DHAVE_OPENSSL

-I/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql-abi_x86_64.amd64/include

-I/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/include

-I/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl   -march=native

-mtune=native -O2 -pipe -felide-constructors -fno-strict-aliasing  -pie -fPIC

-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4 -fno-exceptions

-fno-rtti -DNDEBUG -DDBUG_OFF   -fPIC -o CMakeFiles/mysys_ssl.dir/my_md5.cc.o

-c /var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc: In

function ‘void md5_result(MD5_CONTEXT*, uchar*)’:

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc:86:29:

error: ‘EVP_MD_CTX_cleanup’ was not declared in this scope

   EVP_MD_CTX_cleanup(context);

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc: In

function ‘void my_md5(uchar*, const char*, size_t)’:

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc:102:15:

error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be

defined

   MD5_CONTEXT md5_context;

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc: In

function ‘void my_md5_multi(uchar*, ...)’:

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc:127:15:

error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be

defined

   MD5_CONTEXT md5_context;

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc: In

function ‘size_t my_md5_context_size()’:

/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql/mysys_ssl/my_md5.cc:140:28:

error: invalid application of ‘sizeof’ to incomplete type ‘MD5_CONTEXT {aka

evp_md_ctx_st}’

   return sizeof(MD5_CONTEXT);

make[2]: *** [mysys_ssl/CMakeFiles/mysys_ssl.dir/build.make:111:

mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o] Error 1

make[2]: Leaving directory

'/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql-abi_x86_64.amd64'

make[1]: *** [CMakeFiles/Makefile2:3772:

mysys_ssl/CMakeFiles/mysys_ssl.dir/all] Error 2

make[1]: Leaving directory

'/var/tmp/portage/dev-db/mariadb-10.1.16/work/mysql-abi_x86_64.amd64'

make: *** [Makefile:150: all] Error 2

Comment by Honza Horak [ 2016-11-14 ]

Since openssl 1.1 has landed in Fedora 26, we'll need to do something with not only MariaDB, but also MySQL. I think there will be quite big overlap with MySQL code regarding the openssl porting, so I wanted to ask what might be the best way to not implement things twice differently, but only once correctly – any working pattern to work on such kind of things? Or is anybody already working on this?

Comment by Honza Horak [ 2016-11-14 ]

Btw. patch for PostgreSQL might give some idea about what kind of fixes we'll need:
https://github.com/postgres/postgres/commit/593d4e47db7af1a3a5dd6b6b1971f181b5566dbd

Comment by Georg Richter [ 2016-11-14 ]

The main problem is in mysys_ssl, more specific in my_crypt.cc which abuses OpenSSL API by accessing private members of EVP_MD_CTX - those aren't accessible since OpenSSL 1.1 anymore.

Comment by Honza Horak [ 2017-01-03 ]

I'm wondering whether there are any concrete plans for porting MariaDB code to OpenSSL 1.1, or whether anybody has already tried that. If not for now, are there any reasons to not do it at this point? I can understand there simply might be lack of man-power/priority, so this is just to make it clear to me where we stand..

Comment by Sergei Golubchik [ 2017-01-09 ]

Yes, absolutely, concrete plans. Last email on this topic is https://lists.launchpad.net/maria-developers/msg10254.html which is my review of the patch for this issue, dated 29 Dec 2016. I believe that some new distributions already use OpenSSL 1.1, so we have to do it, there is not much of a choice.

Comment by Honza Horak [ 2017-01-09 ]

That's great news, thank you.

Comment by pgnd [ 2017-03-06 ]

can we pls get a status update on openssl 1.1 support in MDB?

Comment by Sergei Golubchik [ 2017-03-07 ]

georg will correct me, if I'm wrong, but as far as I understand, It's pretty much code-complete now. It cannot connect to YaSSL-builds of MariaDB, because of ~~MDEV-12190~~, this should be fixed soon. And another round of testing after that, to make sure everything works smoothly.

Comment by Christian Hesse [ 2017-03-07 ]

If this is "pretty much code-complete"... Where to find the code? There is no git branch named ~~MDEV-10332~~.

Comment by pgnd [ 2017-03-07 ]

fyi, testing 10.2 branch with my usual configure + build, but linking against openssl v110 libs/headers,

cd src/mariadb

git log | head

	commit 5b07334b32f4add094981b8346fff468436e1e71

	Author: Marko Mäkelä <marko.makela@mariadb.com>

	Date:   Mon Mar 6 11:53:51 2017 +0200

	    Remove an unused declaration.

	    The function thd_supports_xa() was removed in MySQL 5.7 in WL#8843.

	commit 7331b83eede25e0186dda0b8cd96152d64df3d8a

	Author: Marko Mäkelä <marko.makela@mariadb.com>

cmake ..  --debug-output -Wno-dev \

-G "Unix Makefiles" \

-DBUILD_SHARED_LIBS=1 \

-DWITH_MYSQLD_LDFLAGS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lssl -lcrypto" \

 -DWITH_SSL=/usr/local/openssl11 \

 -DOPENSSL_ROOT_DIR=/usr/local/openssl11 \

 -DOPENSSL_INCLUDE_DIR=/usr/local/openssl11/include \

 -DOPENSSL_LIBRARIES=/usr/local/openssl11/lib64/libssl.so \

 -DCRYPTO_LIBRARY=/usr/local/openssl11/lib64/libcrypto.so \

...

	[  0%] Built target abi_check

	[  0%] Built target INFO_SRC

	[  0%] Built target INFO_BIN

	[  0%] Building C object extra/readline/CMakeFiles/readline.dir/readline.c.o

...

	[  7%] Linking C static library libstrings.a

	[  7%] Built target strings

	Scanning dependencies of target mysys_ssl

	[  7%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_sha1.cc.o

	[  7%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_sha2.cc.o

	[  7%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc: In function ‘void md5_result(MD5_CONTEXT*, uchar*)’:

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc:86:29: error: ‘EVP_MD_CTX_cleanup’ was not declared in this scope

	   EVP_MD_CTX_cleanup(context);

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc: In function ‘void my_md5(uchar*, const char*, size_t)’:

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc:102:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

	   MD5_CONTEXT md5_context;

	               ^~~~~~~~~~~

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc: In function ‘void my_md5_multi(uchar*, ...)’:

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc:127:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

	   MD5_CONTEXT md5_context;

	               ^~~~~~~~~~~

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc: In function ‘size_t my_md5_context_size()’:

	/usr/local/src/mariadb/mysys_ssl/my_md5.cc:140:28: error: invalid application of ‘sizeof’ to incomplete type ‘MD5_CONTEXT {aka evp_md_ctx_st}’

	   return sizeof(MD5_CONTEXT);

	mysys_ssl/CMakeFiles/mysys_ssl.dir/build.make:110: recipe for target 'mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o' failed

	make[2]: *** [mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o] Error 1

	CMakeFiles/Makefile2:4104: recipe for target 'mysys_ssl/CMakeFiles/mysys_ssl.dir/all' failed

	make[1]: *** [mysys_ssl/CMakeFiles/mysys_ssl.dir/all] Error 2

	Makefile:138: recipe for target 'all' failed

	make: *** [all] Error 2

fyi, @ openssl11/openssl-1.1.0e/CHANGES

re: "error: ‘EVP_MD_CTX_cleanup’ was not declared in this scope"

     Additional changes:

     1) EVP_MD_CTX_cleanup(), EVP_CIPHER_CTX_cleanup() and

        HMAC_CTX_cleanup() were removed.  HMAC_CTX_reset() and

        EVP_MD_CTX_reset() should be called instead to reinitialise

        an already created structure.

Comment by Sergei Golubchik [ 2017-03-07 ]

eworm, there's a branch now, "~~MDEV-10332~~"

Comment by Christian Hesse [ 2017-03-07 ]

But "~~MDEV-10332~~" is identical to "10.2".

Comment by pgnd [ 2017-03-07 ]

> there's a branch now, "~~MDEV-10332~~"

which fails similary ...

[  7%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o

/usr/local/src/mariadb_MDEV-10332/mysys_ssl/my_md5.cc: In function ‘void my_md5(uchar*, const char*, size_t)’:

/usr/local/src/mariadb_MDEV-10332/mysys_ssl/my_md5.cc:102:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

   MD5_CONTEXT md5_context;

               ^~~~~~~~~~~

/usr/local/src/mariadb_MDEV-10332/mysys_ssl/my_md5.cc: In function ‘void my_md5_multi(uchar*, ...)’:

/usr/local/src/mariadb_MDEV-10332/mysys_ssl/my_md5.cc:127:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

   MD5_CONTEXT md5_context;

               ^~~~~~~~~~~

/usr/local/src/mariadb_MDEV-10332/mysys_ssl/my_md5.cc: In function ‘size_t my_md5_context_size()’:

/usr/local/src/mariadb_MDEV-10332/mysys_ssl/my_md5.cc:140:28: error: invalid application of ‘sizeof’ to incomplete type ‘MD5_CONTEXT {aka evp_md_ctx_st}’

   return sizeof(MD5_CONTEXT);

mysys_ssl/CMakeFiles/mysys_ssl.dir/build.make:110: recipe for target 'mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o' failed

make[2]: *** [mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o] Error 1

CMakeFiles/Makefile2:3694: recipe for target 'mysys_ssl/CMakeFiles/mysys_ssl.dir/all' failed

make[1]: *** [mysys_ssl/CMakeFiles/mysys_ssl.dir/all] Error 2

Makefile:149: recipe for target 'all' failed

make: *** [all] Error 2

Comment by pgnd [ 2017-03-07 ]

> But "~~MDEV-10332~~" is identical to "10.2".

That would explain it ...

Comment by Georg Richter [ 2017-03-08 ]

For testing please try branch ~~MDEV-10332~~ (sorry, didn't notice that last push from yesterday failed due to connection problems).

Please note, that the branch will not work correctly on Windows when using Schannel as TLS library (this requires more testing and push of fix for ~~CONC-241~~).

Comment by pgnd [ 2017-03-08 ]

latest pull

git log | head

	commit f88977321038cf04bed974afce953ae7a0dca2e6

	Author: Georg Richter <georg@mariadb.com>

	Date:   Wed Mar 8 17:39:47 2017 +0100

	    Initial implementation for MDEV-10332:

	      support for OpenSSL 1.1 and LibreSSL

	      tested against OpenSSL 1.0.1, 1.0.2, 1.1.0, Yassl and LibreSSL

	      not working on Windows with native SChannel support, due to wrong cipher mapping: Latter one requires push of CONC-241 fixes.

	      Please note that OpenSSL 0.9.8 and OpenSSL 1.1.0 will not work: Even if the build succeeds, test cases will fail with various errors, especially when using different tls libraries or versions for client and server.

, linking to openssl v11x,

config'd (NOTING change in available flags & names) with

cmake \

...

 -DWITH_MYSQLD_LDFLAGS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lssl -lcrypto" \

 -DOPENSSL_INCLUDE_DIR=/usr/local/openssl11/include \

 -DOPENSSL_SSL_LIBRARY=/usr/local/openssl11/lib64/libssl.so \

 -DOPENSSL_CRYPTO_LIBRARY=/usr/local/openssl11/lib64/libcrypto.so \

...

build fails (plus a number of non-fatal, deprecation warnings, with gcc 6.3.1, fwiw)

make V=1

...

[ 67%] Built target thr_timer

[ 67%] Building C object mysys/CMakeFiles/test_hash.dir/hash.c.o

[ 67%] Linking CXX executable test_hash

[ 67%] Built target test_hash

[ 70%] Built target mariadb_obj

[ 70%] Built target mariadbclient

[ 70%] Linking C shared library libmariadb.so

CMakeFiles/mariadb_obj.dir/secure/openssl.c.o: In function `ma_tls_start':

openssl.c:(.text+0x14d): undefined reference to `SSL_load_error_strings'

openssl.c:(.text+0x154): undefined reference to `OpenSSL_add_all_algorithms'

CMakeFiles/mariadb_obj.dir/secure/openssl.c.o: In function `ma_tls_end':

openssl.c:(.text+0x1ab): undefined reference to `EVP_cleanup'

openssl.c:(.text+0x1b2): undefined reference to `CRYPTO_cleanup_all_ex_data'

openssl.c:(.text+0x1b9): undefined reference to `ERR_free_strings'

openssl.c:(.text+0x1c0): undefined reference to `CONF_modules_free'

CMakeFiles/mariadb_obj.dir/secure/openssl.c.o: In function `ma_tls_verify_server_cert':

openssl.c:(.text+0x8fa): undefined reference to `ASN1_STRING_data'

collect2: error: ld returned 1 exit status

libmariadb/libmariadb/CMakeFiles/libmariadb.dir/build.make:136: recipe for target 'libmariadb/libmariadb/libmariadb.so.3' failed

make[2]: *** [libmariadb/libmariadb/libmariadb.so.3] Error 1

CMakeFiles/Makefile2:3879: recipe for target 'libmariadb/libmariadb/CMakeFiles/libmariadb.dir/all' failed

make[1]: *** [libmariadb/libmariadb/CMakeFiles/libmariadb.dir/all] Error 2

Makefile:149: recipe for target 'all' failed

make: *** [all] Error 2

Comment by Georg Richter [ 2017-03-08 ]

Can you try follwing TLS options

-DWITH_SSL=system -DWITH_SSL_PATH=/usr/local/openssl11

Comment by pgnd [ 2017-03-08 ]

> Can you try follwing TLS options
>
> -DWITH_SSL=system -DWITH_SSL_PATH=/usr/local/openssl11

cmake \

...

 -DWITH_SSL=system \

 -DWITH_SSL_PATH=/usr/local/openssl11 \

 -DWITH_MYSQLD_LDFLAGS="-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lssl -lcrypto" \

 -DOPENSSL_INCLUDE_DIR=/usr/local/openssl11/include \

 -DOPENSSL_SSL_LIBRARY=/usr/local/openssl11/lib64/libssl.so \

 -DOPENSSL_CRYPTO_LIBRARY=/usr/local/openssl11/lib64/libcrypto.so \

...

grep -i SSL CMakeCache.txt | egrep -v "STATIC|msg1|^//" | sort

	FIND_PACKAGE_MESSAGE_DETAILS_OpenSSL:INTERNAL=[/usr/local/openssl11/lib64/libssl.so;/usr/local/openssl11/lib64/libcrypto.so][/usr/local/openssl11/include][v1.1.0e()]

	_OPENSSL_CFLAGS_I:INTERNAL=

	_OPENSSL_CFLAGS:INTERNAL=-I/usr/local/openssl11/include

	_OPENSSL_CFLAGS_OTHER:INTERNAL=

	OPENSSL_CRYPTO_LIBRARY-ADVANCED:INTERNAL=1

	OPENSSL_CRYPTO_LIBRARY:FILEPATH=/usr/local/openssl11/lib64/libcrypto.so

	_OPENSSL_FOUND:INTERNAL=1

	OPENSSL_INCLUDE_DIR-ADVANCED:INTERNAL=1

	_OPENSSL_INCLUDEDIR:INTERNAL=/usr/local/openssl11/include

	OPENSSL_INCLUDE_DIR:PATH=/usr/local/openssl11/include

	_OPENSSL_INCLUDE_DIRS:INTERNAL=/usr/local/openssl11/include

	_OPENSSL_LDFLAGS:INTERNAL=-L/usr/local/openssl11/lib64;-lssl;-lcrypto

	_OPENSSL_LDFLAGS_OTHER:INTERNAL=

	_OPENSSL_LIBDIR:INTERNAL=/usr/local/openssl11/lib64

	_OPENSSL_LIBRARIES:INTERNAL=ssl;crypto

	_OPENSSL_LIBRARY_DIRS:INTERNAL=/usr/local/openssl11/lib64

	_OPENSSL_LIBS:INTERNAL=

	_OPENSSL_LIBS_L:INTERNAL=

	_OPENSSL_LIBS_OTHER:INTERNAL=

	_OPENSSL_LIBS_PATHS:INTERNAL=

	_OPENSSL_openssl_INCLUDEDIR:INTERNAL=

	_OPENSSL_openssl_LIBDIR:INTERNAL=

	_OPENSSL_openssl_PREFIX:INTERNAL=

	_OPENSSL_openssl_VERSION:INTERNAL=

	_OPENSSL_PREFIX:INTERNAL=/usr/local/openssl11

	OPENSSL_SSL_LIBRARY-ADVANCED:INTERNAL=1

	OPENSSL_SSL_LIBRARY:FILEPATH=/usr/local/openssl11/lib64/libssl.so

	_OPENSSL_VERSION:INTERNAL=1.1.0e

	__pkg_config_checked__OPENSSL:INTERNAL=1

	WITH_MYSQLD_LDFLAGS:STRING=-L/usr/local/openssl11/lib64 -Wl,-rpath,/usr/local/openssl11/lib64 -lssl -lcrypto

	WITH_SSL_PATH:UNINITIALIZED=/usr/local/openssl11

	WITH_SSL:UNINITIALIZED=system

make -4

...

	[ 21%] Built target mariadb_obj

	[ 21%] Linking C shared library libmariadb.so

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_same.c.o

	CMakeFiles/mariadb_obj.dir/secure/openssl.c.o: In function `ma_tls_start':

	openssl.c:(.text+0x14d): undefined reference to `SSL_load_error_strings'

	openssl.c:(.text+0x154): undefined reference to `OpenSSL_add_all_algorithms'

	CMakeFiles/mariadb_obj.dir/secure/openssl.c.o: In function `ma_tls_end':

	openssl.c:(.text+0x1ab): undefined reference to `EVP_cleanup'

	openssl.c:(.text+0x1b2): undefined reference to `CRYPTO_cleanup_all_ex_data'

	openssl.c:(.text+0x1b9): undefined reference to `ERR_free_strings'

	openssl.c:(.text+0x1c0): undefined reference to `CONF_modules_free'

	CMakeFiles/mariadb_obj.dir/secure/openssl.c.o: In function `ma_tls_verify_server_cert':

	openssl.c:(.text+0x8fa): undefined reference to `ASN1_STRING_data'

	collect2: error: ld returned 1 exit status

	libmariadb/libmariadb/CMakeFiles/libmariadb.dir/build.make:136: recipe for target 'libmariadb/libmariadb/libmariadb.so.3' failed

	make[2]: *** [libmariadb/libmariadb/libmariadb.so.3] Error 1

	CMakeFiles/Makefile2:4177: recipe for target 'libmariadb/libmariadb/CMakeFiles/libmariadb.dir/all' failed

	make[1]: *** [libmariadb/libmariadb/CMakeFiles/libmariadb.dir/all] Error 2

	make[1]: *** Waiting for unfinished jobs....

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_soundex.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_arr_appstr.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_sort.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_tempfile.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_tempdir.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_unixpath.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mf_wcomp.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/mulalloc.c.o

	[ 21%] Building C object mysys/CMakeFiles/mysys.dir/my_access.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_alloc.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_bitmap.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_bit.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_chsize.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_compress.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_copy.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_create.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_delete.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_div.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_error.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_file.c.o

	[ 22%] Building C object mysys/CMakeFiles/mysys.dir/my_fopen.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_fstream.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_gethwaddr.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_getopt.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_getsystime.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_getwd.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_compare.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_init.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_lib.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_lock.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_malloc.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_mess.c.o

	[ 23%] Building C object mysys/CMakeFiles/mysys.dir/my_mkdir.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_mmap.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_once.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_open.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_pread.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_pthread.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_quick.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_read.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_redel.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_rename.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_seek.c.o

	[ 24%] Building C object mysys/CMakeFiles/mysys.dir/my_sleep.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_static.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_symlink.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_symlink2.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_sync.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_thr_init.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_basename.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/my_write.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/ptr_cmp.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/queues.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/stacktrace.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/string.c.o

	[ 25%] Building C object mysys/CMakeFiles/mysys.dir/thr_alarm.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/thr_lock.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/thr_mutex.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/thr_rwlock.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/thr_timer.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/tree.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/typelib.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/base64.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/my_memmem.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/my_getpagesize.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/lf_alloc-pin.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/lf_dynarray.c.o

	[ 26%] Building C object mysys/CMakeFiles/mysys.dir/lf_hash.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/safemalloc.c.o

	[ 27%] Building CXX object mysys/CMakeFiles/mysys.dir/my_new.cc.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/my_atomic.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/my_getncpus.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/my_safehash.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/my_chmod.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/my_rnd.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/my_uuid.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/waiting_threads.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/wqueue.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/ma_dyncol.c.o

	[ 27%] Building C object mysys/CMakeFiles/mysys.dir/__/sql-common/my_time.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_rdtsc.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_context.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/psi_noop.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_atomic_writes.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/file_logger.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_addr_resolve.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_setuser.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_alarm.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_largepage.c.o

	[ 28%] Building C object mysys/CMakeFiles/mysys.dir/my_lockmem.c.o

	[ 28%] Linking CXX static library libmysys.a

	[ 28%] Built target mysys

	Makefile:138: recipe for target 'all' failed

	make: *** [all] Error 2

...

Comment by pgnd [ 2017-03-09 ]

@Georg Richter

this openssl v10x config of 10.2 branch

cmake \

... \

 -DWITH_MYSQLD_LDFLAGS=-L/usr/local/openssl10/lib64;-Wl,-rpath,/usr/local/openssl10/lib64;-lssl;-lcrypto

 -DWITH_SSL=/usr/local/openssl10

 -DWITH_SSL_PATH=/usr/local/openssl10

 -DOPENSSL_INCLUDE_DIR=/usr/local/openssl10/include

 -DOPENSSL_SSL_LIBRARY=/usr/local/openssl10/lib64/libssl.so

 -DOPENSSL_CRYPTO_LIBRARY=/usr/local/openssl10/lib64/libcrypto.so

...

now builds/installs/runs OK

building ~~MDEV-10332~~ branch using openssl v11x, with similar config

cmake \

... \

 -DWITH_MYSQLD_LDFLAGS=-L/usr/local/openssl11/lib64;-Wl,-rpath,/usr/local/openssl11/lib64;-lssl;-lcrypto

 -DWITH_SSL=/usr/local/openssl11

 -DWITH_SSL_PATH=/usr/local/openssl11

 -DOPENSSL_INCLUDE_DIR=/usr/local/openssl11/include

 -DOPENSSL_SSL_LIBRARY=/usr/local/openssl11/lib64/libssl.so

 -DOPENSSL_CRYPTO_LIBRARY=/usr/local/openssl11/lib64/libcrypto.so

...

build fails

[ 17%] Linking C executable view

[ 17%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_crypt.cc.o

../../libmariadb/libmariadbclient.a(openssl.c.o): In function `ma_tls_start':

openssl.c:(.text+0x14d): undefined reference to `SSL_load_error_strings'

openssl.c:(.text+0x154): undefined reference to `OpenSSL_add_all_algorithms'

../../libmariadb/libmariadbclient.a(openssl.c.o): In function `ma_tls_end':

openssl.c:(.text+0x1ab): undefined reference to `EVP_cleanup'

openssl.c:(.text+0x1b2): undefined reference to `CRYPTO_cleanup_all_ex_data'

openssl.c:(.text+0x1b9): undefined reference to `ERR_free_strings'

openssl.c:(.text+0x1c0): undefined reference to `CONF_modules_free'

collect2: error: ld returned 1 exit status

libmariadb/unittest/libmariadb/CMakeFiles/sqlite3.dir/build.make:100: recipe for target 'libmariadb/unittest/libmariadb/sqlite3' failed

make[2]: *** [libmariadb/unittest/libmariadb/sqlite3] Error 1

CMakeFiles/Makefile2:4478: recipe for target 'libmariadb/unittest/libmariadb/CMakeFiles/sqlite3.dir/all' failed

make[1]: *** [libmariadb/unittest/libmariadb/CMakeFiles/sqlite3.dir/all] Error 2

../../libmariadb/libmariadbclient.a(openssl.c.o): In function `ma_tls_start':

openssl.c:(.text+0x14d): undefined reference to `SSL_load_error_strings'

openssl.c:(.text+0x154): undefined reference to `OpenSSL_add_all_algorithms'

../../libmariadb/libmariadbclient.a(openssl.c.o): In function `ma_tls_end':

openssl.c:(.text+0x1ab): undefined reference to `EVP_cleanup'

openssl.c:(.text+0x1b2): undefined reference to `CRYPTO_cleanup_all_ex_data'

openssl.c:(.text+0x1b9): undefined reference to `ERR_free_strings'

openssl.c:(.text+0x1c0): undefined reference to `CONF_modules_free'

collect2: error: ld returned 1 exit status

libmariadb/unittest/libmariadb/CMakeFiles/view.dir/build.make:100: recipe for target 'libmariadb/unittest/libmariadb/view' failed

make[2]: *** [libmariadb/unittest/libmariadb/view] Error 1

CMakeFiles/Makefile2:4439: recipe for target 'libmariadb/unittest/libmariadb/CMakeFiles/view.dir/all' failed

make[1]: *** [libmariadb/unittest/libmariadb/CMakeFiles/view.dir/all] Error 2

[ 17%] Linking CXX static library libmysys_ssl.a

[ 17%] Built target mysys_ssl

Makefile:149: recipe for target 'all' failed

make: *** [all] Error 2

Comment by Honza Horak [ 2017-03-10 ]

I haven't looked closer whether the patches are related, but there are some in MySQL tracker now: https://bugs.mysql.com/bug.php?id=83814

Comment by Daniel Black [ 2017-04-04 ]

openssl (master %)]$ ./config  --prefix=/usr/local/openssl11

openssl (master %)]$ make -j32

openssl (master %)]$ sudo make  install

build-mariadb-server-10.2]$ cmake ../mariadb-server -DOPENSSL_ROOT_DIR=/usr/local/openssl11

/home/dan/repos/mariadb-server/vio/vio.c: In function ‘vio_end’:

/home/dan/repos/mariadb-server/vio/vio.c:392:3: warning: ‘ERR_remove_thread_state’ is deprecated [-Wdeprecated-declarations]

   ERR_remove_state(0);

   ^~~~~~~~~~~~~~~~

In file included from /usr/local/openssl11/include/openssl/ct.h:13:0,

                 from /usr/local/openssl11/include/openssl/ssl.h:61,

                 from /home/dan/repos/mariadb-server/include/violite.h:146,

                 from /home/dan/repos/mariadb-server/vio/vio_priv.h:27,

                 from /home/dan/repos/mariadb-server/vio/vio.c:24:

/usr/local/openssl11/include/openssl/err.h:246:25: note: declared here

 DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

/usr/local/openssl11/include/openssl/opensslconf.h:111:35: note: in definition of macro ‘DECLARE_DEPRECATED’

 # define DECLARE_DEPRECATED(f)    f __attribute__ ((deprecated));

/usr/local/openssl11/include/openssl/err.h:246:1: note: in expansion of macro ‘DEPRECATEDIN_1_1_0’

 DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

 ^~~~~~~~~~~~~~~~~~

/home/dan/repos/mariadb-server/vio/viosslfactories.c: In function ‘get_dh2048’:

/home/dan/repos/mariadb-server/vio/viosslfactories.c:62:5: error: dereferencing pointer to incomplete type ‘DH {aka struct dh_st}’

   dh->p=BN_bin2bn(dh2048_p,sizeof(dh2048_p),NULL);

^~

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘void md5_result(MD5_CONTEXT*, uchar*)’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:86:29: error: ‘EVP_MD_CTX_cleanup’ was not declared in this scope

   EVP_MD_CTX_cleanup(context);

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘void my_md5(uchar*, const char*, size_t)’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:102:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

   MD5_CONTEXT md5_context;

               ^~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘void my_md5_multi(uchar*, ...)’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:127:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

   MD5_CONTEXT md5_context;

               ^~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘size_t my_md5_context_size()’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:140:28: error: invalid application of ‘sizeof’ to incomplete type ‘MD5_CONTEXT {aka evp_md_ctx_st}’

   return sizeof(MD5_CONTEXT);

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:39:18: error: field ‘ctx’ has incomplete type ‘EVP_CIPHER_CTX {aka evp_cipher_ctx_st}’

   EVP_CIPHER_CTX ctx;

^~~

In file included from /usr/local/openssl11/include/openssl/evp.h:14:0,

                 from /home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:26:

/usr/local/openssl11/include/openssl/ossl_typ.h:90:16: note: forward declaration of ‘EVP_CIPHER_CTX {aka struct evp_cipher_ctx_st}’

 typedef struct evp_cipher_ctx_st EVP_CIPHER_CTX;

                ^~~~~~~~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc: In destructor ‘virtual MyCTX::~MyCTX()’:

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:31:29: warning: ‘void ERR_remove_thread_state(void*)’ is deprecated [-Wdeprecated-declarations]

 #define ERR_remove_state(X) ERR_remove_thread_state(NULL)

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:41:52: note: in expansion of macro ‘ERR_remove_state’

   virtual ~MyCTX() { EVP_CIPHER_CTX_cleanup(&ctx); ERR_remove_state(0); }

                                                    ^~~~~~~~~~~~~~~~

In file included from /usr/local/openssl11/include/openssl/aes.h:13:0,

                 from /home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:27:

/usr/local/openssl11/include/openssl/err.h:246:25: note: declared here

 DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

/usr/local/openssl11/include/openssl/opensslconf.h:111:35: note: in definition of macro ‘DECLARE_DEPRECATED’

 # define DECLARE_DEPRECATED(f)    f __attribute__ ((deprecated));

/usr/local/openssl11/include/openssl/err.h:246:1: note: in expansion of macro ‘DEPRECATEDIN_1_1_0’

 DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

 ^~~~~~~~~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:31:57: warning: ‘void ERR_remove_thread_state(void*)’ is deprecated [-Wdeprecated-declarations]

 #define ERR_remove_state(X) ERR_remove_thread_state(NULL)

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:41:52: note: in expansion of macro ‘ERR_remove_state’

   virtual ~MyCTX() { EVP_CIPHER_CTX_cleanup(&ctx); ERR_remove_state(0); }

                                                    ^~~~~~~~~~~~~~~~

In file included from /usr/local/openssl11/include/openssl/aes.h:13:0,

                 from /home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:27:

/usr/local/openssl11/include/openssl/err.h:246:25: note: declared here

 DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

/usr/local/openssl11/include/openssl/opensslconf.h:111:35: note: in definition of macro ‘DECLARE_DEPRECATED’

 # define DECLARE_DEPRECATED(f)    f __attribute__ ((deprecated));

/usr/local/openssl11/include/openssl/err.h:246:1: note: in expansion of macro ‘DEPRECATEDIN_1_1_0’

 DEPRECATEDIN_1_1_0(void ERR_remove_thread_state(void *))

 ^~~~~~~~~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc: In function ‘int my_random_bytes(uchar*, int)’:

/home/dan/repos/mariadb-server/mysys_ssl/my_crypt.cc:291:35: error: ‘RAND_SSLeay’ was not declared in this scope

   RAND_METHOD *rand = RAND_SSLeay();

mysys_ssl/CMakeFiles/mysys_ssl.dir/build.make:206: recipe for target 'mysys_ssl/CMakeFiles/mysys_ssl.dir/my_crypt.cc.o' failed

make[2]: *** [mysys_ssl/CMakeFiles/mysys_ssl.dir/my_crypt.cc.o] Error 1

make[2]: *** Waiting for unfinished jobs....

[ 24%] Building CXX object mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘void my_md5(uchar*, const char*, size_t)’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:106:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

   MD5_CONTEXT md5_context;

               ^~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘void my_md5_multi(uchar*, ...)’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:131:15: error: aggregate ‘MD5_CONTEXT md5_context’ has incomplete type and cannot be defined

   MD5_CONTEXT md5_context;

               ^~~~~~~~~~~

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc: In function ‘size_t my_md5_context_size()’:

/home/dan/repos/mariadb-server/mysys_ssl/my_md5.cc:144:28: error: invalid application of ‘sizeof’ to incomplete type ‘MD5_CONTEXT {aka evp_md_ctx_st}’

   return sizeof(MD5_CONTEXT);

mysys_ssl/CMakeFiles/mysys_ssl.dir/build.make:182: recipe for target 'mysys_ssl/CMakeFiles/mysys_ssl.dir/my_md5.cc.o' failed

...

my_crypt.cc - appears to require that EVP_CIPHER_CTX ctx is a pointer and operated using: /usr/local/openssl11/include/openssl/evp.h

EVP_CIPHER_CTX *EVP_CIPHER_CTX_new(void);

int EVP_CIPHER_CTX_reset(EVP_CIPHER_CTX *c);

void EVP_CIPHER_CTX_free(EVP_CIPHER_CTX *c);

int EVP_CIPHER_CTX_set_key_length(EVP_CIPHER_CTX *x, int keylen);

int EVP_CIPHER_CTX_set_padding(EVP_CIPHER_CTX *c, int pad);

int EVP_CIPHER_CTX_ctrl(EVP_CIPHER_CTX *ctx, int type, int arg, void *ptr);

int EVP_CIPHER_CTX_rand_key(EVP_CIPHER_CTX *ctx, unsigned char *key);

Comment by Georg Richter [ 2017-04-06 ]

Daniel,

please use ~~MDEV-10332~~ branch

Comment by Daniel Black [ 2017-04-06 ]

georg sorry. I did see that eventually, just forgot to remove comments.

Comment by Georg Richter [ 2018-03-15 ]

According to the OpenSSL docs, beginning of version 1.1.0 future versions will be api/abi compatible, which will alllow you to upgrade the OpenSSL library without any modifications in source code.

Unfortunately this is not the case, since MariaDB still tries to allocate stack via memory instead of using the api functions for allocatiing context objects:

Also:

On Windows it's not possible to build a debug version with OpenSSL 1.1
build with OpenSSL 1.1.1 fails due to different context sizes.

Comment by Sergei Golubchik [ 2018-03-19 ]

I've created ~~MDEV-15596~~ for this

[MDEV-10332] Server 10.2: Add support for OpenSSL 1.1 Created: 2016-07-04 Updated: 2019-03-29 Resolved: 2018-03-19
Status:	Closed
Project:	MariaDB Server
Component/s:	Compiling
Fix Version/s:	10.2.6

[MDEV-10332] Server 10.2: Add support for OpenSSL 1.1 Created: 2016-07-04 Updated: 2019-03-29 Resolved: 2018-03-19