I did not remember there is -DWITH_SSL=path , but it seems to be own invention for something that is already available in standard CMake.

The official way to tell cmake the directory of custom OpenSSL is -DOPENSSL_ROOT_DIR, and this is what I used, together with WITH_SSL=system (to trigger FIND_PACKAGE(OpenSSL)).

The variable is listed under HINTS in CMake documentation in https://cmake.org/cmake/help/v3.22/module/FindOpenSSL.html

Here is how I used it https://github.com/MariaDB/server/commit/2cc5fa65d34a566e7c1a046d5f61c95b357664d2

Please see the description:

Same happens with

cmake . -DWITH_SSL=/home/elenst/openssl3-inst -DSSL_PATH=/home/elenst/openssl3-inst/ -DOPENSSL_ROOT_DIR=/home/elenst/openssl3-inst -DOPENSSL_INCLUDE_DIR=/home/elenst/openssl3-inst/include/

And yes, same also happens with -DOPENSSL_ROOT_DIR -DWITH_SSL=system, without other parameters:

-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "3.0.1")  

-- OPENSSL_INCLUDE_DIR = /home/elenst/openssl3-inst/include

-- OPENSSL_SSL_LIBRARY = /usr/lib/x86_64-linux-gnu/libssl.so

-- OPENSSL_CRYPTO_LIBRARY = /usr/lib/x86_64-linux-gnu/libcrypto.so

-- OPENSSL_VERSION = 3.0.1

-- SSL_LIBRARIES = /usr/lib/x86_64-linux-gnu/libssl.so;/usr/lib/x86_64-linux-gnu/libcrypto.so;dl

If you mean Windows, I have no information about any mixup there. My Windows doesn't have alternative openssl installations.

I did not mean Windows , but since you mentioned it, I can report that, that there exists Appveyor environment, accessible to general public, with 3 versions of openssl 1.1.0, 1.1.1 and 3.0 installed side-by-side (s. https://www.appveyor.com/docs/windows-images-software/)

There, -DWITH_SSL=system -DOPENSSL_ROOT_DIR=C:\path\specific\to\openssl-version alone does help.

• build with 3.0 https://ci.appveyor.com/project/rasmushoj/server/builds/42231228
• build with 1.1.1 https://ci.appveyor.com/project/rasmushoj/server/builds/42255045

I did not mean Windows

So, it worked for you on Linux, then? Which Linux, which cmake? I had the above with at least cmake 3.18.4, maybe others but I don't have other versions handy. If you are saying that it works somewhere, I can check.

> My Windows doesn't have alternative openssl installations.

since you mentioned it, I can report that, that there exists Appveyor environment, accessible to general public, with 3 versions of openssl 1.1.0, 1.1.1 and 3.0

I didn't mean custom installations, I meant having a "default" one while trying to build with non-default, as will be very common on Linuxes for the next many months at least. Getting rid of the default installation of openSSL libraries on Linux seems difficult, at least I couldn't manage it without removing what seems to be a half of the system.
For custom ones, sure, I have several of those on my Windows/Linux machines as well. But naturally installations in non-default locations are extremely unlikely to interfere since cmake wouldn't know anything about them.
I don't know if there is a "default" location for Windows which cmake can try to search. Given that FindOpenSSL.cmake is partially split into separate "Windows/Linux" paths, it wouldn't be surprising if it worked very differently on Linux vs Windows at the end.

On Linux, I did not ever test mix of default and custom openssl. My reports about OPENSSL_ROOT_DIR working are based only on by primary build environment. On Linux, IIRC, I uninstalled default, and "sudo make installed" the 3.0 to get it working.

it's a cmake bug, fixed in 3.23.0. We'll have to implement a workaround.

pushed in a preview branch