Question about 10.5.8 server: Which tls library and version does the server use (show variables like 'version_ssl_library' will give you this information)?

I don't want to hijack this issue, but I have a very similar problem, except that I get:

SSL connection error: An unknown error occurred while processing the certificate. Error 0x80090327(SEC_E_CERT_UNKNOWN)

I'm trying to SSL connect to a 10.5.9 mariadb binary package on debian buster, which is using WolfSSL 4.6.0 internally.

My client is Windows 10 and I tested it with the native mariadb c connector 3.1.11 and did another test with the latest HeidiSQL client - same error.

Generation of the used certs was done on the db server and is pretty basic.
Verifying gives:

openssl verify -CAfile ca-cert.pem server-cert.pem client-cert.pem
server-cert.pem: OK
client-cert.pem: OK

Do you have any advice, cause connecting via SSL is really mandatory for the app.

Marco, this is different. If you think this is a bug, please file a new issue.

Note, that the error does not happen if one just tries to use SSL without the client certificate. It seems like there is something in the client certificate that server requires, and (according to schannel) client does not have. Also since there were no complaints about other Ubuntu versions, I'd expect something is off with openssl configuration on 20.04.

Please note also, that according to pcap file, the server sends CertificateRequest which includes 20 signature hash algorithms, but none of them matches SHA256 RSA,which is what client certificate has.

Frame 9: 2563 bytes on wire (20504 bits), 2563 bytes captured (20504 bits)

Ethernet II, Src: Microsof_86:a3:61 (00:15:5d:86:a3:61), Dst: Microsof_fc:6b:a8 (00:15:5d:fc:6b:a8)

Internet Protocol Version 4, Src: 172.24.147.163, Dst: 172.24.144.1

Transmission Control Protocol, Src Port: 3306, Dst Port: 58323, Seq: 114, Ack: 227, Len: 2509

Transport Layer Security

    TLSv1.2 Record Layer: Handshake Protocol: Server Hello

    TLSv1.2 Record Layer: Handshake Protocol: Certificate

    TLSv1.2 Record Layer: Handshake Protocol: Server Key Exchange

    TLSv1.2 Record Layer: Handshake Protocol: Certificate Request

        Content Type: Handshake (22)

        Version: TLS 1.2 (0x0303)

        Length: 52

        Handshake Protocol: Certificate Request

            Handshake Type: Certificate Request (13)

            Length: 48

            Certificate types count: 3

            Certificate types (3 types)

                Certificate type: RSA Sign (1)

                Certificate type: DSS Sign (2)

                Certificate type: ECDSA Sign (64)

            Signature Hash Algorithms Length: 40

            Signature Hash Algorithms (20 algorithms)

                Signature Algorithm: ecdsa_secp256r1_sha256 (0x0403)

                Signature Algorithm: ecdsa_secp384r1_sha384 (0x0503)

                Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603)

                Signature Algorithm: ed25519 (0x0807)

                Signature Algorithm: ed448 (0x0808)

                Signature Algorithm: rsa_pss_pss_sha256 (0x0809)

                Signature Algorithm: rsa_pss_pss_sha384 (0x080a)

                Signature Algorithm: rsa_pss_pss_sha512 (0x080b)

                Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)

                Signature Algorithm: rsa_pss_rsae_sha384 (0x0805)

                Signature Algorithm: rsa_pss_rsae_sha512 (0x0806)

                Signature Algorithm: rsa_pkcs1_sha256 (0x0401)

                Signature Algorithm: rsa_pkcs1_sha384 (0x0501)

                Signature Algorithm: rsa_pkcs1_sha512 (0x0601)

                Signature Algorithm: SHA224 ECDSA (0x0303)

                Signature Algorithm: SHA224 RSA (0x0301)

                Signature Algorithm: SHA224 DSA (0x0302)

                Signature Algorithm: SHA256 DSA (0x0402)

                Signature Algorithm: SHA384 DSA (0x0502)

                Signature Algorithm: SHA512 DSA (0x0602)

            Distinguished Names Length: 0

    TLSv1.2 Record Layer: Handshake Protocol: Server Hello Done

That's client certificate

The server only accepts SHA224 RSA, of all "sha rsa" signature algorithms family, but not SHA256 RSA.

There appears to be some kind of bug in openssl. According to google, Ubuntu made the decision to compile with non-default settings for openssl, to make it appear more secure.
https://github.com/Ensembl/ensembl-rest/issues/427 discusses one bug, but I'm not sure we're tripping over it. It looks like openssl itself is more lax about the strict requirements it imposes on client certificates, with SECLEVEL=2, than other SSL implementations are.

The workaround

Now, for all people who deal with that problem Ubuntu 20.04, this is one workaround

• use parameter ssl-cipher=DEFAULT:@SECLEVEL=1 for the server, or
• change /etc/ssl/openssl.cnf to add CipherString = DEFAULT:@SECLEVEL=1

There are some variations on CipherString workaround, a popular discussion in https://askubuntu.com/questions/1233186/ubuntu-20-04-how-to-set-lower-ssl-security-level.

This has something with the client certificate, and how it was created, but I frankly have no idea what that exactly can be.

I studied it a little bit more, and looked how other implementations may load private keys and certificates in PEM format and use them in Schannel TLS handshake

Particularly interesting info is in the dotnet bug https://github.com/dotnet/runtime/issues/23749
It turns out, that

One has to use persistent key (stored in the container), rather than ephemeral(in-memory only) , for schannel. This is because a AcquireCredentialsHandle() talks to another process (LSASS.exe), and it does not marshal private key in the communication. Which is definitely Windows issue, but it had been there forever, and until it is solved, if at all, workaround is to use named key containers. To link client certificate with private key, dotnet sets CERT_KEY_PROV_INFO_PROP_ID , which includes persistent key container name.

What .NET does, in many cases, but mostly prominently for self-signed SSL certificates, is to use unique name for key container, and then remove the whole container if it is no longer needed.
The AcquireCredentialsHandle behavior and LSASS is extremely sparsely document, in PFXImportCertStore documentation (of all things), when discussing ephemeral pfx flag PKCS12_NO_PERSIST_KEY.

So, ephemeral keys should not work, but they did so far for us in majority of cases.
Maybe it is a legacy behavior and current process is doing some parts of handshake them rather than LSASS. Ephemeral keys did not work then georg was trying TLSv1.3 . See MS Q&A discussion , where Gary Nebbett even makes the correct suggestion, but was not able to explain why persistent keys worked, and ephemeral did not.

I tried whatever .NET is doing, named key container, and CERT_KEY_PROV_INFO_PROP_ID, and with that, I can create a connection to OpenSSL-based server. Schannel will no longer be using sha1 for signature (this is what disturbed OpenSSL at security level !=0 ), but something more sophisticated, sha256 based.

So this should be the way to go, seems we'd need to implement what this commenter calls "perphemeral" private keys, i.e persistent, with cleanup, unless abnormal termination, or directory %AppData%\Roaming\Microsoft\Crypto\RSA will grow with persistent keys that nobody will use.

wlad, if a key can be automatically cleaned after the connection is established, then it should be very safe to delete all keys that are older than, say, 24 hours. Automatically, from the connector, why should the user know and do something about this?

the user does not need to do anything. We clean up everything after AcquireCredentialsHandle. If process is killed or crashes during this time, something will remain, but let's assume it won't happen often.

We should not be cleaning up keys that do not belong to our application, even if they reside in the same %AppData%\Roaming\Microsoft\Crypto\RSA directory. There is no documented way to find out from file name whether it belongs to our application or not (although at least , currently, one can run "strings" again a file, and the container name will show up in plaintext)

The Windows model for key provider is this - you load key once into container, then use container name instead of always reading the key file. We are not using it, but someone else might.

Right. I mean to clear up keys that belong to our application that were left after the process was killed or crashed. It is based on the assumption that we can somehow tell our keys from all others, e.g. by a specific naming pattern. If it is not possible — then we cannot do it.