[MDEV-25701] Two-way TLS does not work with WolfSSL and version1 certificates Created: 2021-05-17  Updated: 2023-12-15

Status: Confirmed
Project: MariaDB Server
Component/s: Documentation
Fix Version/s: N/A

Type: Task Priority: Major
Reporter: Hartmut Holzgraefe Assignee: Joe Cotellese
Resolution: Unresolved Votes: 1
Labels: None

Attachments: File certificates.tar.gz     File generate-certs-2.sh     File linux2windows.pcapng     File windows2linux.pcapng     File windows2windows.pcapng    
Issue Links:
Issue split
split to MDEV-25798 Windows SChannel clients fail to conn... Closed
PartOf
Relates
relates to CONC-538 Can't connect via SSL Open
relates to MDEV-25799 tls_version=TLSv1.3 does not work wit... Closed

 Description   

I generated certificates according to the instructions in the KnowledgeBase:

https://mariadb.com/kb/en/certificate-creation-with-openssl/

I created a ca-cert, a server and a client certificate

and set up the server using:

[mysqld]
bind-address=0.0.0.0
ssl
ssl-ca=/etc/mysql/ssl/ca-cert.pem
ssl-cert=/etc/mysql/ssl/server-cert.pem
ssl-key=/etc/mysql/ssl/server-key.pem

In the following snippets 192.168.23.15 is my linux server, 192.168.23.66 a Windows10 machine. The database user name I use is "secure".

I can connect from linux clients using the client certificate just fine, e.g.:

mysql -h 192.168.23.15 -p -u secure --ssl-ca=ssl/ca-cert.pem --ssl-key=ssl/client-key.pem --ssl-cert=ssl/client-cert.pem
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
...

and I can verify that encryption is indeed used:

Trying to connect from the Windows machine I can't connect though:

mysql -u secure -p -h 192.168.23.15 --ssl-ca=\ssl\ca-cert.pem --ssl-key=\ssl\client-key.pem --ssl-cert=\ssl\client-cert.pem
Enter password: ******
ERROR 2026 (HY000): SSL connection error: no cipher match. Error 0x80090331(SEC_E_ALGORITHM_MISMATCH)

I CAN connect from windows to linux using encryption when just giving the --ssl option without explicit client certificate:

~ > mysql -h 192.168.23.66 -p -u secure --ssl
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 7
Server version: 10.5.10-MariaDB mariadb.org binary distribution
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
(none) 10.5.10-MariaDB
> show status like 'ssl_version';
+---------------+---------+
| Variable_name | Value   |
+---------------+---------+
| Ssl_version   | TLSv1.2 |
+---------------+---------+
1 row in set (0.006 sec)
 
(none) 10.5.10-MariaDB
> show status like 'ssl_cipher';
+---------------+--------------------+
| Variable_name | Value              |
+---------------+--------------------+
| Ssl_cipher    | DHE-RSA-AES256-SHA |
+---------------+--------------------+
1 row in set (0.016 sec)

When setting up MariaDB server on the windows size with the same server certificate as on Linux, and trying to connect locally using the client certificate I'm getting:

C:\Program Files\MariaDB 10.5\data>mysql -h 192.168.23.66 -u secure -p --ssl --ssl-ca=\ssl\ca-cert.pem --ssl-key=\ssl\client-key.pem --ssl-cert=\ssl\client-cert.pem
Enter password: ******
ERROR 2026 (HY000): SSL connection error: An unknown error occurred while processing the certificate. Error 0x80090327(SEC_E_CERT_UNKNOWN)

And when trying to connect to that windows server from linux (only server IP changed from .15 to .66 after verifying that things work locally):

~ > mysql -h 192.168.23.66 -p -u secure --ssl-ca=/etc/mysql/ssl/ca-cert.pem --ssl-key=/etc/mysql/ssl/client-key.pem --ssl-cert=/etc/mysql/ssl/client-cert.pem --Enter password: 
ERROR 2026 (HY000): SSL connection error: sslv3 alert bad certificate

Again, when just using the --ssl option without explicit client certificate the connection is established using TLS v1.2 just fine.

Also when using MySQL 5.7 on the windows side, which uses OpenSSL instead of WolfSSL, things work just fine both ways with explicit client certificate.

So there seems to be some problem with WolfSSL and certificate verification, at least when the certificates were created using OpenSSL as described in our knowledge base.



 Comments   
Comment by Hartmut Holzgraefe [ 2021-05-17 ]

I also tested with 10.3 instead of 10.5 on both sides, and while some error messages are not exactly the same, the basic problem remains the same

Comment by Vladislav Vaintroub [ 2021-05-23 ]

Did you notice, things work "just fine" only with OpenSSL, neither Schannel (Windows client) nor WolfSSL (Windows server) like that certificate. Since 2 independent SSL libraries think there is some problem with it, maybe there is some problem with it?

Comment by Vladislav Vaintroub [ 2021-05-25 ]

it is clearly possible to generate cerficates with OpenSSL, and use them with all other implementations
Apparently, those certificates are used in our test suite, and are tested with every build, on every platform

https://github.com/MariaDB/server/blob/10.6/mysql-test/lib/generate-ssl-certs.sh - this is how they are generated. I did not check whether KB description works, because I do not usually have a platform where openssl is installed. I suppose it does not even on Linux, once you use tar.gz packages, since they are WolfSSL-based.

Comment by Vladislav Vaintroub [ 2021-05-25 ]

Forwarding to Documentation.
The summary is that the steps as described in the KB https://mariadb.com/kb/en/certificate-creation-with-openssl/ does work only for OpenSSL, but neither for WolfSSL, nor Schannel.
Note that the steps in https://github.com/MariaDB/server/blob/10.6/mysql-test/lib/generate-ssl-certs.sh work for everything.

Comment by Hartmut Holzgraefe [ 2021-05-27 ]

"it is clearly possible to generate cerficates with OpenSSL, and use them with all other implementations
Apparently, those certificates are used in our test suite, and are tested with every build, on every platform

https://github.com/MariaDB/server/blob/10.6/mysql-test/lib/generate-ssl-certs.sh - this is how they are generated."

that solves the Linux->Windows and Windows->Windows problems, by generating v3 certificates instead of v1 ones, which WolfSSL requires for TLS >= v1.2.

So the

ERROR 2026 (HY000): SSL connection error: An unknown error occurred while processing the certificate. Error 0x80090327(SEC_E_CERT_UNKNOWN)

and

ERROR 2026 (HY000): SSL connection error: sslv3 alert bad certificate

Problems reported above are gone when using cacert, server-cert and client-cert from mysql-test/std-data.

But the "No matching cipher" problem when trying to connect from Windows/WolfSSL builds to Linux/OpenSSL is unfortunately still there.

Now checking if the same is true between Linux/OpenSSL and Linux/WolfSSL builds, too ...

Comment by Vladislav Vaintroub [ 2021-05-27 ]

hholzgra, can you upload certiicates that demonstrate "no matching cipher" only, and have anything else fixed?

Comment by Hartmut Holzgraefe [ 2021-05-27 ]

So the "no matching cipher" problem is specific to SChannel based clients on Windows, it is reproducible with both v1 and v3 certificates. Splitting that out to a new MDEV soon ...

Comment by Hartmut Holzgraefe [ 2021-05-27 ]

I now changed the ticket title to refer to the WolfSSL / certificate version problem only.

Comment by Hartmut Holzgraefe [ 2021-05-27 ]

"No matching cipher" part split out to new MDEV-25798

Comment by chylex [ 2023-04-21 ]

Hi, I did some research and was able to generate V3 certificates with:

openssl x509 -req -days 365000 -set_serial 1 -CA ca_cert.pem -CAkey ca_key.pem -in "server_req.pem" -out "server_cert.pem" -extfile /etc/ssl/openssl.cnf -extensions usr_cert

It's very similar to the command in the original documentation. Please try if it fixes the issue, it would be great if the documentation could be updated and this issue finally resolved.

When setting up SSL for my database, I found a few more things I think could improve the documentation, for ex.:

  • It might be a good idea to start recommending something better than 2048-bit RSA, I ended up using Ed25519 based on recommendations from security experts and performance-wise it's very close
  • I found that the documentation step to remove the passphrase from server certificate is redundant, because the previous step already used "-nodes" so there's no passphrase to begin with
  • I found it difficult to understand what the commands actually did, it would be great if the documentation could provide guidance on how to fill out the certificate subject, serial numbers, etc.

Let me know if there's interest in opening a new discussion about these things. I ended up writing a blog post that addresses all the issues I found, but I'd be happy to contribute my findings to the official documentation to make setting up SSL easier for others.

Generated at Thu Feb 08 09:39:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.