Details
-
Task
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Fixed
Description
If the client provides --ssl-* options, but a TLS connection to the server is not possible, then the client simply silently ignores the provided --ssl-* options if the --ssl-verify-server-cert option is not also provided. The user may think that their connection is encrypted, but it is not.
The current behavior is probably the "expected" behavior according to Engineering. See MDEV-16409 for some details on previous discussion. However, I don't think this behavior is the behavior that would be expected by most users. When implementing security features, the industry standard for design is to fail safe. Security features may be mandatory for compliance reasons, and the fault of a security control may silently increase risk.
Consider an example: if you typed https://mybank.com into your browser's URL bar, would you be happy if the browser silently used an unencrypted connection, even though you requested HTTPS? This is basically what MariaDB's clients are doing.
For example, let's say that TLS is not enabled on our server:
$ sudo mariadb --execute="SHOW GLOBAL VARIABLES LIKE 'ssl%'"
|
+---------------+-------+
|
| Variable_name | Value |
|
+---------------+-------+
|
| ssl_ca | |
|
| ssl_capath | |
|
| ssl_cert | |
|
| ssl_cipher | |
|
| ssl_crl | |
|
| ssl_crlpath | |
|
| ssl_key | |
|
+---------------+-------+
|
If a client tries to connect via TLS, then they will not see an error, and their connection will be unencrypted:
$ mariadb --user=myuser --password=password --host=127.0.0.1 --ssl --ssl-ca=/home/vagrant/ca.pem --execute="SHOW SESSION STATUS LIKE 'Ssl_cipher'"
|
+---------------+-------+
|
| Variable_name | Value |
|
+---------------+-------+
|
| Ssl_cipher | |
|
+---------------+-------+
|
The client will only see an error if they also provide the --ssl-verify-server-cert option:
$ mariadb --user=myuser --password=password --host=127.0.0.1 --ssl --ssl-ca=/home/vagrant/ca.pem --ssl-verify-server-cert --execute="SHOW SESSION STATUS LIKE 'Ssl_cipher'"
|
ERROR 2026 (HY000): SSL connection error: SSL is required, but the server does not support it
|
MySQL fixed this problem by introducing the --ssl-mode client option in MySQL 5.7.11 that provides more configurable TLS behavior.
https://dev.mysql.com/doc/refman/5.7/en/connection-options.html#option_general_ssl-mode
Part of the new MySQL behavior made it so that if the client specifies the --ssl-ca or --ssl-capath option, then that will imply --ssl-mode=VERIFY_CA. by default.
The --ssl-mode option interacts with CA certificate options as follows:
- If --ssl-mode is not explicitly set otherwise, use of --ssl-ca or --ssl-capath implies --ssl-mode=VERIFY_CA.
- For --ssl-mode values of VERIFY_CA or VERIFY_IDENTITY, --ssl-ca or --ssl-capath is also required, to supply a CA certificate that matches the one used by the server.
- An explicit --ssl-mode option with a value other than VERIFY_CA or VERIFY_IDENTITY, together with an explicit --ssl-ca or --ssl-capath option, produces a warning that no verification of the server certificate will be done, despite a CA certificate option being specified.
In my opinion, we should take one of the following actions:
- Throw a warning if the client provides --ssl-* options, but a TLS connection to the server is not possible.
- Or make the --ssl-ca and --ssl-capath options imply the --ssl-verify-server-cert option by default (similar to MySQL 5.7+ behavior).
- Or port MySQL's --ssl-mode behavior.
Attachments
Issue Links
- is blocked by
-
MDEV-31857 enable --ssl-verify-server-cert by default
-
- Closed
-
- relates to
-
CONC-656 TLS-using MariaDB clients tell MITM attackers if they can be undetectably MITM'ed
-
- Closed
-
-
-
- Open
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
- links to
Activity
Still, from a user point of view — they had MariaDB working, ssl enabled, so "everything was secure".
But from a user point of view, all of that was clearly not true.
- "ssl enabled": No. Even though they were specifying "I want to use SSL" in the client, they were not getting SSL.
- "they had MariaDB working": Do you consider "all of my database traffic is flowing inplaintext when I wanted it to be secured with SSL" to count as "working"?
Only those few that have proper (not self-signed) certs, so most likely already use --ssl-verify-server-cert, only those will keep working.
I'd say that is the prudent and correct approach for MariaDB to take.
Every MariaDB client→server connection not using server-cert-verification is already broken: even if they think they're "using SSL", their connections can be trivially MITM'ed (or just downgraded to unencrypted ones) by their ISP, cloud provider, 3-letter intelligence agency, etc.
And note that after 10.10 (
MDEV-27105) ssl is enabled [in the client] by default.
Yes, this is a small step in the right direction, but …
- This change applies to the mariadb command-line client, not to the actual Connector/C or Connector/J libraries.
- The server has not changed to REQUIRE_SECURE_TRANSPORT=ON by default
- Server cert validation is still not enabled by default
- And most importantly, the vulnerability described in this JIRA right here still exists: the command-line client will still allow itself to be downgraded silently to a non-SSL connection
… so this small change towards opportunistic-SSL-by-default is largely ineffectual. It won't protect against the kind of pervasive attackers that everyone has incorporated into their threat model at least since 2013.
Just to make it clear, I'm not saying we should not do it at all. I'm saying, we need to figure out how to do it without breaking everything.
Keep in mind that almost all other client/server software that purports to use TLS has consistently made the opposite choice for at least 1 decade: opt for actual security, rather than backwards-compatibility, when a vulnerability is found. MariaDB appears to be far behind the curve in this regard.
Why not simply put out very clear documentation with an upcoming MariaDB release that there is going to be substantial backwards-incompatible changes in MariaDB server and client libraries in order to catch up with longstanding best practices for TLS? That documentation can include explaining about how to revert to the old behavior with the new server/client versions, as well as the risks of doing so.
What about this approach:
- --ssl will print the message "Deprecated, don't use it" and will do nothing (indeed, as ssl is enabled by default anyway)
- --disable-ssl will still work as before
- other ssl options, like --ssl-ca, --tls-version, --tls-fp, etc — they all will require ssl and the connection will fail if ssl cannot be established.
This should cover most use cases:
- the server has no certificates configured (almost all installations) — client will try ssl, fail, continue unencrypted as before.
- the server has proper certificates (very very uncommon) — the client likely uses --ssl-verify-server-cert already.
- the server has self-signed certificates (most servers that use ssl) — the client needs --ssl-ca or --tls-fp to be able to validate the server, makes sense to require ssl in that case.
Basically, this solution relies on the fact that one needs either --ssl-ca or --tls-fp or --ssl-verify-server-cert to make sure the connection is secure. So in all those cases we'll refuse to connect without ssl. Simple --ssl does not guarantee anything, it means the user doesn't care, so we won't require ssl in that case, but we'll issue a warning, telling the user that he should care and how to enable ssl correctly.
Hi serg,
In my opinion, that plan sounds good. It looks like your proposed implementation covers the same use cases as MySQL's --ssl-mode, but your idea does it by preserving backwards compatibility:
- --ssl-mode=PREFERRED (default): Your idea seems to have equivalent default functionality without using a new option.
- --ssl-mode=VERIFY_CA: To require TLS with certificate verification, your idea reuses options like --ssl-ca, which is needed for certificate verification anyway, and it sounds like the behavior is backward-compatible.
- --ssl-mode=VERIFY_IDENTITY: To require TLS with certificate verification and host verification, your idea reuses the existing --ssl-verify-server-cert, and it sounds like the behavior is backward-compatible.
- --ssl-mode=REQUIRED: To require TLS without requiring either certificate verification or host verification, your idea reuses existing options, and it sounds like the behavior is backward-compatible.
- --ssl-mode=DISABLED: To disable TLS, your idea reuses the existing --disable-ssl, and it sounds like the behavior is backward-compatible.
The idea to add a deprecation message for --ssl also sounds good to me.
A potential downside is that MySQL's implementation offers more flexibility in some cases. For example, let's say that a DBA has many servers that require TLS with certificate verification, but they also have a handful of other servers in which TLS is not enabled for whatever reason.
The DBA might want to add their certificates and keys to their configuration file to save them typing:
[client-mariadb]
|
ssl-ca=/home/me/tls/ca.pem
|
ssl-key=/home/me/tls/key.pem
|
ssl-cert=/home/me/tls/cert.pem
|
With MySQL 8.0, it sounds to me like the DBA would be able to do this *and* still connect to all of the servers: those that require TLS with certificate verification and also those that don't have TLS enabled, since MySQL 8.0's default mode is --ssl-mode=PREFERRED and since the other --ssl* options don't implicitly change the mode anymore in MySQL 8.0. Although, in MySQL 5.7, the other --ssl* options do implicitly change the mode according to the MySQL 5.7 documentation.
In contrast, with your proposed implementation, it sounds to me like the DBA would not be able to connect to the servers that don't have TLS enabled, because having the --ssl-ca option in the configuration file would implicitly set behavior equivalent to --ssl-mode=VERIFY_CA. When the DBA tried to connect, the client would require TLS with certificate verification.
We could solve this case by treating --ssl* options in configuration files differently than they are treated on the command-line. However, that might break some backward-compatibility.
Either way, this case might not be important. Your idea is still a good one.
Thanks!
This became obsolete by MDEV-31855 and MDEV-31857.
Within MDEV-31855 fingerprint checks become non-optional, if the fingerprint is specified, the connection will be aborted unless the certificate has the specified fingerprint.
Within MDEV-31857 --ssl-verify-server-cert got to be enabled by default. --ssl was already enabled by default in MDEV-27105. So, now all clients require SSL by default and verify the certificate to ensure the connection is secure.
dlenski, I generally agree with you. --ssl on itself doesn't guarantee anything without --ssl-verify-server-cert, it's 2023, and so on.
Still, from a user point of view — they had MariaDB working, ssl enabled, so "everything was secure". Then after an upgrade the application is down, clients cannot connect anymore. Who broke user application (mission critical, of course) — we did.
And note that after 10.10 (
MDEV-27105) ssl is enabled by default. So forcing server cert verification we'll break everything, every single installation that doesn't have ssl certs, every single installation that has self-signed certs. Only those few that have proper (not self-signed) certs, so most likely already use --ssl-verify-server-cert, only those will keep working.I know that you're right and ssl should be enforced and certs should be validated. But the transition from what it is now to what it should be comes with such an immense harm — I don't think we can just go and break pretty much every​*) single mariadb installation there is.
Just to make it clear, I'm not saying we should not do it at all. I'm saying, we need to figure out how to do it without breaking everything.
—
*) statistically