"vanilla" ssl connections work, but assigning a cert to the server and a client cert causes "ssl protocol failures", sometimes obscure version mismatch errors.
On investigation it turns out the that problem is in viossl.c and how ssl errors are handled.
ssl_should_retry calls SSL_get_error to check for the current error, but the documentation for SSL_get_error says:
"The current thread's
error queue must be empty before the TLS/SSL I/O operation is
attempted, or SSL_get_error() will not work reliably."
(attaching segment of mysqld.trace of the working, patched version...note how X509_R_CERT_ALREADY_IN_HASH_TABLE
errors get dropped by the patched code; otherwise they would cause a fatal error in the SSL connection, even though they are harmless, coming from the X509 certs being taken from both the server and the client)
It took a couple of iterations to clean up these errors, because there was some race conditions on when the errors came in vs. when they get handled in the code.
Solution: add ERR_clear_error() before each SSL_(read|write|etc) calls. And check for whatever errors are in the "queue" in the ssl_should_retry routine. The mysqld.trace shows that sometimes several errors are queued up, so just dismissing the first one won't be enough.
Patch attached; it has some extra whitespace changes, and extra DBUG_PRINT's for diagnostics
NOTE: v10.6 viossl.c has the same code in the 10.2 viossl.c, with minor changes that do not alter the problem noted in this bugreport