Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23740

ssl connection fails when server and client certs signed by same CA

    XMLWordPrintable

    Details

      Description

      "vanilla" ssl connections work, but assigning a cert to the server and a client cert causes "ssl protocol failures", sometimes obscure version mismatch errors.

      On investigation it turns out the that problem is in viossl.c and how ssl errors are handled.
      ssl_should_retry calls SSL_get_error to check for the current error, but the documentation for SSL_get_error says:
      "The current thread's
      error queue must be empty before the TLS/SSL I/O operation is
      attempted, or SSL_get_error() will not work reliably."

      (attaching segment of mysqld.trace of the working, patched version...note how X509_R_CERT_ALREADY_IN_HASH_TABLE
      errors get dropped by the patched code; otherwise they would cause a fatal error in the SSL connection, even though they are harmless, coming from the X509 certs being taken from both the server and the client)

      It took a couple of iterations to clean up these errors, because there was some race conditions on when the errors came in vs. when they get handled in the code.

      Solution: add ERR_clear_error() before each SSL_(read|write|etc) calls. And check for whatever errors are in the "queue" in the ssl_should_retry routine. The mysqld.trace shows that sometimes several errors are queued up, so just dismissing the first one won't be enough.

      Patch attached; it has some extra whitespace changes, and extra DBUG_PRINT's for diagnostics

      NOTE: v10.6 viossl.c has the same code in the 10.2 viossl.c, with minor changes that do not alter the problem noted in this bugreport

        Attachments

        1. MariaDB.patch
          3 kB
        2. mariaSSL.tgz
          9 kB
        3. mysqld.trace
          2 kB

          Issue Links

            Activity

              People

              Assignee:
              wlad Vladislav Vaintroub
              Reporter:
              celane Charles Lane
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: