[MXS-1553] GaleraMon ignores server's SSL configuration Created: 2017-11-30  Updated: 2020-08-25  Resolved: 2017-12-04

Status: Closed
Project: MariaDB MaxScale
Component/s: galeramon
Affects Version/s: 2.0.5, 2.1.8
Fix Version/s: 2.1.12

Type: Bug Priority: Major
Reporter: Geoff Montee (Inactive) Assignee: markus makela
Resolution: Fixed Votes: 1
Labels: galeramon, ssl

Issue Links:
Relates
relates to MXS-1463 Would like Galera Monitor to support TLS Closed

 Description   

If a server's configuration section has "ssl=required" set, GaleraMon will still happily connect to the backend node without using SSL. The easiest way to reproduce this is by pointing MaxScale to backend servers that do not support SSL, and setting "ssl=required" anyway. GaleraMon will connect without SSL, but if any user queries are routed to the backend nodes, the user's connection will fail with SSL errors.

For example, let's say that we have the following configuration:

[maxscale]
threads=4
syslog=1
maxlog=1
#log_to_shm=1
log_warning=1
log_notice=1
 
[C1N1]
type=server
address=172.30.0.249
port=3306
protocol=MySQLBackend
persistpoolmax=100
persistmaxtime=601
ssl=required
ssl_cert=/var/lib/maxscale/certificates/client-cert.pem
ssl_key=/var/lib/maxscale/certificates/client-key.pem
ssl_ca_cert=/var/lib/maxscale/certificates/ca.pem
 
[C1N2]
type=server
address=172.30.0.32
port=3306
protocol=MySQLBackend
persistpoolmax=100
persistmaxtime=601
ssl=required
ssl_cert=/var/lib/maxscale/certificates/client-cert.pem
ssl_key=/var/lib/maxscale/certificates/client-key.pem
ssl_ca_cert=/var/lib/maxscale/certificates/ca.pem
 
[C1N3]
type=server
address=172.30.0.46
port=3306
protocol=MySQLBackend
persistpoolmax=100
persistmaxtime=601
ssl=required
ssl_cert=/var/lib/maxscale/certificates/client-cert.pem
ssl_key=/var/lib/maxscale/certificates/client-key.pem
ssl_ca_cert=/var/lib/maxscale/certificates/ca.pem
 
[Galera Monitor]
type=monitor
module=galeramon
servers=C1N1,
        C1N2,
        C1N3
user=maxscale
passwd=password
monitor_interval=10000
 
[Read Listener]
type=listener
service=Splitter Service
protocol=MySQLClient
port=3306
 
[Splitter Service]
type=service
router=readwritesplit
servers=C1N1,
        C1N2,
        C1N3
user=maxscale
passwd=password
max_slave_connections=100%
 
[MaxAdmin Service]
type=service
router=cli

In the above configuration, only one of my servers (C1N1) is configured with SSL certificates. The others do not have SSL configured. Since all servers have "ssl=required" set, you would expect the monitor to fail to connect to C1N2 and C1N3. In reality, MaxScale says that all nodes are available:

log:

2017-11-30 16:29:29   notice : Server changed state: C1N1[172.30.0.249:3306]: new_slave. [Running] -> [Slave, Synced, Running]
2017-11-30 16:29:29   notice : Server changed state: C1N2[172.30.0.32:3306]: new_master. [Running] -> [Master, Synced, Running]
2017-11-30 16:29:29   notice : Server changed state: C1N3[172.30.0.46:3306]: new_slave. [Running] -> [Slave, Synced, Running]

maxadmin's "show servers":

MaxScale> show servers
Server 0x1fe8fa0 (C1N1)
        Server:                              172.30.0.249
        Status:                              Slave, Synced, Running
        Protocol:                            MySQLBackend
        Port:                                3306
        Server Version:                      10.1.26-MariaDB
        Node Id:                             2
        Master Id:                           -1
        Slave Ids:
        Repl Depth:                          0
        Number of connections:               0
        Current no. of conns:                0
        Current no. of operations:           0
        Persistent pool size:                0
        Persistent measured pool size:       0
        Persistent actual size max:          0
        Persistent pool size limit:          100
        Persistent max time (secs):          601
        Connections taken from pool:         0
        Pool availability:                   0.00%
        SSL initialized:                     yes
        SSL method type:                     MAX
        SSL certificate verification depth:  9
        SSL certificate:                     /var/lib/maxscale/certificates/client-cert.pem
        SSL key:                             /var/lib/maxscale/certificates/client-key.pem
        SSL CA certificate:                  /var/lib/maxscale/certificates/ca.pem
Server 0x1fe3920 (C1N2)
        Server:                              172.30.0.32
        Status:                              Master, Synced, Running
        Protocol:                            MySQLBackend
        Port:                                3306
        Server Version:                      10.1.26-MariaDB
        Node Id:                             0
        Master Id:                           -1
        Slave Ids:
        Repl Depth:                          0
        Number of connections:               0
        Current no. of conns:                0
        Current no. of operations:           0
        Persistent pool size:                0
        Persistent measured pool size:       0
        Persistent actual size max:          0
        Persistent pool size limit:          100
        Persistent max time (secs):          601
        Connections taken from pool:         0
        Pool availability:                   0.00%
        SSL initialized:                     yes
        SSL method type:                     MAX
        SSL certificate verification depth:  9
        SSL certificate:                     /var/lib/maxscale/certificates/client-cert.pem
        SSL key:                             /var/lib/maxscale/certificates/client-key.pem
        SSL CA certificate:                  /var/lib/maxscale/certificates/ca.pem
Server 0x1fdbfa0 (C1N3)
        Server:                              172.30.0.46
        Status:                              Slave, Synced, Running
        Protocol:                            MySQLBackend
        Port:                                3306
        Server Version:                      10.1.26-MariaDB
        Node Id:                             1
        Master Id:                           -1
        Slave Ids:
        Repl Depth:                          0
        Number of connections:               0
        Current no. of conns:                0
        Current no. of operations:           0
        Persistent pool size:                0
        Persistent measured pool size:       0
        Persistent actual size max:          0
        Persistent pool size limit:          100
        Persistent max time (secs):          601
        Connections taken from pool:         0
        Pool availability:                   0.00%
        SSL initialized:                     yes
        SSL method type:                     MAX
        SSL certificate verification depth:  9
        SSL certificate:                     /var/lib/maxscale/certificates/client-cert.pem
        SSL key:                             /var/lib/maxscale/certificates/client-key.pem
        SSL CA certificate:                  /var/lib/maxscale/certificates/ca.pem

In the above output, we can see that all servers say "SSL initialized: yes", even though 2 of the servers are not configured with SSL certificates. If GaleraMon had connected to these servers using SSL, it would have encountered an error and realized this.

When a user connection is routed to one of these nodes, it will encounter an error:

MySQL [(none)]> SELECT 1;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    2246
Current database: *** NONE ***
 
ERROR 2013 (HY000): Lost connection to MySQL server during query

log:

2017-11-30 16:35:08   error  : (4) SSL operation failed in dcb_connect_SSL, dcb 0x7efda0013e30 in state DCB_STATE_POLLING fd 17 return code -1. More details may follow.
2017-11-30 16:35:08   error  : (4) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
2017-11-30 16:35:08   error  : (4) SSL operation failed in dcb_connect_SSL, dcb 0x7efda0014570 in state DCB_STATE_POLLING fd 19 return code -1. More details may follow.
2017-11-30 16:35:08   error  : (4) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
2017-11-30 16:35:10   error  : (5) SSL operation failed in dcb_connect_SSL, dcb 0x2020f00 in state DCB_STATE_POLLING fd 19 return code -1. More details may follow.
2017-11-30 16:35:10   error  : (5) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
2017-11-30 16:35:10   error  : (5) SSL operation failed in dcb_connect_SSL, dcb 0x203a830 in state DCB_STATE_POLLING fd 17 return code -1. More details may follow.
2017-11-30 16:35:10   error  : (5) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

Note that even after the failed connection attempts, MaxScale still thinks these nodes are available to use:

MaxScale> list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
C1N1               | 172.30.0.249    |  3306 |           0 | Slave, Synced, Running
C1N2               | 172.30.0.32     |  3306 |           0 | Master, Synced, Running
C1N3               | 172.30.0.46     |  3306 |           0 | Slave, Synced, Running
-------------------+-----------------+-------+-------------+--------------------



 Comments   
Comment by markus makela [ 2017-11-30 ]

As MaxScale uses Connector-C to connect to the servers, this can be classified either as a Connector-C bug, where TLS is not used even if the client implies it, or as MaxScale not doing the TLS checks. As the server is less strict with TLS related requirements (USE SSL must be defined for the user), I believe the monitor could check if SHOW STATUS LIKE 'Ssl_version' actually returns something.

Comment by markus makela [ 2017-12-04 ]

As the connector exposes whether TLS was enabled for the connection (via the active cipher), it is simpler to use that to check that TLS is in use.

Generated at Thu Feb 08 04:07:38 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.