[MXS-2499] MaxScale still authenticates clients locally when proxy_protocol=on Created: 2019-05-22  Updated: 2020-07-21  Resolved: 2020-07-21

Status: Closed
Project: MariaDB MaxScale
Component/s: Authenticator, Protocol
Affects Version/s: 2.3.7
Fix Version/s: 2.5.0

Type: New Feature Priority: Major
Reporter: Geoff Montee (Inactive) Assignee: Esa Korhonen
Resolution: Fixed Votes: 2
Labels: None

Issue Links:
Relates
relates to MXS-381 support for transparent proxy Closed
relates to MXS-2497 Support all MariaDBClient-compatible ... Closed

 Description   

I was told by the MaxScale team that MaxScale should skip authenticating clients locally when proxy_protocol=on is set for the backend servers.

The documentation doesn't seem to imply that this should be supported. The documentation seems to say that the proxy protocol is mostly intended to make user account management easier. Since the proxy protocol passes the original client IP address to the backend server in the proxy protocol header, the backend server isn't required to have user accounts for the specific user that resolve to both the client IP and the MaxScale IP. The user account just needs to resolve to the original client IP address.

https://mariadb.com/kb/en/mariadb-maxscale-23-mariadb-maxscale-configuration-usage-scenarios/#proxy_protocol

https://mariadb.com/kb/en/library/proxy-protocol-support/

To confirm whether MaxScale skips authenticating clients locally when proxy_protocol=on is set for the backend servers, I ran a test.

I ran the test with the following:

1 MaxScale server running MaxScale 2.3.7
3 backend Galera nodes running a mixture of MariaDB 10.3.14 and 10.3.15

On the backend Galera nodes, I set proxy_protocol_networks to the IP address of the MaxScale server:

MariaDB [(none)]> SET GLOBAL proxy_protocol_networks='172.30.0.106';
Query OK, 0 rows affected (0.000 sec)

On the MaxScale server, I used the following configuration:

[maxscale]
threads=4
syslog=1
maxlog=1
log_warning=1
log_notice=1
log_info=1
admin_host=127.0.0.1
admin_port=8989
admin_auth=1
admin_enabled=1
connector_plugindir=/usr/lib64/mysql/plugin/
 
[C1N1]
type=server
address=172.30.0.105
port=3306
protocol=MariaDBBackend
authenticator=PAMBackendAuth
proxy_protocol=on
 
[C1N2]
type=server
address=172.30.0.96
port=3306
protocol=MariaDBBackend
authenticator=PAMBackendAuth
proxy_protocol=on
 
[C1N3]
type=server
address=172.30.0.126
port=3306
protocol=MariaDBBackend
authenticator=PAMBackendAuth
proxy_protocol=on
 
[Galera-Monitor]
type=monitor
module=galeramon
servers=C1N1,
        C1N2,
        C1N3
user=maxscale
password=password
monitor_interval=10000
 
[Read-Listener]
type=listener
service=Splitter-Service
port=3306
protocol=MariaDBClient
authenticator=PAMAuth
 
[Splitter-Service]
type=service
router=readwritesplit
servers=C1N1,
        C1N2,
        C1N3
user=maxscale
password=password
max_slave_connections=100%

The specific user account involved uses PAM authentication, so I also needed to create the Unix user on the MaxScale instance, and all 3 backend nodes:

sudo adduser alice
sudo passwd alice

I also wanted to make sure that PAM authentication attempts are very thoroughly logged, so I followed these steps:

https://mariadb.com/kb/en/library/authentication-plugin-pam/#custom-logging-with-pam_exec

i.e. I created the following script:

tee /tmp/pam_log_script.sh <<EOF
#!/bin/bash
echo "\${PAM_SERVICE}:\${PAM_TYPE} - \${PAM_RUSER}@\${PAM_RHOST} is authenticating as \${PAM_USER}" 
EOF
chmod 0775 /tmp/pam_log_script.sh

And then used the following PAM service configuration in /etc/pam.d/mariadb:

auth optional pam_exec.so log=/tmp/pam_output.txt /tmp/pam_log_script.sh
auth required pam_unix.so audit
account optional pam_exec.so log=/tmp/pam_output.txt /tmp/pam_log_script.sh
account required pam_unix.so audit

And I created the following user on the backend nodes:

CREATE USER 'alice'@'%' IDENTIFIED VIA pam USING 'mariadb';
GRANT SELECT ON *.* TO 'alice'@'%';

Then I logged in through MaxScale:

$ mysql -u alice -h 127.0.0.1
[mariadb] Password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 10.3.14-MariaDB-log MariaDB Server
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MariaDB [(none)]> SELECT USER(), CURRENT_USER();
+-----------------+----------------+
| USER()          | CURRENT_USER() |
+-----------------+----------------+
| alice@localhost | alice@%        |
+-----------------+----------------+
1 row in set (0.001 sec)
 
MariaDB [(none)]> \q
Bye

The MaxScale log shows that the MaxScale server is properly sending the proxy protocol headers to the backend servers:

2019-05-21 21:55:56   info   : Found 1 valid PAM user entry for 'alice'@'::ffff:127.0.0.1'.
2019-05-21 21:55:56   info   : Servers and router connection counts:
2019-05-21 21:55:56   info   : current operations : 0 in        [172.30.0.105]:3306 RUNNING SLAVE
2019-05-21 21:55:56   info   : current operations : 0 in        [172.30.0.96]:3306 RUNNING MASTER
2019-05-21 21:55:56   info   : current operations : 0 in        [172.30.0.126]:3306 RUNNING SLAVE
2019-05-21 21:55:56   info   : Selected Master: C1N2
2019-05-21 21:55:56   info   : Selected Slave: C1N1
2019-05-21 21:55:56   info   : Selected Slave: C1N3
2019-05-21 21:55:56   info   : Started Splitter-Service client session [3] for 'alice' from ::ffff:127.0.0.1
2019-05-21 21:55:56   info   : (3) Sending proxy-protocol header 'PROXY TCP6 ::ffff:127.0.0.1 ::ffff:127.0.0.1 34473 3306^M
' to backend C1N3.
2019-05-21 21:55:56   info   : (3) Connected to 'C1N3' with thread id 21
2019-05-21 21:55:56   info   : (3) Sending proxy-protocol header 'PROXY TCP6 ::ffff:127.0.0.1 ::ffff:127.0.0.1 34473 3306^M
' to backend C1N1.
2019-05-21 21:55:56   info   : (3) Connected to 'C1N1' with thread id 28
2019-05-21 21:55:56   info   : (3) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 37, type: QUERY_TYPE_READ|QUERY_TYPE_SYSVAR_READ, stmt: select @@version_comment limit 1
2019-05-21 21:55:56   info   : (3) Route query to slave: C1N1   [172.30.0.105]:3306 <
2019-05-21 21:55:56   info   : (3) Sending proxy-protocol header 'PROXY TCP6 ::ffff:127.0.0.1 ::ffff:127.0.0.1 34473 3306^M
' to backend C1N2.
2019-05-21 21:55:56   info   : (3) Connected to 'C1N2' with thread id 21
2019-05-21 21:55:56   info   : (3) Reply complete, last reply from C1N1
2019-05-21 21:55:59   info   : (3) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 34, type: QUERY_TYPE_READ, stmt: SELECT USER(), CURRENT_USER()
2019-05-21 21:55:59   info   : (3) Route query to slave: C1N1   [172.30.0.105]:3306 <
2019-05-21 21:55:59   info   : (3) Reply complete, last reply from C1N1
2019-05-21 21:56:00   info   : (3) > Autocommit: [enabled], trx is [not open], cmd: (0x01) COM_QUIT, plen: 5, type: QUERY_TYPE_SESSION_WRITE, stmt:
2019-05-21 21:56:00   info   : (3) Session write, routing to all servers.
2019-05-21 21:56:00   info   : (3) Route query to slave: C1N1   [172.30.0.105]:3306
2019-05-21 21:56:00   info   : (3) Route query to master: C1N2  [172.30.0.96]:3306
2019-05-21 21:56:00   info   : (3) Route query to slave: C1N3   [172.30.0.126]:3306
2019-05-21 21:56:00   info   : Stopped Splitter-Service client session [3]

However, when I check /tmp/pam_output.txt on the MaxScale instance, it is clear that MaxScale's PAMAuth authenticator module is still authenticating the client locally as well:

*** Tue May 21 21:55:56 2019
mariadb:auth - @ is authenticating as alice
*** Tue May 21 21:55:56 2019
mariadb:account - @ is authenticating as alice

This makes sense to me, because I checked the source code, and I did not find any indication that setting proxy_protocol=on for a server gets rid of the local authentication step that MaxScale performs.

proxy_protocol isn't checked before calling gw_read_do_authentication:

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/protocol/MySQL/mariadbclient/mysql_client.cc#L544

And proxy_protocol isn't checked within gw_read_do_authentication:

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/protocol/MySQL/mariadbclient/mysql_client.cc#L705

And it still calls the authenticate() function for the authenticator module:

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/protocol/MySQL/mariadbclient/mysql_client.cc#L753

And the function that's called for PAMAuth doesn't seem to take any shortcuts if proxy_protocol is defined either:

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/authenticator/PAM/PAMAuth/pam_auth.cc#L169

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/authenticator/PAM/PAMAuth/pam_auth.cc#L104

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/authenticator/PAM/PAMAuth/pam_client_session.cc#L358

Is this working as-intended, or is MaxScale actually supposed to skip the local authentication step when the proxy protocol is used with the backend servers?



 Comments   
Comment by markus makela [ 2019-05-22 ]

Yes, MaxScale does seem to authenticate the user regardless of what the state of proxy_protocol is. Due to the fact that some servers might not have proxy_protocol on, a configuration option might be required on the service level. Another option is to add a service level proxy_protocol option that automatically enables it for all servers used by that service.

Comment by markus makela [ 2019-05-22 ]

Also, this can be worked around by using the default authenticator and `authenticator_options=skip_authentication=true`.

Comment by Geoff Montee (Inactive) [ 2019-05-22 ]

Thanks, markus makela.

Unfortunately, that workaround doesn't work either. Ironically, even though authentication is supposed to be skipped, PAM accounts fail to authenticate because MaxScale returns "User not found."

I tried the following configuration:

[maxscale]
threads=4
syslog=1
maxlog=1
log_warning=1
log_notice=1
log_info=1
admin_host=127.0.0.1
admin_port=8989
admin_auth=1
admin_enabled=1
connector_plugindir=/usr/lib64/mysql/plugin/
 
[C1N1]
type=server
address=172.30.0.105
port=3306
protocol=MariaDBBackend
proxy_protocol=on
 
[C1N2]
type=server
address=172.30.0.96
port=3306
protocol=MariaDBBackend
proxy_protocol=on
 
[C1N3]
type=server
address=172.30.0.126
port=3306
protocol=MariaDBBackend
proxy_protocol=on
 
[Galera-Monitor]
type=monitor
module=galeramon
servers=C1N1,
        C1N2,
        C1N3
user=maxscale
password=password
monitor_interval=10000
 
[Read-Listener]
type=listener
service=Splitter-Service
port=3306
protocol=MariaDBClient
authenticator_options=skip_authentication=true
 
[Splitter-Service]
type=service
router=readwritesplit
servers=C1N1,
        C1N2,
        C1N3
user=maxscale
password=password
max_slave_connections=100%

I also tried with {{

{authenticator=PAMBackendAuth}

}} for the backend servers, but that had the same result.

And now my logins fail for PAM accounts:

[ec2-user@ip-172-30-0-106 ~]$ mysql -u alice -h 127.0.0.1 -p
Enter password:
ERROR 1045 (28000): Access denied for user 'alice'@'::ffff:127.0.0.1' (using password: YES)
[ec2-user@ip-172-30-0-106 ~]$ mysql -u alice -h 172.30.0.106 -p
Enter password:
ERROR 1045 (28000): Access denied for user 'alice'@'::ffff:172.30.0.106' (using password: YES)

The MaxScale error log shows the following:

2019-05-22 04:13:56   info   : Added user: INSERT OR REPLACE INTO mysqlauth_users VALUES ('sstuser', 'localhost', NULL, 1, '2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19')
2019-05-22 04:13:56   info   : Added user: INSERT OR REPLACE INTO mysqlauth_users VALUES ('maxscale', '%', NULL, 1, '2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19')
2019-05-22 04:13:56   info   : Added user: INSERT OR REPLACE INTO mysqlauth_users VALUES ('alice', '%', NULL, 1, 'mariadb')
2019-05-22 04:13:56   warning: [MySQLAuth] Splitter-Service: login attempt for user 'alice'@[::ffff:127.0.0.1]:56509, authentication failed. User not found.
2019-05-22 04:14:09   info   : Added user: INSERT OR REPLACE INTO mysqlauth_users VALUES ('sstuser', 'localhost', NULL, 1, '2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19')
2019-05-22 04:14:09   info   : Added user: INSERT OR REPLACE INTO mysqlauth_users VALUES ('maxscale', '%', NULL, 1, '2470C0C06DEE42FD1618BB99005ADCA2EC9D1E19')
2019-05-22 04:14:09   info   : Added user: INSERT OR REPLACE INTO mysqlauth_users VALUES ('alice', '%', NULL, 1, 'mariadb')
2019-05-22 04:14:09   warning: [MySQLAuth] Splitter-Service: login attempt for user 'alice'@[::ffff:172.30.0.106]:58593, authentication failed. User not found.

The strange thing is that, even though it says "User not found", I can see in the error log that MySQLAuth is inserting information about the 'alice'@'%' user into the mysqlauth_users table, so it is definitely getting the information about the 'alice' account from the backend servers. However, since the account on the backend servers actually uses the PAM authentication plugin, the information that MySQLAuth grabs will not entirely be valid for the MySQLAuth module. We can see that it thinks the PAM service name ('mariadb') is a password hash, which is obviously incorrect, but that shouldn't matter if authenticator_options=skip_authentication=true is set anyway.

Regardless, after the fix for MXS-2494 is released, MySQLAuth won't grab information about PAM accounts at all. If your workaround requires authenticator=MySQLAuth to be set, then I think that even if we can get your workaround working for PAM accounts right now, it will break for PAM accounts once MaxScale 2.3.8 is released with that fix. I still think that fixing MXS-2494 is a good idea, because I saw a case where a user was able to log into MaxScale without a password when they shouldn't have been able to due to that bug.

I think something like MXS-2499 is the ultimate fix for this MySQLAuth vs. PAMAuth problem, since it is pretty awkward and counter-intuitive for one listener to be restricted to just a single authenticator.

In the mean time, do you think we need to implement authenticator_options=skip_authentication=true for the PAMAuth module in order to fix this for PAM accounts?

Your workaround does seem to work fine for accounts that use mysql_native_password.

Client:

[ec2-user@ip-172-30-0-106 ~]$ mysql -u maxscale -h 172.30.0.106 -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 10.3.14-MariaDB-log MariaDB Server
 
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
 
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
 
MariaDB [(none)]> SELECT USER(), CURRENT_USER();
+-----------------------------------------------------+----------------+
| USER()                                              | CURRENT_USER() |
+-----------------------------------------------------+----------------+
| maxscale@ip-172-30-0-106.us-west-2.compute.internal | maxscale@%     |
+-----------------------------------------------------+----------------+
1 row in set (0.001 sec)
 
MariaDB [(none)]> \q
Bye

Log:

2019-05-22 04:14:43   info   : Started Splitter-Service client session [6] for 'maxscale' from ::ffff:172.30.0.106
2019-05-22 04:14:43   info   : (6) Sending proxy-protocol header 'PROXY TCP6 ::ffff:172.30.0.106 ::ffff:172.30.0.106 58599 3306^M
' to backend C1N1.
2019-05-22 04:14:43   info   : (6) Connected to 'C1N1' with thread id 63
2019-05-22 04:14:43   info   : (6) Sending proxy-protocol header 'PROXY TCP6 ::ffff:172.30.0.106 ::ffff:172.30.0.106 58599 3306^M
' to backend C1N2.
2019-05-22 04:14:43   info   : (6) Connected to 'C1N2' with thread id 25
2019-05-22 04:14:43   info   : (6) Sending proxy-protocol header 'PROXY TCP6 ::ffff:172.30.0.106 ::ffff:172.30.0.106 58599 3306^M
' to backend C1N3.
2019-05-22 04:14:43   info   : (6) Connected to 'C1N3' with thread id 25
2019-05-22 04:14:43   info   : (6) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 37, type: QUERY_TYPE_READ|QUERY_TYPE_SYSVAR_READ, stmt: select @@version_comment limit 1
2019-05-22 04:14:43   info   : (6) Route query to slave: C1N2   [172.30.0.96]:3306 <
2019-05-22 04:14:43   info   : (6) Reply complete, last reply from C1N2
2019-05-22 04:14:45   info   : (6) > Autocommit: [enabled], trx is [not open], cmd: (0x03) COM_QUERY, plen: 34, type: QUERY_TYPE_READ, stmt: SELECT USER(), CURRENT_USER()
2019-05-22 04:14:45   info   : (6) Route query to slave: C1N2   [172.30.0.96]:3306 <
2019-05-22 04:14:45   info   : (6) Reply complete, last reply from C1N2
2019-05-22 04:14:47   info   : (6) > Autocommit: [enabled], trx is [not open], cmd: (0x01) COM_QUIT, plen: 5, type: QUERY_TYPE_SESSION_WRITE, stmt:
2019-05-22 04:14:47   info   : (6) Session write, routing to all servers.
2019-05-22 04:14:47   info   : (6) Route query to master: C1N1  [172.30.0.105]:3306
2019-05-22 04:14:47   info   : (6) Route query to slave: C1N2   [172.30.0.96]:3306
2019-05-22 04:14:47   info   : (6) Route query to slave: C1N3   [172.30.0.126]:3306
2019-05-22 04:14:47   info   : Stopped Splitter-Service client session [6]

Thanks!

Comment by markus makela [ 2019-05-22 ]

The skip_authentication option should prevent any authentication from taking place so there's no point in implementing it anywhere else if it works as intended.

Comment by Geoff Montee (Inactive) [ 2019-05-22 ]

I see.

I looked a bit more, and I think this check in mysql_auth_authenticate might be the thing that is failing for PAM accounts:

https://github.com/mariadb-corporation/MaxScale/blob/maxscale-2.3.7/server/modules/authenticator/MySQLAuth/mysql_auth.cc#L328

Maybe it should actually be this:

if (!instance->skip_auth && !client_data->correct_authenticator)

Comment by markus makela [ 2019-07-02 ]

I think that would work with PAM authentication as it doesn't require the password in order to work but the problem of missing users still remains. This might end up making the most sense to implement in the PAM authenticator.

Comment by markus makela [ 2020-07-03 ]

Changing to New Feature as this is expected behavior and seems to mostly be a performance related change.

Generated at Thu Feb 08 04:14:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.