[MXS-2499] MaxScale still authenticates clients locally when proxy_protocol=on Created: 2019-05-22 Updated: 2020-07-21 Resolved: 2020-07-21 |
|
| Status: | Closed |
| Project: | MariaDB MaxScale |
| Component/s: | Authenticator, Protocol |
| Affects Version/s: | 2.3.7 |
| Fix Version/s: | 2.5.0 |
| Type: | New Feature | Priority: | Major |
| Reporter: | Geoff Montee (Inactive) | Assignee: | Esa Korhonen |
| Resolution: | Fixed | Votes: | 2 |
| Labels: | None | ||
| Issue Links: |
|
||||||||||||
| Description |
|
I was told by the MaxScale team that MaxScale should skip authenticating clients locally when proxy_protocol=on is set for the backend servers. The documentation doesn't seem to imply that this should be supported. The documentation seems to say that the proxy protocol is mostly intended to make user account management easier. Since the proxy protocol passes the original client IP address to the backend server in the proxy protocol header, the backend server isn't required to have user accounts for the specific user that resolve to both the client IP and the MaxScale IP. The user account just needs to resolve to the original client IP address. https://mariadb.com/kb/en/library/proxy-protocol-support/ To confirm whether MaxScale skips authenticating clients locally when proxy_protocol=on is set for the backend servers, I ran a test. I ran the test with the following: 1 MaxScale server running MaxScale 2.3.7 On the backend Galera nodes, I set proxy_protocol_networks to the IP address of the MaxScale server:
On the MaxScale server, I used the following configuration:
The specific user account involved uses PAM authentication, so I also needed to create the Unix user on the MaxScale instance, and all 3 backend nodes:
I also wanted to make sure that PAM authentication attempts are very thoroughly logged, so I followed these steps: https://mariadb.com/kb/en/library/authentication-plugin-pam/#custom-logging-with-pam_exec i.e. I created the following script:
And then used the following PAM service configuration in /etc/pam.d/mariadb:
And I created the following user on the backend nodes:
Then I logged in through MaxScale:
The MaxScale log shows that the MaxScale server is properly sending the proxy protocol headers to the backend servers:
However, when I check /tmp/pam_output.txt on the MaxScale instance, it is clear that MaxScale's PAMAuth authenticator module is still authenticating the client locally as well:
This makes sense to me, because I checked the source code, and I did not find any indication that setting proxy_protocol=on for a server gets rid of the local authentication step that MaxScale performs. proxy_protocol isn't checked before calling gw_read_do_authentication: And proxy_protocol isn't checked within gw_read_do_authentication: And it still calls the authenticate() function for the authenticator module: And the function that's called for PAMAuth doesn't seem to take any shortcuts if proxy_protocol is defined either: Is this working as-intended, or is MaxScale actually supposed to skip the local authentication step when the proxy protocol is used with the backend servers? |
| Comments |
| Comment by markus makela [ 2019-05-22 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yes, MaxScale does seem to authenticate the user regardless of what the state of proxy_protocol is. Due to the fact that some servers might not have proxy_protocol on, a configuration option might be required on the service level. Another option is to add a service level proxy_protocol option that automatically enables it for all servers used by that service. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by markus makela [ 2019-05-22 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Also, this can be worked around by using the default authenticator and `authenticator_options=skip_authentication=true`. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Geoff Montee (Inactive) [ 2019-05-22 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thanks, markus makela. Unfortunately, that workaround doesn't work either. Ironically, even though authentication is supposed to be skipped, PAM accounts fail to authenticate because MaxScale returns "User not found." I tried the following configuration:
I also tried with {{ {authenticator=PAMBackendAuth}}} for the backend servers, but that had the same result. And now my logins fail for PAM accounts:
The MaxScale error log shows the following:
The strange thing is that, even though it says "User not found", I can see in the error log that MySQLAuth is inserting information about the 'alice'@'%' user into the mysqlauth_users table, so it is definitely getting the information about the 'alice' account from the backend servers. However, since the account on the backend servers actually uses the PAM authentication plugin, the information that MySQLAuth grabs will not entirely be valid for the MySQLAuth module. We can see that it thinks the PAM service name ('mariadb') is a password hash, which is obviously incorrect, but that shouldn't matter if authenticator_options=skip_authentication=true is set anyway. Regardless, after the fix for I think something like In the mean time, do you think we need to implement authenticator_options=skip_authentication=true for the PAMAuth module in order to fix this for PAM accounts? Your workaround does seem to work fine for accounts that use mysql_native_password. Client:
Log:
Thanks! | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by markus makela [ 2019-05-22 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
The skip_authentication option should prevent any authentication from taking place so there's no point in implementing it anywhere else if it works as intended. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Geoff Montee (Inactive) [ 2019-05-22 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I see. I looked a bit more, and I think this check in mysql_auth_authenticate might be the thing that is failing for PAM accounts: Maybe it should actually be this:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by markus makela [ 2019-07-02 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I think that would work with PAM authentication as it doesn't require the password in order to work but the problem of missing users still remains. This might end up making the most sense to implement in the PAM authenticator. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by markus makela [ 2020-07-03 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Changing to New Feature as this is expected behavior and seems to mostly be a performance related change. |