Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Fixed
-
None
-
None
-
MXS-SPRINT-275
Description
Maxscale variable max_auth_errors_until_block blocking complete hosts and causing application outage.
As the default value is 10, after 10 failure attempts, it blocks the complete host/IP rather than just the user with authentication failure issue. This blocking causing the application outage.
- This behaviour impacts the complete business, rather it should be blocking only the user which is failing authentication and not the whole application.
- Client point of view is that this is a bug in implementation, this behaviour affected the trading system where the whole application got blocked. The 60 seconds gap caused the outage in client environment, and it keeps repeating after 60 seconds if there are more failures.
- So, for example if there is a new feature rollout for one of the application and during the automation if there is some wrong password or authentication fails for any reason, the complete host is blocked which is running the load balancer and the same load balancer has multiple applications running using different databases in MariaDB and this cause outage in other non-related applications as well. Multiple applications trying to access and all gets blocked, if one user fails authentication.
- Client also wants to understand the reason for which this variable was introduced and how fast this can be fixed.
Implementation
MaxScale 25.10
The new max_connect_errors setting which implements a mechanism that's similar to the similarly named feature in MariaDB. It defaults to 0 and needs to be enabled manually for this feature to be taken into use. The old max_auth_errors_until_block is still enabled by default with a default value of 10.
Compared to the MariaDB implementation, the implementation in MaxScale accepts between max_connect_errors and threads * max_connect_errors numbers of failures before blockng the host for 60 seconds. Unlike MariaDB, which blocks the host until FLUSH HOSTS is executed, MaxScale only blocks it for 60 second. If a further connection attempt is done from that host in those 60 seconds that it is blocked, the block is refreshed. The block is only cleared once there has been no connection attempts from that host for 60 seconds.
MaxScale 26.10
The default value of max_connect_errors is set to 100 and the old max_auth_errors_until_block is disabled by default (default set to 0).