Can you attach your maxscale.cnf with all sensitive information removed?
[maxscale]
|
threads=auto
|
local_address=10.10.x.x
|
query_classifier_cache_size=100M
|
|
|
[db1]
|
type=server
|
address=10.10.x.x
|
port=3307
|
protocol=MariaDBBackend
|
proxy_protocol=on
|
persistpoolmax=50
|
persistmaxtime=900
|
|
|
[db2]
|
type=server
|
address=10.10.x.x
|
port=3307
|
protocol=MariaDBBackend
|
proxy_protocol=on
|
persistpoolmax=50
|
persistmaxtime=900
|
|
|
[db3]
|
type=server
|
address=10.10.x.x
|
port=3307
|
protocol=MariaDBBackend
|
proxy_protocol=on
|
persistpoolmax=50
|
persistmaxtime=900
|
|
|
[Galera-Monitor]
|
type=monitor
|
module=galeramon
|
servers=db1,db2,db3
|
user=maxscale
|
password=xxx
|
monitor_interval=2000
|
available_when_donor=true
|
disable_master_failback=true
|
|
|
[Read-Write-Service]
|
type=service
|
router=readwritesplit
|
servers=db1,db2,db3
|
user=maxscale
|
password=xxxx
|
connection_keepalive=30
|
max_sescmd_history=50
|
disable_sescmd_history=false
|
master_accept_reads=false
|
master_failure_mode=fail_on_write
|
enable_root_user=true
|
master_reconnection=true
|
prune_sescmd_history=true
|
session_track_trx_state=true
|
causal_reads=true
|
causal_reads_timeout=2
|
|
|
[Read-Write-Listener]
|
type=listener
|
service=Read-Write-Service
|
protocol=MariaDBClient
|
port=3306
|
Does the error message go away or change to something else if you remove proxy_protocol=on? Another thing to test would be to disable both persistpoolmax and persistmaxtime.
proxy_protocol=off makes no difference
proxy_protocol=on and persistpoolmax+persistmaxtime removed from the config fixes the error, so it's somehow related to persistent connections.
What version of MariaDB are you using with MaxScale?
MariaDB version is 10.3.25
been running on 2.5.5 for 3 days without persistpoolmax+persistmaxtime without having this issue, so it's definitely related. queryload was approx. 3500q/s 90% write, 10% read
OK, that suggests there's something wrong with the resetting of the connection when it is taken out of the connection pool.
The same issue with module=mariadbmon also
it seems the persistpoolmax+persistmaxtime in the server is causing the error in all monitors in 2.5.5
The issue is not there in Maxscale 2.4.x and 2.3.x
So either a downgrade of Maxscale or commenting out the persistpoolmax+persistmaxtime fix the error for now
Have you had a chance to upgrade to the latest 2.5 release of MaxScale? I'd be interested in knowing if you still see this problem.
Running 2.5.9 for some minutes with persistent connections enabled and haven't seen the issue. Will monitor for some more time and let you know some stats / info later. So far so good.
Errors are gone 
running stable for some hours now. I do have a question though
connection_pool_empty 788003
connections 1446
max_connections 1596
max_pool_size 12
persistent_connections 99
reused_connections 326937
total_connections 1114940
does this mean my persistent settings are too low (100 conn, 15min) because conn_pool_empty is high compared to reused_connections?
I think the problem of low pool usage is already reported as MXS-3141. Since this issue was about the pool not working at all, I'd recommend reporting it there as well.
If you can, let us know how how the latest release behaves over a longer period. I'll leave the issue open for now.
haven't seen the auth issues any more, uptime 14 days.
connection_pool_empty 41540154
connections 1444
max_connections 2091
max_pool_size 13
persistent_connections 100
reused_connections 17979572
how do I interpret these numbers?
connection_pool_empty is the number of times a new connection needed to be created and reused_connections is how often a connection was taken from the pool. There are a couple of reasons why a new connection would be created even there are connections in the pool. In the case of proxy_protocol, the address must match to the one in the connection pool. This is done to prevent clients from a certain IP appearing to be from another.
connections is the current number of connections and max_connections is the historical maximum of the current number of connections. max_pool_size is the maximum number of connections that were simultaneously in the pool and persistent_connections is the current number of connections that are in the pool.
about those last two, persistent_connections is a static number and max_pool_size the current available?
persistent_connections is the current pool size (a bit of an unfortunate name for the field, should've been current_pool_size) and max_pool_size is the maximum number ever reached. The latter should eventually be the same value as persistpoolmax in the configuration file.
then how can max_pool_size < persistent_connections ? or am i missing something
I missed that part and it definitely looks strange: I think the pool maximum might be for a single thread and the total count might be for all the threads. If you can, could you open a separate bug report for this?
I'll close this as Fixed in 2.5.9 since you haven't seen any problems with that release. If you see any problems, let us know and we'll reopen this.