[MXS-2773] SkySQL: Maxscale 2.4.2 takes longer to allow connections after restart compared to 2.3.9 Created: 2019-11-18  Updated: 2019-12-02  Resolved: 2019-12-02

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: None
Fix Version/s: 2.3.15

Type: Bug Priority: Major
Reporter: Nedyalko Petrov (Inactive) Assignee: markus makela
Resolution: Fixed Votes: 0
Labels: SkySQLMVP


 Description   

Summary:
We had identified an increase of the MaxScale 2.4 start-up time compared to MaxScale 2.3 Seems like restarting in 2.4 is much slower than 2.3 and even after the restart it seems to not allow queries for about 60s (compared to about 5s for 2.3).

Sample Problematic Scenatio:
topology: 1 maxsclae with 1 mariadb server behind it.
We start a sysbech test with N threads in order to have a load on the server.
We delete the maxscale pod.
Kubernetes starts a new maxscale pod.
After the restart we re-add the mariadb server to maxscale (done with a script, so it's within 2-3 seconds of maxscale restarting).

Impact:
With maxscale 2.3, about 5-6 seconds after the pod deletion, the new maxscale takes over and properly routes the queries.
With maxscale 2.4 (tested with 2.4.0, 2.4.2 and 2.4.4) it takes about 60s before the new maxscale routes the queries correctly.
Interestingly, when looking at "maxctrl list servers", maxscale reports that the mariadb server is up and running and has N connections to it, but the queries just don't seem to go through.I'm pretty sure we are hitting an issue with users_refresh_time=0 in 2.4
W've tested without it, and the problem is gone.
Unfortunately we need this setting for the time being in order to work around https://jira.mariadb.org/browse/MXS-2745

For 2.4 we use users_refresh_time=0, which should be an invalid value, but we need it to work around https://jira.mariadb.org/browse/MXS-2745
For 2.3 we do not set users_refresh_time, so it uses the default value, which is also 0, due to a known bug in 2.3

Logs for reference:

sysbench log for maxscale 2.4

[ 1s ] thds: 16 tps: 138.21 qps: 4221.87 (r/w/o: 3800.28/421.59/0.00) lat (ms,95%): 155.80 err/s: 0.00 reconn/s: 0.00
[ 2s ] thds: 16 tps: 144.30 qps: 4208.79 (r/w/o: 3776.89/431.90/0.00) lat (ms,95%): 170.48 err/s: 0.00 reconn/s: 0.00
[ 3s ] thds: 16 tps: 191.56 qps: 5531.22 (r/w/o: 4958.55/572.68/0.00) lat (ms,95%): 127.81 err/s: 0.00 reconn/s: 0.00
[ 4s ] thds: 16 tps: 162.58 qps: 4870.30 (r/w/o: 4386.58/483.72/0.00) lat (ms,95%): 167.44 err/s: 0.00 reconn/s: 0.00
[ 5s ] thds: 16 tps: 172.75 qps: 4911.78 (r/w/o: 4393.54/518.24/0.00) lat (ms,95%): 132.49 err/s: 0.00 reconn/s: 0.00
[ 6s ] thds: 16 tps: 111.75 qps: 3237.87 (r/w/o: 2902.61/335.26/0.00) lat (ms,95%): 179.94 err/s: 0.00 reconn/s: 0.00
[ 7s ] thds: 16 tps: 144.56 qps: 4305.76 (r/w/o: 3872.07/433.69/0.00) lat (ms,95%): 179.94 err/s: 0.00 reconn/s: 0.00
[ 8s ] thds: 16 tps: 128.01 qps: 3502.14 (r/w/o: 3118.13/384.02/0.00) lat (ms,95%): 176.73 err/s: 0.00 reconn/s: 0.00
[ 9s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 10s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 11s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 12s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 13s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 14s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 15s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 16s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 17s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 18s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 19s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 21s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 22s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 23s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 24s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 25s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 26s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 27s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 28s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 29s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 31s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 32s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 33s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 34s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 35s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 36s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 37s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 38s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 39s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 41s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 42s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 43s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 44s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 45s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 46s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 47s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 48s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 49s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 51s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 52s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 53s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 54s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 55s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 56s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 57s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 58s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 59s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 61s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 62s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 63s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 64s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 65s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 66s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 67s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 68s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 69s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 70s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 71s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 72s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 73s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 74s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 75s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 76s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 77s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 78s ] thds: 16 tps: 132.07 qps: 4245.37 (r/w/o: 3838.14/407.23/0.00) lat (ms,95%): 69758.52 err/s: 0.00 reconn/s: 16.01
[ 79s ] thds: 16 tps: 207.03 qps: 5946.93 (r/w/o: 5336.83/610.10/0.00) lat (ms,95%): 102.97 err/s: 0.00 reconn/s: 0.00
[ 80s ] thds: 16 tps: 176.00 qps: 5002.96 (r/w/o: 4474.97/528.00/0.00) lat (ms,95%): 144.97 err/s: 0.00 reconn/s: 0.00
[ 81s ] thds: 16 tps: 154.46 qps: 4439.99 (r/w/o: 3973.66/466.33/0.00) lat (ms,95%): 153.02 err/s: 0.00 reconn/s: 0.00
[ 82s ] thds: 16 tps: 131.14 qps: 3906.68 (r/w/o: 3516.31/390.36/0.00) lat (ms,95%): 158.63 err/s: 0.00 reconn/s: 0.00
[ 83s ] thds: 16 tps: 190.00 qps: 5461.86 (r/w/o: 4887.87/573.99/0.00) lat (ms,95%): 97.55 err/s: 0.00 reconn/s: 0.00
[ 84s ] thds: 16 tps: 176.02 qps: 4911.50 (r/w/o: 4387.45/524.05/0.00) lat (ms,95%): 144.97 err/s: 0.00 reconn/s: 0.00

– maxscale log
Note the "Host '10.16.2.209' blocked for 60 seconds due to too many authentication failures."

Version:    2.4.2-1
Git commit: 98e1ce004f593c86653a41451861f8c1eacf1acb
Built:      2019-11-04_10:46:35
 
Info : MaxScale will be run in the terminal process.
Configuration file : /etc/maxscale-cfg/maxscale.cnf
Log directory      : /var/log/maxscale
Data directory     : /var/lib/maxscale
Module directory   : /usr/lib/x86_64-linux-gnu/maxscale
Service cache      : /var/cache/maxscale
 
2019-11-18 12:52:30   notice : syslog logging is enabled.
2019-11-18 12:52:30   notice : maxlog logging is enabled.
2019-11-18 12:52:30   notice : Using up to 1.10GiB of memory for query classifier cache
2019-11-18 12:52:30   notice : The collection of SQLite memory allocation statistics turned off.
2019-11-18 12:52:30   notice : Threading mode of SQLite set to Multi-thread.
2019-11-18 12:52:30   notice : MariaDB MaxScale 2.4.2 started (Commit: aad4148d77bf2dfbaa0042bc45abda30c101cad2)
2019-11-18 12:52:30   notice : MaxScale is running in process 20
2019-11-18 12:52:30   notice : Configuration file: /etc/maxscale-cfg/maxscale.cnf
2019-11-18 12:52:30   notice : Log directory: /var/log/maxscale
2019-11-18 12:52:30   notice : Data directory: /var/lib/maxscale
2019-11-18 12:52:30   notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale
2019-11-18 12:52:30   notice : Service cache: /var/cache/maxscale
2019-11-18 12:52:30   notice : No query classifier specified, using default 'qc_sqlite'.
2019-11-18 12:52:30   notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so
2019-11-18 12:52:30   notice : Query classification results are cached and reused. Memory used per thread: 560.83MiB
2019-11-18 12:52:30   notice : Loading /etc/maxscale-cfg/maxscale.cnf.
2019-11-18 12:52:30   notice : /etc/maxscale-cfg/maxscale.cnf.d does not exist, not reading.
2019-11-18 12:52:30   notice : Loaded module mariadbclient: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbclient.so
2019-11-18 12:52:30   notice : [readwritesplit] Initializing statement-based read/write split router module.
2019-11-18 12:52:30   notice : Loaded module readwritesplit: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadwritesplit.so
2019-11-18 12:52:30   notice : [readconnroute] Initialise readconnroute router module.
2019-11-18 12:52:30   notice : Loaded module readconnroute: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadconnroute.so
2019-11-18 12:52:30   notice : [mariadbmon] Initialise the MariaDB Monitor module.
2019-11-18 12:52:30   notice : Loaded module mariadbmon: V1.5.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbmon.so
2019-11-18 12:52:30   notice : Encrypted password file /var/lib/maxscale/.secrets can't be accessed (No such file or directory). Password encryption is not used.
2019-11-18 12:52:30   notice : Loaded module mariadbauth: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbauth.so
2019-11-18 12:52:30   notice : Started REST API on [127.0.0.1]:8989
2019-11-18 12:52:30   notice : MaxScale started with 2 worker threads, each with a stack size of 8388608 bytes.
2019-11-18 12:52:30   notice : Starting a total of 2 services...
2019-11-18 12:52:30   notice : Listening for connections at [0.0.0.0]:4006
2019-11-18 12:52:30   notice : Service 'Read-Write-Service' started (1/2)
2019-11-18 12:52:30   notice : Listening for connections at [0.0.0.0]:4008
2019-11-18 12:52:30   notice : Service 'Read-Only-Service' started (2/2)
2019-11-18 12:52:30   warning: Removing stale journal file for monitor 'MariaDB-Monitor'.
2019-11-18 12:52:30   warning: [mariadbmon] Tried to find a master but no valid master server found.
2019-11-18 12:52:30   error  : [mariadbmon] No Master can be determined
2019-11-18 12:52:32   warning: (1) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47282, authentication failed. User not found.
2019-11-18 12:52:32   warning: (2) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47284, authentication failed. User not found.
2019-11-18 12:52:32   warning: (3) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47286, authentication failed. User not found.
2019-11-18 12:52:32   warning: (4) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47288, authentication failed. User not found.
2019-11-18 12:52:32   warning: (5) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47292, authentication failed. User not found.
2019-11-18 12:52:32   warning: (6) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47294, authentication failed. User not found.
2019-11-18 12:52:32   warning: (7) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47296, authentication failed. User not found.
2019-11-18 12:52:32   warning: (8) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47194, authentication failed. User not found.
2019-11-18 12:52:32   warning: (17) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47210, authentication failed. User not found.
2019-11-18 12:52:32   warning: (10) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.16.2.211]:47208, authentication failed. User not found. (subsequent similar messages suppressed for 10000 milliseconds)
2019-11-18 12:52:32   notice : (9) Host '10.16.2.211' blocked for 60 seconds due to too many authentication failures.
2019-11-18 12:52:32   notice : (22) Host '10.16.2.211' blocked for 60 seconds due to too many authentication failures.
2019-11-18 12:52:32   notice : Loaded module MariaDBBackend: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackend.so
2019-11-18 12:52:32   notice : Loaded module mariadbbackendauth: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackendauth.so
2019-11-18 12:52:32   notice : Loaded server states from journal file: /var/lib/maxscale/MariaDB-Monitor/monitor.dat
2019-11-18 12:52:32   notice : Added server '10.16.4.5' to monitor 'MariaDB-Monitor'
2019-11-18 12:52:32   notice : Added server '10.16.4.5' to service 'Read-Only-Service'
2019-11-18 12:52:32   notice : Added server '10.16.4.5' to service 'Read-Write-Service'
2019-11-18 12:52:32   notice : Server '10.16.4.5' version: 10.4.8-3-MariaDB-enterprise-log
2019-11-18 12:52:32   notice : [mariadbmon] Selecting new master server.
2019-11-18 12:52:32   notice : [mariadbmon] Setting '10.16.4.5' as master.
2019-11-18 12:52:32   notice : Server changed state: 10.16.4.5[10.16.4.5:3306]: new_master. [Running] -> [Master, Running]
2019-11-18 12:52:33   notice : Updated 'passive' from 'true' to 'false'
2019-11-18 12:52:36   notice : Create network user 'admin_tmp'
OK
OK
2019-11-18 12:52:38   notice : Deleted network user 'admin_tmp'
OK

for reference, sysbench logs for 2.3
it reconnects much quicker

[ 18s ] thds: 16 tps: 191.58 qps: 5482.06 (r/w/o: 4924.27/557.78/0.00) lat (ms,95%): 99.33 err/s: 0.00 reconn/s: 0.00
[ 19s ] thds: 16 tps: 131.29 qps: 3836.51 (r/w/o: 3429.61/406.90/0.00) lat (ms,95%): 170.48 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 16 tps: 173.01 qps: 5032.44 (r/w/o: 4525.39/507.04/0.00) lat (ms,95%): 137.35 err/s: 0.00 reconn/s: 0.00
[ 21s ] thds: 16 tps: 161.88 qps: 4717.38 (r/w/o: 4207.77/509.61/0.00) lat (ms,95%): 144.97 err/s: 0.00 reconn/s: 0.00
[ 22s ] thds: 16 tps: 45.03 qps: 1117.79 (r/w/o: 1006.72/111.08/0.00) lat (ms,95%): 121.08 err/s: 0.00 reconn/s: 0.00
[ 23s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 24s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 25s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 26s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 27s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 28s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 29s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 31s ] thds: 16 tps: 0.00 qps: 0.00 (r/w/o: 0.00/0.00/0.00) lat (ms,95%): 0.00 err/s: 0.00 reconn/s: 0.00
[ 32s ] thds: 16 tps: 14.00 qps: 407.06 (r/w/o: 365.05/42.01/0.00) lat (ms,95%): 62.19 err/s: 0.00 reconn/s: 1.00
[ 33s ] thds: 16 tps: 23.00 qps: 685.02 (r/w/o: 616.02/69.00/0.00) lat (ms,95%): 53.85 err/s: 0.00 reconn/s: 0.00

and the maxscale logs. Note there is no 60 seconds block

Version:    2.3.9-0
Git commit: 74143ba6a955d0c58468185fefc199530ddc3faf
Built:      2019-08-30_09:39:22
 
Info : MaxScale will be run in the terminal process.
Configuration file : /etc/maxscale-cfg/maxscale.cnf
Log directory      : /var/log/maxscale
Data directory     : /var/lib/maxscale
Module directory   : /usr/lib/x86_64-linux-gnu/maxscale
Service cache      : /var/cache/maxscale
 
2019-11-18 13:05:11   notice : syslog logging is enabled.
2019-11-18 13:05:11   notice : maxlog logging is enabled.
2019-11-18 13:05:11   notice : Using up to 1.10GiB of memory for query classifier cache
2019-11-18 13:05:11   notice : The collection of SQLite memory allocation statistics turned off.
2019-11-18 13:05:11   notice : Threading mode of SQLite set to Multi-thread.
2019-11-18 13:05:11   notice : MariaDB MaxScale 2.3.9 started (Commit: 66d6ce6fa598a43b20af052ef6c9f637a3279fa1)
2019-11-18 13:05:11   notice : MaxScale is running in process 19
2019-11-18 13:05:11   notice : Configuration file: /etc/maxscale-cfg/maxscale.cnf
2019-11-18 13:05:11   notice : Log directory: /var/log/maxscale
2019-11-18 13:05:11   notice : Data directory: /var/lib/maxscale
2019-11-18 13:05:11   notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale
2019-11-18 13:05:11   notice : Service cache: /var/cache/maxscale
2019-11-18 13:05:11   notice : No query classifier specified, using default 'qc_sqlite'.
2019-11-18 13:05:11   notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so
2019-11-18 13:05:11   notice : Query classification results are cached and reused. Memory used per thread: 560.83MiB
2019-11-18 13:05:11   notice : Loading /etc/maxscale-cfg/maxscale.cnf.
2019-11-18 13:05:11   notice : /etc/maxscale-cfg/maxscale.cnf.d does not exist, not reading.
2019-11-18 13:05:11   notice : Initialise the MariaDB Monitor module.
2019-11-18 13:05:11   notice : Loaded module mariadbmon: V1.5.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbmon.so
2019-11-18 13:05:11   notice : Loaded module mariadbclient: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbclient.so
2019-11-18 13:05:11   notice : Initializing statement-based read/write split router module.
2019-11-18 13:05:11   notice : Loaded module readwritesplit: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadwritesplit.so
2019-11-18 13:05:11   notice : Initialise readconnroute router module.
2019-11-18 13:05:11   notice : Loaded module readconnroute: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadconnroute.so
2019-11-18 13:05:11   notice : Loaded module MySQLAuth: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmysqlauth.so
2019-11-18 13:05:11   notice : Housekeeper thread started.
2019-11-18 13:05:11   warning: Removing stale journal file for monitor 'MariaDB-Monitor'.
2019-11-18 13:05:11   notice : Encrypted password file /var/lib/maxscale/.secrets can't be accessed (No such file or directory). Password encryption is not used.
2019-11-18 13:05:11   notice : Starting a total of 2 services...
2019-11-18 13:05:11   notice : Listening for connections at [0.0.0.0]:4006 with protocol MySQL
2019-11-18 13:05:11   notice : Service 'Read-Write-Service' started (1/2)
2019-11-18 13:05:11   notice : Listening for connections at [0.0.0.0]:4008 with protocol MySQL
2019-11-18 13:05:11   notice : Service 'Read-Only-Service' started (2/2)
2019-11-18 13:05:11   warning: No valid master server found.
2019-11-18 13:05:11   error  : No Master can be determined
2019-11-18 13:05:11   notice : Started REST API on [127.0.0.1]:8989
2019-11-18 13:05:11   notice : MaxScale started with 2 worker threads, each with a stack size of 8388608 bytes.
2019-11-18 13:05:12   notice : Loaded module MariaDBBackend: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackend.so
2019-11-18 13:05:12   notice : Loaded module MySQLBackendAuth: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmysqlbackendauth.so
2019-11-18 13:05:12   notice : Created server '10.16.4.6' at 10.16.4.6:3306
2019-11-18 13:05:12   notice : Loaded server states from journal file: /var/lib/maxscale/MariaDB-Monitor/monitor.dat
2019-11-18 13:05:12   notice : Added server '10.16.4.6' to monitor 'MariaDB-Monitor'
2019-11-18 13:05:12   notice : Added server '10.16.4.6' to service 'Read-Only-Service'
2019-11-18 13:05:12   notice : Added server '10.16.4.6' to service 'Read-Write-Service'
2019-11-18 13:05:12   notice : Selecting new master server.
2019-11-18 13:05:12   notice : Setting '10.16.4.6' as master.
2019-11-18 13:05:12   notice : Server changed state: 10.16.4.6[10.16.4.6:3306]: new_master. [Running] -> [Master, Running]
2019-11-18 13:05:13   notice : Updated 'passive' from 'true' to 'false'
2019-11-18 13:05:17   notice : Create network user 'admin_tmp'
OK
2019-11-18 13:05:17   notice : Deleted network user 'admin'
2019-11-18 13:05:17   notice : Create network user 'admin'
OK
2019-11-18 13:05:18   notice : Deleted network user 'admin_tmp'
OK



 Comments   
Comment by Timofey Turenko [ 2019-11-19 ]

I tried to simply start Maxscale in a Docker container.

2.4:

turenko@turenko-home:~/docker$ mdbci up docker1
2019-11-19T14:19:36 INFO: MariaDB CI CLI, version: 80, 533fcdbd57711eb6f315d25d5f740d8ad2622548, 2019-10-04
2019-11-19T14:19:36 INFO: mdbci called with: up docker1
2019-11-19T14:19:36 INFO: Loading MDBCI configuration file
2019-11-19T14:19:36 INFO: Loading repository configuration files
2019-11-19T14:19:36 INFO: Looking up for repos in '/tmp/.mount_mdbcihPQ5ok/mdbci/config/repo.d'
2019-11-19T14:19:36 INFO: Looking up for repos in '/home/turenko/.config/mdbci/repo.d'
2019-11-19T14:19:40 INFO: Loaded repos: 9969
2019-11-19T14:19:40 INFO: Bringing up docker nodes
2019-11-19T14:19:40 INFO: Selecting Docker Swarm services to be brought up
2019-11-19T14:19:40 INFO: Checking the bridge network state
2019-11-19T14:19:40 INFO: Invoking command: docker network ls -f name=docker1_mdbci_config_bridge_network --format .Name
2019-11-19T14:19:41 INFO: Creating the bridge network
2019-11-19T14:19:41 INFO: Invoking command: docker network create docker1_mdbci_config_bridge_network
2019-11-19T14:19:42 INFO: Invoking command: docker stack deploy --with-registry-auth -c /home/turenko/vms/docker1/docker-partial-configuration.yaml docker1
2019-11-19T14:19:42 INFO: Creating network docker1_default
2019-11-19T14:19:42 INFO: Creating service docker1_maxscale
2019-11-19T14:19:48 INFO: Waiting for stack services to become ready
2019-11-19T14:19:48 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:48 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:49 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:49 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:50 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:50 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:51 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:52 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:53 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:53 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:54 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:54 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:55 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:55 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:56 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:56 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:57 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:57 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:58 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:58 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:19:59 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:19:59 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:00 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:00 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:01 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:02 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:03 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:03 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:04 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:04 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:05 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:05 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:06 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:06 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:07 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:07 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:08 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:08 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:09 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:09 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:10 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:10 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:11 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:12 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:13 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:13 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:14 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:14 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:15 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:15 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:16 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:16 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:17 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:17 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:18 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:18 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:19 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:19 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:20 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:21 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:22 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:22 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:23 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:23 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:24 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:24 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:25 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:25 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:26 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:26 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:27 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:27 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:28 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:28 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:29 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:29 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:30 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:31 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:32 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:32 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:33 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:33 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:34 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:34 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:35 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:35 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:36 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:36 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:37 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:37 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:38 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:38 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:39 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:39 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:41 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:41 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:42 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:42 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:43 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:43 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:44 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:44 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:45 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:45 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:46 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:46 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:47 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:47 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:48 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:48 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:49 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:50 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:51 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:51 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:52 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:52 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:53 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:53 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:54 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:54 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:55 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:55 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:56 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:56 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:57 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:57 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:20:58 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:20:58 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:00 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:00 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:01 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:01 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:02 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:02 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:03 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:03 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:04 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:04 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:05 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:05 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:06 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:06 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:07 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:07 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:08 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:08 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:09 INFO: Invoking command: docker stack ps --format '.ID' docker1
2019-11-19T14:21:10 INFO: Invoking command: docker inspect udxphlsupd1s
2019-11-19T14:21:10 INFO: Invoking command: docker inspect q44ttvwhzu7x5hawgumyhko85
2019-11-19T14:21:10 INFO: Invoking command: docker exec f25c00eaecfe4f54086e7e65f08d22d02bcb43969ea8013a246c1da6d49c5cae cat /proc/net/fib_trie
2019-11-19T14:21:12 INFO: Checking that all required services are running
2019-11-19T14:21:12 INFO: Waiting for applications to become available
2019-11-19T14:21:12 INFO: Invoking command: docker exec f25c00eaecfe4f54086e7e65f08d22d02bcb43969ea8013a246c1da6d49c5cae maxctrl show maxscale
2019-11-19T14:21:12 INFO: Attaching containers to network
2019-11-19T14:21:12 INFO: Invoking command: docker network inspect docker1_mdbci_config_bridge_network
2019-11-19T14:21:13 INFO: Invoking command: docker network connect docker1_mdbci_config_bridge_network f25c00eaecfe4f54086e7e65f08d22d02bcb43969ea8013a246c1da6d49c5cae
2019-11-19T14:21:14 INFO: Invoking command: docker network inspect docker1_mdbci_config_bridge_network
2019-11-19T14:21:14 INFO: Generating network configuration file
2019-11-19T14:21:14 INFO: Success with
2019-11-19T14:21:14 DEBUG: MDBCI success with code: 0
turenko@turenko-home:~/docker$

2.3:

turenko@turenko-home:~/docker$ mdbci up docker23
2019-11-19T14:30:31 INFO: MariaDB CI CLI, version: 80, 533fcdbd57711eb6f315d25d5f740d8ad2622548, 2019-10-04
2019-11-19T14:30:31 INFO: mdbci called with: up docker23
2019-11-19T14:30:31 INFO: Loading MDBCI configuration file
2019-11-19T14:30:31 INFO: Loading repository configuration files
2019-11-19T14:30:31 INFO: Looking up for repos in '/tmp/.mount_mdbciuCkpHE/mdbci/config/repo.d'
2019-11-19T14:30:31 INFO: Looking up for repos in '/home/turenko/.config/mdbci/repo.d'
2019-11-19T14:30:35 INFO: Loaded repos: 9969
2019-11-19T14:30:35 INFO: Bringing up docker nodes
2019-11-19T14:30:35 INFO: Selecting Docker Swarm services to be brought up
2019-11-19T14:30:35 INFO: Checking the bridge network state
2019-11-19T14:30:35 INFO: Invoking command: docker network ls -f name=docker23_mdbci_config_bridge_network --format .Name
2019-11-19T14:30:35 INFO: Creating the bridge network
2019-11-19T14:30:35 INFO: Invoking command: docker network create docker23_mdbci_config_bridge_network
2019-11-19T14:30:36 INFO: Invoking command: docker stack deploy --with-registry-auth -c /home/turenko/vms/docker23/docker-partial-configuration.yaml docker23
2019-11-19T14:30:36 INFO: Creating network docker23_default
2019-11-19T14:30:36 INFO: Creating service docker23_maxscale
2019-11-19T14:30:41 INFO: Waiting for stack services to become ready
2019-11-19T14:30:41 INFO: Invoking command: docker stack ps --format '.ID' docker23
2019-11-19T14:30:41 INFO: Invoking command: docker inspect 6x54et6dlawr
2019-11-19T14:30:42 INFO: Invoking command: docker stack ps --format '.ID' docker23
2019-11-19T14:30:42 INFO: Invoking command: docker inspect 6x54et6dlawr
2019-11-19T14:30:43 INFO: Invoking command: docker stack ps --format '.ID' docker23
2019-11-19T14:30:43 INFO: Invoking command: docker inspect 6x54et6dlawr
2019-11-19T14:30:44 INFO: Invoking command: docker stack ps --format '.ID' docker23
2019-11-19T14:30:44 INFO: Invoking command: docker inspect 6x54et6dlawr
2019-11-19T14:30:45 INFO: Invoking command: docker stack ps --format '.ID' docker23
2019-11-19T14:30:45 INFO: Invoking command: docker inspect 6x54et6dlawr
2019-11-19T14:30:45 INFO: Invoking command: docker inspect q2bo16bzrkvkpaxryvkyx09uo
2019-11-19T14:30:46 INFO: Invoking command: docker exec ccd77c4690f389ee4a956e17e90dcc168f822502c28d06bd2ded6eb5e82792d6 cat /proc/net/fib_trie
2019-11-19T14:30:48 INFO: Checking that all required services are running
2019-11-19T14:30:48 INFO: Waiting for applications to become available
2019-11-19T14:30:48 INFO: Invoking command: docker exec ccd77c4690f389ee4a956e17e90dcc168f822502c28d06bd2ded6eb5e82792d6 maxctrl show maxscale
2019-11-19T14:30:50 INFO: Attaching containers to network
2019-11-19T14:30:50 INFO: Invoking command: docker network inspect docker23_mdbci_config_bridge_network
2019-11-19T14:30:50 INFO: Invoking command: docker network connect docker23_mdbci_config_bridge_network ccd77c4690f389ee4a956e17e90dcc168f822502c28d06bd2ded6eb5e82792d6
2019-11-19T14:30:51 INFO: Invoking command: docker network inspect docker23_mdbci_config_bridge_network
2019-11-19T14:30:51 INFO: Generating network configuration file
2019-11-19T14:30:51 INFO: Success with
2019-11-19T14:30:51 DEBUG: MDBCI success with code: 0

Comment by Timofey Turenko [ 2019-11-19 ]

2.3 start takes 7 seconds, 2.4 - about 1.5 mintes

Comment by markus makela [ 2019-12-02 ]

The max_auth_errors_until_block parameter now supports the value 0 which disables the host blocking mechanism.

Generated at Thu Feb 08 04:16:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.