SkySQL Dev/Test

This might be caused by MXS-2443. Can you check whether the database is created in one connection and the tables are filled in another one? You can see the session number (MariaDB thread ID) in the log with log_info enabled.

Further investigation suggests that this bug is not caused by replication.
I can reliably replicate it with a single mariadb server behind the maxscale.
If I do something like:

mysql -h <host> -P <port> -u <user> '-p<pass>' --ssl

and then

use sbtest;

I have no problems.

If I do

mysql -h <host> -P <port> -u <user> '-p<pass>' --ssl --database=sbtest

It almost always fails the first 1 or 2 tries.
Note, this only happens on a fresh install of maxscale and mariadb.
It does NOT happen if I do the calls directly to the mariadb server through 127.0.0.1 (so maxscale matters here).

I think this has something to do with auth/cred caching on the maxscale side. I don't know enough about it though.

It might be that MaxScale for some reason fails to fetch the authentication data from the backend servers. Does the MaxScale log contain any errors or messages when this happens?

2019-10-30 09:48:40   warning: (2) [Read-Write-Service] Refresh rate limit (once every 30 seconds) exceeded for load of users' table.

2019-10-30 09:48:40   warning: (2) [MariaDBAuth] Read-Write-Service: login attempt for user 'skysql_admin'@[10.0.4.4]:38782, authentication failed. Unknown database: sbtest

And again, it only happens if I start the mysql client with the --database=a parameter.
If I skip --database=a and do "USE a;" after that instead, it works correctly.

I think there might be a problem where the user loading rate limit is triggered too early. If you add users_refresh_time=0 under the [maxscale] section, does it work?

users_refresh_time=0 seems to work, thank you.
I'll check if it's a safe setting to have and remove my hacky workaround.
So the only thing left to say here is that whether it's a bug or not, the behavior is different from 2.3.9, where we had no such problem.

From the documentation:

In MaxScale 2.3.9 and older versions, the minimum allowed value was 10 seconds but, due to a bug, the default value was 0 which allowed infinite refreshes.

This is still probably a bug in MaxScale and how it limits the user loading at startup.

There's already code in place that prevents the user loading rate limitation from triggerting shortly after startup which means this is most likely expected behavior and the rate limitation should be disabled.