2022-11-01 00:00:01 notice : Log rotation complete 2022-11-01 01:01:36 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-01 01:01:38 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-01 01:01:39 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-01 01:01:41 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-01 01:01:43 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-01 01:01:44 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-01 01:01:46 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-01 01:01:48 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-01 01:01:50 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-01 01:01:51 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-01 01:01:53 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-01 01:01:54 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-01 01:01:56 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-01 01:01:58 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-01 01:01:59 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-01 01:02:01 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-01 01:02:03 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-01 01:02:04 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-01 01:02:06 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-01 01:02:08 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-01 01:02:10 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-01 01:02:11 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-01 01:02:13 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-01 01:02:14 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-01 01:02:16 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-01 01:02:18 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-01 01:02:19 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-01 01:02:21 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-01 01:02:22 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-01 01:02:24 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-01 01:02:32 error : (107059888) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb51' is in maintenance. 2022-11-01 01:02:32 error : (107059888) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-01 01:02:39 error : (107052171) [readconnroute] Failed to route MySQL command 14 to backend server. Server 'cmkprod02-mdb75' is in maintenance. 2022-11-01 01:02:39 error : (107052171) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-01 01:02:41 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:42 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:44 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:46 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:47 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:49 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:52 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:55 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 01:02:57 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.114:3306 2022-11-01 01:02:59 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.202:3306 2022-11-01 01:03:00 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.57:3306 2022-11-01 01:03:02 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.73:3306 2022-11-01 01:03:03 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.95:3306 2022-11-01 01:03:05 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.251:3306 2022-11-01 01:03:06 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.89:3306 2022-11-01 01:03:08 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.233:3306 2022-11-01 01:03:09 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.102:3306 2022-11-01 01:03:11 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.18:3306 2022-11-01 10:28:56 notice : Journal file was created 33961 seconds ago. Maximum journal age is 28800 seconds. 2022-11-01 10:28:56 notice : Removing stale journal file for monitor 'DatabaseMonitor'. 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-01 10:28:56 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-01 10:28:56 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 10:28:56 notice : [mariadbmon] Selecting new master server. 2022-11-01 10:28:56 notice : [mariadbmon] Setting 'cmkprod02-mdb1' as master. 2022-11-01 10:28:56 notice : Server changed state: cmkprod02-mdb52[10.0.3.68:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 10:28:56 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-01 10:28:56 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-01 10:28:56 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-01 10:28:56 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-01 10:28:56 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 10:28:56 notice : Server changed state: cmkprod02-mdb50[10.0.4.94:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 10:28:56 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-01 10:28:58 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 10:28:58 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-01 10:28:58 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-01 10:28:58 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-01 10:28:58 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-01 10:29:01 error : Monitor timed out when connecting to server cmkprod02-mdb51[10.0.3.177:3306] : 'Can't connect to MySQL server on '10.0.3.177' (110)' 2022-11-01 10:29:01 notice : Server changed state: cmkprod02-mdb51[10.0.3.177:3306]: server_down. [Running] -> [Down] 2022-11-01 10:29:07 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-01 10:29:07 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 10:29:07 notice : Server changed state: cmkprod02-mdb51[10.0.3.177:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-01 10:29:07 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-01 11:17:01 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 11:17:01 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-01 11:17:01 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-01 11:17:01 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-01 11:17:01 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-01 11:17:01 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-01 11:17:01 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 11:17:01 notice : Server changed state: cmkprod02-mdb60[10.0.4.102:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 11:17:01 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-01 11:17:25 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 11:17:25 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-01 11:17:25 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-01 11:17:25 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-01 11:17:25 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-01 11:17:28 error : Monitor timed out when connecting to server cmkprod02-mdb61[10.0.3.233:3306] : 'Can't connect to MySQL server on '10.0.3.233' (110)' 2022-11-01 11:17:28 notice : Server changed state: cmkprod02-mdb61[10.0.3.233:3306]: server_down. [Running] -> [Down] 2022-11-01 11:17:34 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-01 11:17:34 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 11:17:34 notice : Server changed state: cmkprod02-mdb61[10.0.3.233:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-01 11:17:34 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-01 15:35:00 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 15:35:00 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-01 15:35:00 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-01 15:35:00 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-01 15:35:00 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-01 15:35:00 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-01 15:35:00 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 15:35:00 notice : Server changed state: cmkprod02-mdb79[10.0.3.90:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 15:35:00 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-01 15:35:01 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 15:35:01 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-01 15:35:01 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-01 15:35:01 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-01 15:35:01 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-01 15:35:01 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-01 15:35:01 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 15:35:01 notice : Server changed state: cmkprod02-mdb77[10.0.3.66:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 15:35:01 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-01 15:35:15 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 15:35:15 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-01 15:35:15 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-01 15:35:15 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-01 15:35:15 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-01 15:35:15 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-01 15:35:15 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 15:35:15 notice : Server changed state: cmkprod02-mdb78[10.0.3.6:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 15:35:15 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-01 15:35:16 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 15:35:16 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-01 15:35:16 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-01 15:35:16 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-01 15:35:16 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-01 15:35:16 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-01 15:35:16 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 15:35:16 notice : Server changed state: cmkprod02-mdb75[10.0.4.167:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 15:35:16 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-01 15:35:24 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-01 15:35:24 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-01 15:35:24 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-01 15:35:24 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-01 15:35:24 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-01 15:35:24 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-01 15:35:24 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-01 15:35:24 notice : Server changed state: cmkprod02-mdb76[10.0.4.62:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-01 15:35:24 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-02 02:06:23 notice : Stopped MaxScale REST API 2022-11-02 02:06:23 notice : All workers have shut down. 2022-11-02 02:06:23 notice : MaxScale received signal SIGTERM. Exiting. 2022-11-02 02:06:23 notice : MaxScale is shutting down. 2022-11-02 02:06:23 notice : MaxScale shutdown completed. 2022-11-02 02:06:23 MariaDB MaxScale is shut down. ---------------------------------------------------- MariaDB MaxScale /var/log/maxscale/maxscale.log Wed Nov 2 02:06:47 2022 ---------------------------------------------------------------------------- 2022-11-02 02:06:47 notice : syslog logging is enabled. 2022-11-02 02:06:47 notice : maxlog logging is enabled. 2022-11-02 02:06:47 notice : Using up to 1.14GiB of memory for query classifier cache 2022-11-02 02:06:47 notice : Working directory: /var/log/maxscale 2022-11-02 02:06:47 notice : The collection of SQLite memory allocation statistics turned off. 2022-11-02 02:06:47 notice : Threading mode of SQLite set to Multi-thread. 2022-11-02 02:06:47 notice : MariaDB MaxScale 2.4.17 started (Commit: a8bfbbe254095d931aed351871fee72ce9ea645c) 2022-11-02 02:06:47 notice : MaxScale is running in process 771 2022-11-02 02:06:47 notice : Configuration file: /etc/maxscale.cnf 2022-11-02 02:06:47 notice : Log directory: /var/log/maxscale 2022-11-02 02:06:47 notice : Data directory: /var/lib/maxscale 2022-11-02 02:06:47 notice : Module directory: /usr/lib/x86_64-linux-gnu/maxscale 2022-11-02 02:06:47 notice : Service cache: /var/cache/maxscale 2022-11-02 02:06:47 notice : Worker message queue size: 1.00MiB 2022-11-02 02:06:47 notice : No query classifier specified, using default 'qc_sqlite'. 2022-11-02 02:06:47 notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libqc_sqlite.so 2022-11-02 02:06:47 notice : Query classification results are cached and reused. Memory used per thread: 292.78MiB 2022-11-02 02:06:47 notice : The systemd watchdog is Enabled. Internal timeout = 30s 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/Hint.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb3.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb51.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb11.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb4.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/SlaveOnlyListener.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb50.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb78.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/DatabaseMonitor.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/MasterOnlyService.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb60.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb61.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb75.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/ReadOnlyService.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/MasterOnlyListener.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/ReadOnlyListener.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb79.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb2.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/SlaveOnlyService.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb76.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb6.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb1.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb5.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb77.cnf. 2022-11-02 02:06:47 notice : Loading /etc/maxscale.cnf.d/cmkprod02-mdb52.cnf. 2022-11-02 02:06:47 notice : Runtime configuration changes have been done to MaxScale. Loading persisted configuration files and applying them on top of the main configuration file. These changes can override the values of the main configuration file: To revert them, remove all the files in '/var/lib/maxscale/maxscale.cnf.d'. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb3.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb51.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb11.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb4.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb50.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb78.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/DatabaseMonitor.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/MasterOnlyService.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb60.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb61.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb75.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/ReadOnlyService.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb79.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb2.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/SlaveOnlyService.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb76.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb6.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb1.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb5.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb77.cnf. 2022-11-02 02:06:47 notice : Loading /var/lib/maxscale/maxscale.cnf.d/cmkprod02-mdb52.cnf. 2022-11-02 02:06:47 notice : Loaded module MariaDBBackend: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackend.so 2022-11-02 02:06:47 notice : [readconnroute] Initialise readconnroute router module. 2022-11-02 02:06:47 notice : Loaded module readconnroute: V2.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libreadconnroute.so 2022-11-02 02:06:47 notice : Loaded module MariaDBClient: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbclient.so 2022-11-02 02:06:47 notice : [mariadbmon] Initialise the MariaDB Monitor module. 2022-11-02 02:06:47 notice : Loaded module mariadbmon: V1.5.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbmon.so 2022-11-02 02:06:47 notice : Loaded module hintfilter: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libhintfilter.so 2022-11-02 02:06:47 notice : Loaded module mariadbbackendauth: V1.0.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbbackendauth.so 2022-11-02 02:06:47 notice : Using encrypted passwords. Encryption key: '/var/lib/maxscale/.secrets'. 2022-11-02 02:06:47 notice : Loaded module mariadbauth: V1.1.0 from /usr/lib/x86_64-linux-gnu/maxscale/libmariadbauth.so 2022-11-02 02:06:47 notice : Started REST API on [127.0.0.1]:8989 2022-11-02 02:06:47 notice : MaxScale started with 4 worker threads, each with a stack size of 8388608 bytes. 2022-11-02 02:06:47 notice : Starting a total of 3 services... 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb11' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb11' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb2' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb2' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb3' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb3' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb1' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb1' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb4' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb4' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb5' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb5' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb6' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb6' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-02 02:06:47 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 02:06:47 notice : [MariaDBAuth] [MasterOnlyService] Loaded 89 MySQL users for listener 'MasterOnlyListener' from server 'cmkprod02-mdb50' with checksum 0x50249a7. 2022-11-02 02:06:47 error : (1) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:47 error : (1) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-02 02:06:47 notice : Listening for connections at [::]:3306 2022-11-02 02:06:47 notice : Service 'MasterOnlyService' started (1/3) 2022-11-02 02:06:47 error : (2) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:47 error : (2) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (3) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (3) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (4) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (4) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 notice : [MariaDBAuth] [SlaveOnlyService] Loaded 89 MySQL users for listener 'SlaveOnlyListener' from server 'cmkprod02-mdb50' with checksum 0x50249a7. 2022-11-02 02:06:48 notice : Listening for connections at [::]:3307 2022-11-02 02:06:48 notice : Service 'SlaveOnlyService' started (2/3) 2022-11-02 02:06:48 error : (5) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (5) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (6) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (6) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (7) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (8) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (8) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (7) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (9) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-02 02:06:48 error : (9) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-02 02:06:48 error : (10) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-02 02:06:48 error : (10) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-02 02:06:48 notice : [MariaDBAuth] [ReadOnlyService] Loaded 89 MySQL users for listener 'ReadOnlyListener' from server 'cmkprod02-mdb50' with checksum 0x50249a7. 2022-11-02 02:06:48 notice : Listening for connections at [::]:3308 2022-11-02 02:06:48 notice : Service 'ReadOnlyService' started (3/3) 2022-11-02 02:06:48 notice : Journal file was created 37884 seconds ago. Maximum journal age is 28800 seconds. 2022-11-02 02:06:48 notice : Removing stale journal file for monitor 'DatabaseMonitor'. 2022-11-02 02:06:48 notice : [mariadbmon] Selecting new master server. 2022-11-02 02:06:48 notice : [mariadbmon] Setting 'cmkprod02-mdb1' as master. 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: new_master. [Running] -> [Master, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb11[10.0.3.30:3307]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb2[10.0.4.69:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb3[10.0.3.223:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb4[10.0.4.123:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb5[10.0.3.41:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb6[10.0.3.13:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb52[10.0.3.68:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb50[10.0.4.94:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb51[10.0.3.177:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb60[10.0.4.102:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb61[10.0.3.233:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb79[10.0.3.90:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb77[10.0.3.66:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb78[10.0.3.6:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb75[10.0.4.167:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 02:06:48 notice : Server changed state: cmkprod02-mdb76[10.0.4.62:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 03:02:02 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-02 03:02:04 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-02 03:02:05 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-02 03:02:07 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-02 03:02:09 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-02 03:02:10 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-02 03:02:12 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-02 03:02:14 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-02 03:02:15 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-02 03:02:17 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-02 03:02:19 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-02 03:02:21 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-02 03:02:23 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-02 03:02:25 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-02 03:02:27 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-02 03:02:28 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-02 03:02:30 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-02 03:02:32 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-02 03:02:34 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-02 03:02:35 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-02 03:02:37 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-02 03:02:39 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-02 03:02:40 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-02 03:02:42 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-02 03:02:44 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-02 03:02:45 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-02 03:02:47 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-02 03:02:49 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-02 03:02:50 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-02 03:02:52 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-02 03:03:09 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:11 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:12 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:14 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:16 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:17 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:19 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:20 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:22 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:23 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 03:03:25 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.94:3306 2022-11-02 03:03:26 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.102:3306 2022-11-02 03:03:28 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.167:3306 2022-11-02 03:03:30 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.62:3306 2022-11-02 03:03:31 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.177:3306 2022-11-02 03:03:33 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.68:3306 2022-11-02 03:03:34 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.233:3306 2022-11-02 03:03:36 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.66:3306 2022-11-02 03:03:37 error : (93006) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb79' is in maintenance. 2022-11-02 03:03:37 error : (93006) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-02 03:03:37 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.6:3306 2022-11-02 03:03:39 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.90:3306 2022-11-02 10:29:09 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 10:29:09 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-02 10:29:09 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-02 10:29:09 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-02 10:29:09 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-02 10:29:09 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-02 10:29:09 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 10:29:09 notice : Server changed state: cmkprod02-mdb52[10.0.3.148:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 10:29:09 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-02 10:29:49 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-02 10:29:49 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-02 10:29:49 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 10:29:49 notice : Server changed state: cmkprod02-mdb51[10.0.3.69:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 10:29:49 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-02 10:29:49 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-02 10:29:49 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-02 10:29:49 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-02 10:29:49 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 10:29:49 notice : Server changed state: cmkprod02-mdb50[10.0.4.170:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 10:29:49 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-02 11:28:10 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 11:28:10 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-02 11:28:10 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-02 11:28:10 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-02 11:28:10 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-02 11:28:10 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-02 11:28:10 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 11:28:10 notice : Server changed state: cmkprod02-mdb60[10.0.4.235:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 11:28:10 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-02 11:28:50 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 11:28:50 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-02 11:28:50 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-02 11:28:50 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-02 11:28:50 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-02 11:28:50 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-02 11:28:50 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 11:28:50 notice : Server changed state: cmkprod02-mdb61[10.0.3.125:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 11:28:50 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-02 15:35:16 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 15:35:16 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-02 15:35:16 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-02 15:35:16 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-02 15:35:16 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-02 15:35:17 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-02 15:35:17 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 15:35:17 notice : Server changed state: cmkprod02-mdb76[10.0.4.65:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 15:35:17 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-02 15:35:29 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 15:35:29 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-02 15:35:29 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-02 15:35:29 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-02 15:35:29 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-02 15:35:29 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-02 15:35:29 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 15:35:30 notice : Server changed state: cmkprod02-mdb78[10.0.3.96:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 15:35:30 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-02 15:35:39 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 15:35:39 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-02 15:35:39 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-02 15:35:39 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-02 15:35:39 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-02 15:35:39 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-02 15:35:39 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 15:35:39 notice : Server changed state: cmkprod02-mdb79[10.0.3.135:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 15:35:39 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-02 15:35:43 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 15:35:43 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-02 15:35:43 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-02 15:35:43 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-02 15:35:43 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-02 15:35:43 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-02 15:35:43 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 15:35:43 notice : Server changed state: cmkprod02-mdb77[10.0.3.87:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 15:35:43 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-02 15:36:18 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-02 15:36:18 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-02 15:36:18 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-02 15:36:18 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-02 15:36:18 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-02 15:36:18 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-02 15:36:18 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-02 15:36:18 notice : Server changed state: cmkprod02-mdb75[10.0.4.200:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-02 15:36:18 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-02 18:04:19 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-02 18:31:35 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb52': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-02 20:02:03 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb52': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-02 21:01:33 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb79': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-02 23:08:48 error : [mariadbmon] Could not query events of 'cmkprod02-mdb2': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-02 23:08:50 error : [mariadbmon] Could not query events of 'cmkprod02-mdb2': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-02 23:08:52 error : [mariadbmon] Could not query events of 'cmkprod02-mdb2': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-03 03:01:39 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-03 03:01:40 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-03 03:01:42 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-03 03:01:44 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-03 03:01:45 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-03 03:01:47 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-03 03:01:49 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-03 03:01:50 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-03 03:01:52 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-03 03:01:54 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-03 03:01:55 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-03 03:01:57 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-03 03:01:59 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-03 03:02:00 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-03 03:02:02 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-03 03:02:04 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-03 03:02:05 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-03 03:02:07 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-03 03:02:09 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-03 03:02:10 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-03 03:02:12 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-03 03:02:13 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-03 03:02:15 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-03 03:02:17 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-03 03:02:19 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-03 03:02:20 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-03 03:02:22 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-03 03:02:24 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-03 03:02:25 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-03 03:02:27 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-03 03:02:45 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:47 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:48 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:50 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:53 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:56 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:02:59 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 03:03:01 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.170:3306 2022-11-03 03:03:02 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.235:3306 2022-11-03 03:03:04 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.200:3306 2022-11-03 03:03:05 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.65:3306 2022-11-03 03:03:07 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.69:3306 2022-11-03 03:03:08 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.148:3306 2022-11-03 03:03:10 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.125:3306 2022-11-03 03:03:11 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.87:3306 2022-11-03 03:03:13 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.96:3306 2022-11-03 03:03:14 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.135:3306 2022-11-03 10:30:11 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 10:30:11 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-03 10:30:11 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-03 10:30:11 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-03 10:30:11 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-03 10:30:11 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-03 10:30:11 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 10:30:11 notice : Server changed state: cmkprod02-mdb50[10.0.4.206:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 10:30:11 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-03 10:30:18 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 10:30:18 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-03 10:30:18 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-03 10:30:18 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-03 10:30:18 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-03 10:30:18 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-03 10:30:18 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 10:30:18 notice : Server changed state: cmkprod02-mdb52[10.0.3.40:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 10:30:18 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-03 10:30:27 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 10:30:27 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-03 10:30:27 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-03 10:30:27 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-03 10:30:27 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-03 10:30:27 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-03 10:30:27 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 10:30:27 notice : Server changed state: cmkprod02-mdb51[10.0.3.242:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 10:30:28 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-03 11:38:30 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 11:38:30 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-03 11:38:30 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-03 11:38:30 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-03 11:38:30 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-03 11:38:30 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-03 11:38:30 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 11:38:30 notice : Server changed state: cmkprod02-mdb61[10.0.3.38:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 11:38:30 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-03 11:38:58 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 11:38:58 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-03 11:38:58 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-03 11:38:58 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-03 11:38:58 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-03 11:38:58 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-03 11:38:58 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 11:38:58 notice : Server changed state: cmkprod02-mdb60[10.0.4.253:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 11:38:58 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-03 15:31:03 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 15:31:03 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-03 15:31:03 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-03 15:31:03 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-03 15:31:03 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-03 15:31:03 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-03 15:31:03 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 15:31:03 notice : Server changed state: cmkprod02-mdb76[10.0.4.231:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 15:31:03 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-03 15:32:08 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 15:32:08 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-03 15:32:08 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-03 15:32:08 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-03 15:32:08 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-03 15:32:08 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-03 15:32:08 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 15:32:09 notice : Server changed state: cmkprod02-mdb75[10.0.4.134:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 15:32:09 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-03 15:32:09 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 15:32:09 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-03 15:32:09 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-03 15:32:09 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-03 15:32:09 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-03 15:32:09 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-03 15:32:09 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 15:32:09 notice : Server changed state: cmkprod02-mdb79[10.0.3.18:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 15:32:10 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-03 15:32:23 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 15:32:23 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-03 15:32:23 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-03 15:32:23 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-03 15:32:23 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-03 15:32:23 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-03 15:32:23 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 15:32:23 notice : Server changed state: cmkprod02-mdb78[10.0.3.32:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 15:32:24 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-03 15:32:26 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-03 15:32:26 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-03 15:32:26 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-03 15:32:26 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-03 15:32:26 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-03 15:32:26 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-03 15:32:26 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-03 15:32:27 notice : Server changed state: cmkprod02-mdb77[10.0.3.26:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-03 15:32:27 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-03 22:29:26 warning: (8988525) [MariaDBAuth] MasterOnlyService: login attempt for user 'crossmark'@[::ffff:10.0.1.78]:54438 to database 'crossmark', authentication failed. Wrong password. 2022-11-04 00:20:22 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb6': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-04 03:01:43 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-04 03:01:45 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-04 03:01:46 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-04 03:01:48 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-04 03:01:50 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-04 03:01:52 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-04 03:01:53 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-04 03:01:55 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-04 03:01:57 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-04 03:01:58 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-04 03:02:00 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-04 03:02:02 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-04 03:02:03 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-04 03:02:05 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-04 03:02:07 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-04 03:02:08 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-04 03:02:10 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-04 03:02:11 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-04 03:02:13 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-04 03:02:15 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-04 03:02:16 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-04 03:02:18 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-04 03:02:20 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-04 03:02:21 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-04 03:02:23 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-04 03:02:24 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-04 03:02:26 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-04 03:02:27 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-04 03:02:29 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-04 03:02:31 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-04 03:02:40 error : (9593751) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb51' is in maintenance. 2022-11-04 03:02:40 error : (9593751) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-04 03:02:43 error : (9670964) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb77' is in maintenance. 2022-11-04 03:02:43 error : (9670964) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-04 03:02:48 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:49 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:51 error : (9672371) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb79' is in maintenance. 2022-11-04 03:02:51 error : (9672371) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-04 03:02:52 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:55 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:02:58 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:03:00 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:03:02 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 03:03:03 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.206:3306 2022-11-04 03:03:05 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.253:3306 2022-11-04 03:03:06 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.134:3306 2022-11-04 03:03:08 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.231:3306 2022-11-04 03:03:09 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.38:3306 2022-11-04 03:03:11 error : (9672126) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb79' is in maintenance. 2022-11-04 03:03:11 error : (9672126) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-04 03:03:11 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.242:3306 2022-11-04 03:03:12 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.40:3306 2022-11-04 03:03:14 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.26:3306 2022-11-04 03:03:15 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.32:3306 2022-11-04 03:03:17 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.18:3306 2022-11-04 10:42:41 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 10:42:41 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-04 10:42:41 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-04 10:42:41 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-04 10:42:41 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-04 10:42:41 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-04 10:42:41 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 10:42:41 notice : Server changed state: cmkprod02-mdb52[10.0.3.48:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 10:42:41 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-04 10:43:22 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-04 10:43:22 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-04 10:43:22 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 10:43:22 notice : Server changed state: cmkprod02-mdb51[10.0.3.205:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 10:43:22 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-04 10:43:22 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-04 10:43:22 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-04 10:43:22 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-04 10:43:22 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 10:43:22 notice : Server changed state: cmkprod02-mdb50[10.0.4.173:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 10:43:22 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-04 11:22:14 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 11:22:14 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-04 11:22:14 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-04 11:22:14 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-04 11:22:14 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-04 11:22:14 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-04 11:22:14 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 11:22:14 notice : Server changed state: cmkprod02-mdb60[10.0.4.189:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 11:22:14 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-04 11:22:32 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 11:22:32 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-04 11:22:32 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-04 11:22:32 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-04 11:22:32 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-04 11:22:32 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-04 11:22:32 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 11:22:32 notice : Server changed state: cmkprod02-mdb61[10.0.3.20:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 11:22:32 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-04 15:40:06 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 15:40:06 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-04 15:40:06 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-04 15:40:06 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-04 15:40:06 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-04 15:40:06 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-04 15:40:06 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 15:40:06 notice : Server changed state: cmkprod02-mdb78[10.0.3.56:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 15:40:06 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-04 15:40:08 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 15:40:08 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-04 15:40:08 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-04 15:40:08 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-04 15:40:08 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-04 15:40:08 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-04 15:40:08 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 15:40:08 notice : Server changed state: cmkprod02-mdb77[10.0.3.71:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 15:40:08 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-04 15:40:13 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 15:40:13 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-04 15:40:13 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-04 15:40:13 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-04 15:40:13 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-04 15:40:13 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-04 15:40:13 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 15:40:14 notice : Server changed state: cmkprod02-mdb79[10.0.3.171:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 15:40:14 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-04 15:40:31 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 15:40:31 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-04 15:40:31 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-04 15:40:31 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-04 15:40:31 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-04 15:40:31 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-04 15:40:31 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 15:40:31 notice : Server changed state: cmkprod02-mdb76[10.0.4.63:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 15:40:32 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-04 15:40:32 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-04 15:40:32 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-04 15:40:32 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-04 15:40:32 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-04 15:40:32 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-04 15:40:32 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-04 15:40:32 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-04 15:40:32 notice : Server changed state: cmkprod02-mdb75[10.0.4.148:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-04 15:40:32 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-04 17:52:52 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb5': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-04 22:53:13 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb60': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-04 22:53:21 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb50': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-05 03:01:46 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-05 03:01:48 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-05 03:01:49 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-05 03:01:51 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-05 03:01:53 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-05 03:01:55 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-05 03:01:56 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-05 03:01:58 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-05 03:01:59 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-05 03:02:01 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-05 03:02:03 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-05 03:02:05 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-05 03:02:06 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-05 03:02:08 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-05 03:02:10 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-05 03:02:11 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-05 03:02:13 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-05 03:02:15 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-05 03:02:16 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-05 03:02:18 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-05 03:02:19 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-05 03:02:21 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-05 03:02:23 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-05 03:02:24 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-05 03:02:26 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-05 03:02:27 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-05 03:02:29 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-05 03:02:31 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-05 03:02:32 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-05 03:02:34 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-05 03:02:47 error : (13288342) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb61' is in maintenance. 2022-11-05 03:02:47 error : (13288342) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-05 03:02:50 error : (14383736) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb60' is in maintenance. 2022-11-05 03:02:50 error : (14383736) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-05 03:02:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:02:52 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:02:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:02:55 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:02:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:02:58 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:03:00 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:03:02 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:03:03 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:03:05 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 03:03:06 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.173:3306 2022-11-05 03:03:08 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.189:3306 2022-11-05 03:03:09 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.148:3306 2022-11-05 03:03:11 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.63:3306 2022-11-05 03:03:12 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.205:3306 2022-11-05 03:03:14 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.48:3306 2022-11-05 03:03:16 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.20:3306 2022-11-05 03:03:17 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.71:3306 2022-11-05 03:03:19 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.56:3306 2022-11-05 03:03:20 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.171:3306 2022-11-05 10:37:41 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 10:37:41 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-05 10:37:41 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-05 10:37:41 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-05 10:37:41 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-05 10:37:41 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-05 10:37:41 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-05 10:37:41 notice : Server changed state: cmkprod02-mdb50[10.0.4.112:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-05 10:37:41 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-05 10:38:22 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 10:38:22 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-05 10:38:22 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-05 10:38:22 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-05 10:38:22 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-05 10:38:22 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-05 10:38:22 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-05 10:38:22 notice : Server changed state: cmkprod02-mdb52[10.0.3.107:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-05 10:38:22 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-05 10:38:30 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-05 10:38:30 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-05 10:38:30 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-05 10:38:30 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-05 10:38:30 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-05 10:38:33 error : Monitor timed out when connecting to server cmkprod02-mdb51[10.0.3.171:3306] : 'Can't connect to MySQL server on '10.0.3.171' (110)' 2022-11-05 10:38:33 notice : Server changed state: cmkprod02-mdb51[10.0.3.171:3306]: server_down. [Running] -> [Down] 2022-11-05 10:38:39 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-05 10:38:39 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-05 10:38:39 notice : Server changed state: cmkprod02-mdb51[10.0.3.171:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-05 10:38:39 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-06 01:01:19 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-06 01:01:21 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-06 01:01:23 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-06 01:01:24 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-06 01:01:26 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-06 01:01:28 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-06 01:01:29 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-06 01:01:31 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-06 01:01:33 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-06 01:01:35 error : (17892862) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb50' is in maintenance. 2022-11-06 01:01:35 error : (17892862) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-06 01:01:36 error : (17897078) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb51' is in maintenance. 2022-11-06 01:01:36 error : (17897078) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-06 01:01:36 error : (17903607) [readconnroute] Failed to route MySQL command 1 to backend server. Server 'cmkprod02-mdb50' is in maintenance. 2022-11-06 01:01:36 error : (17903607) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-06 01:01:39 error : (17913574) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb52' is in maintenance. 2022-11-06 01:01:39 error : (17913574) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-06 01:01:43 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-06 01:01:45 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-06 01:01:46 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-06 01:01:48 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.112:3306 2022-11-06 01:01:50 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.171:3306 2022-11-06 01:01:52 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.107:3306 2022-11-06 11:12:36 notice : Journal file was created 36649 seconds ago. Maximum journal age is 28800 seconds. 2022-11-06 11:12:36 notice : Removing stale journal file for monitor 'DatabaseMonitor'. 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-06 11:12:36 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-06 11:12:36 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-06 11:12:36 notice : [mariadbmon] Selecting new master server. 2022-11-06 11:12:36 notice : [mariadbmon] Setting 'cmkprod02-mdb1' as master. 2022-11-06 11:12:36 notice : Server changed state: cmkprod02-mdb50[10.0.4.52:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-06 11:12:36 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-06 11:12:36 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-06 11:12:36 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-06 11:12:36 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-06 11:12:36 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-06 11:12:36 notice : Server changed state: cmkprod02-mdb51[10.0.3.214:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-06 11:12:36 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-06 11:12:37 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-06 11:12:37 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-06 11:12:37 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-06 11:12:37 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-06 11:12:37 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-06 11:12:37 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-06 11:12:37 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-06 11:12:37 notice : Server changed state: cmkprod02-mdb52[10.0.3.46:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-06 11:12:37 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-07 02:01:27 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-07 02:01:29 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-07 02:01:31 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-07 02:01:32 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-07 02:01:34 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-07 02:01:35 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-07 02:01:37 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-07 02:01:39 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-07 02:01:40 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-07 02:01:45 error : (19990049) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb52' is in maintenance. 2022-11-07 02:01:45 error : (19990049) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 02:01:47 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 02:01:49 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 02:01:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 02:01:53 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.52:3306 2022-11-07 02:01:54 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.214:3306 2022-11-07 02:01:56 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.46:3306 2022-11-07 11:34:00 notice : Journal file was created 34329 seconds ago. Maximum journal age is 28800 seconds. 2022-11-07 11:34:00 notice : Removing stale journal file for monitor 'DatabaseMonitor'. 2022-11-07 11:34:00 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-07 11:34:00 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-07 11:34:00 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-07 11:34:00 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-07 11:34:00 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-07 11:34:00 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 11:34:00 notice : [mariadbmon] Selecting new master server. 2022-11-07 11:34:00 notice : [mariadbmon] Setting 'cmkprod02-mdb1' as master. 2022-11-07 11:34:00 notice : Server changed state: cmkprod02-mdb50[10.0.4.216:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 11:34:00 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-07 11:34:36 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 11:34:36 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-07 11:34:36 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-07 11:34:36 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-07 11:34:36 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-07 11:34:37 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-07 11:34:37 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 11:34:37 notice : Server changed state: cmkprod02-mdb52[10.0.3.220:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 11:34:37 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-07 11:34:55 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 11:34:55 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-07 11:34:55 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-07 11:34:55 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-07 11:34:55 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-07 11:34:55 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-07 11:34:55 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 11:34:55 notice : Server changed state: cmkprod02-mdb51[10.0.3.172:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 11:34:55 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-07 12:19:20 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 12:19:20 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-07 12:19:20 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-07 12:19:20 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-07 12:19:20 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-07 12:19:20 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-07 12:19:20 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 12:19:20 notice : Server changed state: cmkprod02-mdb61[10.0.3.248:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 12:19:20 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-07 12:19:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 12:19:51 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-07 12:19:51 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-07 12:19:51 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-07 12:19:51 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-07 12:19:51 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-07 12:19:51 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 12:19:51 notice : Server changed state: cmkprod02-mdb60[10.0.4.208:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 12:19:51 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-07 16:01:56 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb60': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:01:58 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb2': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:01:58 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb6': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:01:58 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb3': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:01 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb11': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:01 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb5': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:03 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb52': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:03 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb50': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:05 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb60': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:05 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb51': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:05 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb4': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:07 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb6': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:07 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb5': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:07 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb3': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:07 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb2': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:09 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb52': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:09 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb11': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:11 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb60': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:11 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb50': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:13 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb61': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:13 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb51': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:13 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb6': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:17 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@gtid_current_pos, @@gtid_binlog_pos;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-07 16:02:19 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb50': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:21 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb4': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:02:44 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110' 2022-11-07 16:02:44 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-07 16:02:44 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-07 16:02:44 error : (21687540) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687540) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687541) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687541) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687544) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687544) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687545) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687545) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687542) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687547) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687547) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687548) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687548) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687546) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687546) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687542) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687543) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:44 error : (21687543) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:44 error : (21687549) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:02:44 error : (21687549) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:02:45 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-07 16:02:45 notice : Server changed state: cmkprod02-mdb60[10.0.4.208:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:54 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110' 2022-11-07 16:02:54 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-07 16:02:54 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-07 16:02:56 error : (21688491) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688491) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688492) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688492) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688493) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688493) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688495) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688495) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688498) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688498) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688500) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688500) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688510) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688510) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688512) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688512) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688511) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:02:56 error : (21688511) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:02:56 error : (21688513) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:02:56 error : (21688513) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb11[10.0.3.30:3307]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb2[10.0.4.69:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb3[10.0.3.223:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb5[10.0.3.41:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb6[10.0.3.13:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb50[10.0.4.216:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb52[10.0.3.220:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb51[10.0.3.172:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:02:57 notice : Server changed state: cmkprod02-mdb61[10.0.3.248:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:03:00 notice : Server changed state: cmkprod02-mdb4[10.0.4.123:3306]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:03:06 error : [mariadbmon] 'cmkprod02-mdb1' does not have any slaves to promote. 2022-11-07 16:03:06 error : [mariadbmon] Could not autoselect promotion target for failover. 2022-11-07 16:03:06 error : [mariadbmon] Cluster gtid domain is unknown. This is usually caused by the cluster never having a master server while MaxScale was running. 2022-11-07 16:03:06 warning: [mariadbmon] Not performing automatic failover. Will keep retrying with most error messages suppressed. 2022-11-07 16:03:07 error : (21689048) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689048) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689049) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689049) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689050) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689050) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689051) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689051) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689052) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689052) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689053) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689053) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689054) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689054) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689055) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689055) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689056) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:03:07 error : (21689056) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. 2022-11-07 16:03:07 error : (21689057) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:03:07 error : (21689057) Failed to create new router session for service 'SlaveOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:03:08 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-07 16:04:38 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-07 16:05:35 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:05:50 error : [mariadbmon] Could not query events of 'cmkprod02-mdb1': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-07 16:07:14 error : [mariadbmon] Could not query events of 'cmkprod02-mdb1': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-07 16:08:14 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:08:53 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@gtid_current_pos, @@gtid_binlog_pos;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:09:20 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@gtid_current_pos, @@gtid_binlog_pos;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:09:44 warning: [mariadbmon] Error during monitor permissions test for server 'cmkprod02-mdb1': Query 'SHOW SLAVE STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:10:32 error : [mariadbmon] Could not query events of 'cmkprod02-mdb1': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-07 16:10:42 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:11:07 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-07 16:11:26 error : Failed to set latin1 character set: Lost connection to MySQL server during query 2022-11-07 16:11:26 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server during query' 2022-11-07 16:11:26 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-07 16:11:26 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-07 16:11:26 error : (21714596) Failed to write message to worker 2: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714596) Consider increasing pipe buffer size (sysctl fs.pipe-max-size) 2022-11-07 16:11:26 error : (21714652) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714415) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714611) Failed to write message to worker 2: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714653) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714417) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714466) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714419) Failed to write message to worker 0: 11, Resource temporarily unavailable (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:11:26 error : (21714661) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21714615) Failed to write message to worker 2: 11, Resource temporarily unavailable 2022-11-07 16:11:26 error : (21692562) Write to Client DCB ::ffff:10.0.2.242 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21700599) Write to Client DCB ::ffff:10.0.5.10 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21706035) Write to Client DCB ::ffff:10.0.2.242 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21698624) Write to Client DCB ::ffff:10.0.1.66 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21701024) Write to Client DCB ::ffff:10.0.2.130 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21702207) Write to Client DCB ::ffff:10.0.5.13 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21704592) Write to Client DCB ::ffff:10.0.1.66 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21706763) Write to Client DCB ::ffff:10.0.1.66 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21707952) Write to Client DCB ::ffff:10.0.1.78 in state DCB_STATE_POLLING failed: 104, Connection reset by peer (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:11:26 error : (21709339) Write to Client DCB ::ffff:10.0.5.10 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:11:26 error : (21714651) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714651) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714687) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714687) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714897) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714897) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714773) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714773) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21715082) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21715082) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714815) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714815) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714615) [readconnroute] Failed to route MySQL command 1 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714615) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714884) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714884) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714654) [readconnroute] Failed to route MySQL command 1 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714654) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714670) [readconnroute] Failed to route MySQL command 1 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714670) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714648) [readconnroute] Failed to route MySQL command 1 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714648) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714666) [readconnroute] Failed to route MySQL command 1 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714666) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714621) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:11:26 error : (21714621) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714703) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714671) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-07 16:11:26 error : (21714730) [mariadbclient] Routing the query failed. Session will be closed. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:11:26 error : (21714754) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21714746) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21715043) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-07 16:11:26 error : (21715213) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715213) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715214) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715214) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715215) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715215) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715216) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715216) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715217) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715217) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715218) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715218) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715219) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715219) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715220) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715220) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715221) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:11:26 error : (21715221) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:11:26 error : (21715222) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:11:26 error : (21715222) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:11:31 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-07 16:16:26 error : [mariadbmon] Could not query events of 'cmkprod02-mdb1': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-07 16:16:36 error : [mariadbmon] Could not query events of 'cmkprod02-mdb1': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-07 16:17:00 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@gtid_current_pos, @@gtid_binlog_pos;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:17:13 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:17:25 error : Failed to set latin1 character set: Lost connection to MySQL server during query 2022-11-07 16:17:25 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server during query' 2022-11-07 16:17:25 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-07 16:17:25 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-07 16:17:25 error : (21719716) Write to Client DCB ::ffff:10.0.1.66 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21721988) Write to Client DCB ::ffff:10.0.2.93 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21724518) Write to Client DCB ::ffff:10.0.2.52 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21725750) Write to Client DCB ::ffff:10.0.5.10 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21727121) Write to Client DCB ::ffff:10.0.1.66 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21716401) Write to Client DCB ::ffff:10.0.5.250 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21722026) Write to Client DCB ::ffff:10.0.5.250 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21723852) Write to Client DCB ::ffff:10.0.5.131 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21720931) Write to Client DCB ::ffff:10.0.1.66 in state DCB_STATE_POLLING failed: 104, Connection reset by peer 2022-11-07 16:17:25 error : (21726873) Write to Client DCB ::ffff:10.0.5.13 in state DCB_STATE_POLLING failed: 104, Connection reset by peer (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:17:25 error : (21732128) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732128) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732132) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732132) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732133) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732133) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732134) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732134) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732136) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732136) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732138) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732138) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732163) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732163) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732164) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732174) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-07 16:17:25 error : (21732174) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732179) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:17:25 error : (21732179) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-07 16:17:25 error : (21732180) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-07 16:17:29 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-07 16:17:46 notice : Server changed state: cmkprod02-mdb11[10.0.3.30:3307]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:17:50 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb11': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:18:53 notice : Server changed state: cmkprod02-mdb11[10.0.3.30:3307]: lost_slave. [Slave, Running] -> [Running] 2022-11-07 16:18:59 notice : Server changed state: cmkprod02-mdb11[10.0.3.30:3307]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:26:06 notice : Server changed state: cmkprod02-mdb2[10.0.4.69:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:26:11 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb2': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:26:18 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb2': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:26:21 notice : Server changed state: cmkprod02-mdb3[10.0.3.223:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:26:26 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb3': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:26:33 notice : Server changed state: cmkprod02-mdb5[10.0.3.41:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:26:38 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb5': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:26:49 notice : Server changed state: cmkprod02-mdb4[10.0.4.123:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:26:54 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb4': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:27:41 notice : Server changed state: cmkprod02-mdb6[10.0.3.13:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:27:46 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb6': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:28:14 notice : Server changed state: cmkprod02-mdb50[10.0.4.216:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:28:19 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb50': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:28:32 notice : Server changed state: cmkprod02-mdb51[10.0.3.172:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:28:37 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb51': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:29:04 notice : Server changed state: cmkprod02-mdb52[10.0.3.220:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:29:09 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb52': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:29:19 notice : Server changed state: cmkprod02-mdb60[10.0.4.208:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:29:25 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb60': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:29:33 notice : Server changed state: cmkprod02-mdb61[10.0.3.248:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:29:38 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb61': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-07 16:49:42 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 16:49:42 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-07 16:49:42 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-07 16:49:42 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-07 16:49:42 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-07 16:49:42 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-07 16:49:42 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 16:49:42 notice : Server changed state: cmkprod02-mdb75[10.0.4.43:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:49:42 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-07 16:49:55 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 16:49:55 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-07 16:49:55 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-07 16:49:55 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-07 16:49:55 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-07 16:49:55 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-07 16:49:55 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 16:49:57 notice : Server changed state: cmkprod02-mdb76[10.0.4.183:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:49:57 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-07 16:50:16 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 16:50:16 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-07 16:50:16 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-07 16:50:16 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-07 16:50:16 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-07 16:50:16 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-07 16:50:16 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 16:50:18 notice : Server changed state: cmkprod02-mdb77[10.0.3.194:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:50:18 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-07 16:50:18 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 16:50:18 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-07 16:50:18 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-07 16:50:18 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-07 16:50:18 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-07 16:50:18 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-07 16:50:18 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 16:50:18 notice : Server changed state: cmkprod02-mdb79[10.0.3.54:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:50:18 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-07 16:50:34 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-07 16:50:34 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-07 16:50:34 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-07 16:50:34 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-07 16:50:34 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-07 16:50:34 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-07 16:50:34 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-07 16:50:34 notice : Server changed state: cmkprod02-mdb78[10.0.3.161:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-07 16:50:34 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-08 02:01:21 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-08 02:01:23 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-08 02:01:24 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-08 02:01:26 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-08 02:01:28 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-08 02:01:29 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-08 02:01:31 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-08 02:01:33 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-08 02:01:34 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-08 02:01:36 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-08 02:01:37 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-08 02:01:39 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-08 02:01:41 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-08 02:01:42 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-08 02:01:44 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-08 02:01:45 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-08 02:01:47 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-08 02:01:49 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-08 02:01:50 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-08 02:01:52 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-08 02:01:53 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-08 02:01:55 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-08 02:01:57 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-08 02:01:58 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-08 02:02:00 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-08 02:02:01 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-08 02:02:03 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-08 02:02:04 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-08 02:02:06 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-08 02:02:08 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-08 02:02:21 error : (24061741) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb52' is in maintenance. 2022-11-08 02:02:21 error : (24061741) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-08 02:02:25 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:26 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:28 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:31 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:31 error : (24062039) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb79' is in maintenance. 2022-11-08 02:02:31 error : (24062039) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-08 02:02:32 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:34 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:35 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:37 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:38 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:40 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 02:02:44 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.208:3306 2022-11-08 02:02:45 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.216:3306 2022-11-08 02:02:47 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.43:3306 2022-11-08 02:02:48 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.183:3306 2022-11-08 02:02:50 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.248:3306 2022-11-08 02:02:51 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.172:3306 2022-11-08 02:02:53 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.220:3306 2022-11-08 02:02:54 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.194:3306 2022-11-08 02:02:56 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.161:3306 2022-11-08 02:02:57 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.54:3306 2022-11-08 11:29:10 notice : Journal file was created 33990 seconds ago. Maximum journal age is 28800 seconds. 2022-11-08 11:29:10 notice : Removing stale journal file for monitor 'DatabaseMonitor'. 2022-11-08 11:29:10 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-08 11:29:10 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-08 11:29:10 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-08 11:29:10 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-08 11:29:10 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-08 11:29:10 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 11:29:10 notice : [mariadbmon] Selecting new master server. 2022-11-08 11:29:10 notice : [mariadbmon] Setting 'cmkprod02-mdb1' as master. 2022-11-08 11:29:10 notice : Server changed state: cmkprod02-mdb50[10.0.4.211:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 11:29:10 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-08 11:29:41 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 11:29:41 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-08 11:29:41 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-08 11:29:41 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-08 11:29:41 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-08 11:29:41 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-08 11:29:41 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 11:29:41 notice : Server changed state: cmkprod02-mdb51[10.0.3.36:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 11:29:41 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-08 11:30:03 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 11:30:03 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-08 11:30:03 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-08 11:30:03 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-08 11:30:03 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-08 11:30:06 error : Monitor timed out when connecting to server cmkprod02-mdb52[10.0.3.242:3306] : 'Can't connect to MySQL server on '10.0.3.242' (110)' 2022-11-08 11:30:06 notice : Server changed state: cmkprod02-mdb52[10.0.3.242:3306]: server_down. [Running] -> [Down] 2022-11-08 11:30:12 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-08 11:30:12 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 11:30:12 notice : Server changed state: cmkprod02-mdb52[10.0.3.242:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-08 11:30:12 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-08 12:28:27 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 12:28:27 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-08 12:28:27 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-08 12:28:27 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-08 12:28:27 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-08 12:28:27 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-08 12:28:27 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 12:28:27 notice : Server changed state: cmkprod02-mdb60[10.0.4.31:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 12:28:27 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-08 12:28:29 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 12:28:29 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-08 12:28:29 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-08 12:28:29 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-08 12:28:29 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-08 12:28:29 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-08 12:28:29 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 12:28:29 notice : Server changed state: cmkprod02-mdb61[10.0.3.79:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 12:28:29 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-08 16:41:30 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 16:41:30 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-08 16:41:30 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-08 16:41:30 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-08 16:41:30 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-08 16:41:30 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-08 16:41:30 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 16:41:30 notice : Server changed state: cmkprod02-mdb76[10.0.4.42:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 16:41:30 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-08 16:41:58 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 16:41:58 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-08 16:41:58 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-08 16:41:58 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-08 16:41:58 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-08 16:41:58 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-08 16:41:58 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 16:41:58 notice : Server changed state: cmkprod02-mdb79[10.0.3.85:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 16:41:58 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-08 16:42:06 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 16:42:06 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-08 16:42:06 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-08 16:42:06 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-08 16:42:06 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-08 16:42:06 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-08 16:42:06 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 16:42:07 notice : Server changed state: cmkprod02-mdb75[10.0.4.77:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 16:42:07 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-08 16:42:17 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 16:42:17 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-08 16:42:17 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-08 16:42:17 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-08 16:42:17 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-08 16:42:20 error : Monitor timed out when connecting to server cmkprod02-mdb77[10.0.3.26:3306] : 'Can't connect to MySQL server on '10.0.3.26' (110)' 2022-11-08 16:42:20 notice : Server changed state: cmkprod02-mdb77[10.0.3.26:3306]: server_down. [Running] -> [Down] 2022-11-08 16:42:26 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-08 16:42:26 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 16:42:26 notice : Server changed state: cmkprod02-mdb77[10.0.3.26:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-08 16:42:26 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-08 16:42:59 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-08 16:42:59 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-08 16:42:59 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-08 16:42:59 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-08 16:42:59 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-08 16:42:59 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-08 16:42:59 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-08 16:43:00 notice : Server changed state: cmkprod02-mdb78[10.0.3.72:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-08 16:43:00 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-09 04:01:39 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-09 04:01:41 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-09 04:01:43 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-09 04:01:44 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-09 04:01:46 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-09 04:01:48 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-09 04:01:50 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-09 04:01:51 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-09 04:01:53 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-09 04:01:55 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-09 04:01:56 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-09 04:01:58 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-09 04:02:00 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-09 04:02:01 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-09 04:02:03 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-09 04:02:05 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-09 04:02:06 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-09 04:02:08 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-09 04:02:09 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-09 04:02:11 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-09 04:02:13 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-09 04:02:14 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-09 04:02:16 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-09 04:02:17 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-09 04:02:19 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-09 04:02:20 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-09 04:02:22 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-09 04:02:24 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-09 04:02:25 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-09 04:02:27 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-09 04:02:30 error : (28898693) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb60' is in maintenance. 2022-11-09 04:02:30 error : (28898693) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-09 04:02:36 error : (28902996) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb51' is in maintenance. 2022-11-09 04:02:36 error : (28902996) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-09 04:02:44 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:46 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:47 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:49 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:50 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:52 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:55 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:02:58 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 04:03:00 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.211:3306 2022-11-09 04:03:01 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.31:3306 2022-11-09 04:03:03 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.77:3306 2022-11-09 04:03:04 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.42:3306 2022-11-09 04:03:06 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.79:3306 2022-11-09 04:03:07 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.36:3306 2022-11-09 04:03:09 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.242:3306 2022-11-09 04:03:10 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.26:3306 2022-11-09 04:03:12 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.72:3306 2022-11-09 04:03:13 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.85:3306 2022-11-09 11:17:38 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 11:17:38 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-09 11:17:38 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-09 11:17:38 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-09 11:17:38 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-09 11:17:38 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-09 11:17:38 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 11:17:38 notice : Server changed state: cmkprod02-mdb50[10.0.4.71:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 11:17:38 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-09 11:17:39 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-09 11:17:39 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-09 11:17:39 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 11:17:39 notice : Server changed state: cmkprod02-mdb51[10.0.3.231:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 11:17:39 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-09 11:17:39 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-09 11:17:39 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-09 11:17:39 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-09 11:17:39 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 11:17:39 notice : Server changed state: cmkprod02-mdb52[10.0.3.67:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 11:17:39 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-09 12:23:41 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 12:23:41 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-09 12:23:41 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-09 12:23:41 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-09 12:23:41 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-09 12:23:41 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-09 12:23:41 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 12:23:41 notice : Server changed state: cmkprod02-mdb61[10.0.3.24:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 12:23:41 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-09 12:24:02 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 12:24:02 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-09 12:24:02 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-09 12:24:02 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-09 12:24:02 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-09 12:24:02 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-09 12:24:02 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 12:24:02 notice : Server changed state: cmkprod02-mdb60[10.0.4.205:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 12:24:02 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-09 16:37:45 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 16:37:45 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-09 16:37:45 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-09 16:37:45 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-09 16:37:45 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-09 16:37:45 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-09 16:37:45 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 16:37:45 notice : Server changed state: cmkprod02-mdb77[10.0.3.213:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 16:37:45 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-09 16:37:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 16:37:57 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-09 16:37:57 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-09 16:37:57 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-09 16:37:57 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-09 16:37:57 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-09 16:37:57 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 16:37:58 notice : Server changed state: cmkprod02-mdb78[10.0.3.244:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 16:37:58 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-09 16:38:00 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 16:38:00 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-09 16:38:00 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-09 16:38:00 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-09 16:38:00 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-09 16:38:00 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-09 16:38:00 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 16:38:00 notice : Server changed state: cmkprod02-mdb76[10.0.4.180:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 16:38:00 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-09 16:38:24 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 16:38:24 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-09 16:38:24 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-09 16:38:24 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-09 16:38:24 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-09 16:38:24 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-09 16:38:24 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 16:38:24 notice : Server changed state: cmkprod02-mdb75[10.0.4.223:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 16:38:24 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-09 16:38:29 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-09 16:38:29 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-09 16:38:29 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-09 16:38:29 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-09 16:38:29 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-09 16:38:29 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-09 16:38:29 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-09 16:38:29 notice : Server changed state: cmkprod02-mdb79[10.0.3.146:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-09 16:38:29 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-10 04:01:37 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-10 04:01:38 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-10 04:01:40 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-10 04:01:42 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-10 04:01:43 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-10 04:01:45 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-10 04:01:47 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-10 04:01:49 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-10 04:01:51 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-10 04:01:52 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-10 04:01:54 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-10 04:01:56 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-10 04:01:58 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-10 04:01:59 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-10 04:02:01 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-10 04:02:03 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-10 04:02:05 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-10 04:02:07 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-10 04:02:09 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-10 04:02:10 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-10 04:02:12 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-10 04:02:14 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-10 04:02:15 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-10 04:02:17 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-10 04:02:19 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-10 04:02:21 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-10 04:02:22 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-10 04:02:24 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-10 04:02:26 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-10 04:02:28 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-10 04:02:45 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:47 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:48 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:50 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:51 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:53 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:56 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:02:59 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 04:03:01 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.71:3306 2022-11-10 04:03:02 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.205:3306 2022-11-10 04:03:04 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.223:3306 2022-11-10 04:03:05 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.180:3306 2022-11-10 04:03:07 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.24:3306 2022-11-10 04:03:08 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.231:3306 2022-11-10 04:03:10 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.67:3306 2022-11-10 04:03:12 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.213:3306 2022-11-10 04:03:13 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.244:3306 2022-11-10 04:03:15 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.146:3306 2022-11-10 09:33:43 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110' 2022-11-10 09:33:43 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:33:43 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:33:43 error : (34449161) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449162) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449161) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449162) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449160) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449160) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449164) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449164) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449165) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449165) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449166) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449166) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449163) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449163) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449168) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449168) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449167) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:33:43 error : (34449167) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:33:43 error : (34449171) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:33:43 error : (34449171) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:33:45 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:34:06 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server during query' 2022-11-10 09:34:06 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:34:06 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:34:06 error : (34450671) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450671) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450672) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450672) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450674) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450674) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450677) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450677) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450678) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450678) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450685) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450685) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450686) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450686) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450688) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450687) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:06 error : (34450688) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450687) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:06 error : (34450689) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:06 error : (34450689) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:06 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:34:38 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server during query' 2022-11-10 09:34:38 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:34:38 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:34:38 error : (34453022) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453140) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453024) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453149) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453037) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453188) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453059) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453189) Failed to write message to worker 1: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453060) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-10 09:34:38 error : (34453277) Failed to write message to worker 3: 11, Resource temporarily unavailable (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:38 error : (34453210) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453210) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453253) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453253) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453267) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453267) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453266) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453266) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453255) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453255) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453306) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453306) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453328) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453328) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453326) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453326) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453277) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453277) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453271) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453339) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453339) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453343) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453343) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453344) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453344) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453365) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:34:38 error : (34453365) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453414) [readconnroute] Failed to route MySQL command 2 to backend server. Server 'cmkprod02-mdb1' is down. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:38 error : (34453414) [mariadbclient] Routing the query failed. Session will be closed. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:38 error : (34453271) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:34:38 error : (34453415) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453415) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453416) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453416) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453417) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453417) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453420) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453420) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453432) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453432) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453434) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453434) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453423) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453423) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453424) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453424) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453425) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:34:38 error : (34453425) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:34:38 error : (34453427) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:38 error : (34453427) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:34:41 notice : [mariadbmon] Slave 'cmkprod02-mdb11' is still connected to 'cmkprod02-mdb1' and received a new gtid or heartbeat event 3.0 seconds ago. Delaying failover for at least 7.0 seconds. 2022-11-10 09:34:44 notice : [mariadbmon] Slave 'cmkprod02-mdb11' is still connected to 'cmkprod02-mdb1' and received a new gtid or heartbeat event 6.0 seconds ago. Delaying failover for at least 4.0 seconds. 2022-11-10 09:34:46 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:34:50 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:35:12 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:35:24 error : An encrypted connection to 'cmkprod02-mdb1' could not be created, ensure that TLS is enabled on the target server. 2022-11-10 09:35:24 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110' 2022-11-10 09:35:24 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:35:24 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:35:24 error : (34458498) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458498) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458499) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458499) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458500) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458500) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458501) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458501) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458502) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458502) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458504) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458504) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458509) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458509) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458505) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458505) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458506) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:35:24 error : (34458506) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:35:24 error : (34458507) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:35:24 error : (34458507) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:35:27 notice : [mariadbmon] Slave 'cmkprod02-mdb11' is still connected to 'cmkprod02-mdb1' and received a new gtid or heartbeat event 3.0 seconds ago. Delaying failover for at least 7.0 seconds. 2022-11-10 09:35:30 notice : [mariadbmon] Slave 'cmkprod02-mdb11' is still connected to 'cmkprod02-mdb1' and received a new gtid or heartbeat event 3.0 seconds ago. Delaying failover for at least 7.0 seconds. 2022-11-10 09:35:30 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:36:00 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:36:17 error : Failed to set latin1 character set: Lost connection to MySQL server during query 2022-11-10 09:36:17 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server during query' 2022-11-10 09:36:17 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:36:17 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:36:17 error : (34463007) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb1' is down. 2022-11-10 09:36:17 error : (34463007) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 09:36:17 error : (34463194) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463251) Failed to write message to worker 0: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463203) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463206) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463214) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463221) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463223) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463230) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463241) Failed to write message to worker 3: 11, Resource temporarily unavailable 2022-11-10 09:36:17 error : (34463246) Failed to write message to worker 3: 11, Resource temporarily unavailable (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:36:17 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:36:30 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:38:09 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110' 2022-11-10 09:38:09 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:38:09 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:38:09 error : (34467875) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467875) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467876) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467876) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467879) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467879) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467877) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467877) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467881) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467881) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467883) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467883) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467880) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467880) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467884) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467884) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467878) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:38:09 error : (34467878) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:38:09 error : (34467885) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:38:09 error : (34467885) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:38:12 notice : [mariadbmon] Slave 'cmkprod02-mdb11' is still connected to 'cmkprod02-mdb1' and received a new gtid or heartbeat event 3.0 seconds ago. Delaying failover for at least 7.0 seconds. 2022-11-10 09:38:13 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:38:21 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:38:35 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@gtid_current_pos, @@gtid_binlog_pos;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:39:21 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110' 2022-11-10 09:39:21 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:39:21 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:39:21 error : (34471289) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471289) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471290) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471290) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471294) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471294) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471295) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471295) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471296) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471296) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471297) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471297) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471292) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471292) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471298) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471298) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471299) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:39:21 error : (34471299) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:39:21 error : (34471304) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:39:21 error : (34471304) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. (subsequent similar messages suppressed for 10000 milliseconds) 2022-11-10 09:39:22 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:39:33 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SELECT @@global.server_id, @@read_only, @@global.gtid_domain_id;' failed: 'Lost connection to MySQL server during query'. 2022-11-10 09:42:14 warning: [mariadbmon] Error during monitor permissions test for server 'cmkprod02-mdb1': Query 'SHOW SLAVE STATUS;' failed: 'Lost connection to MySQL server during query'. 2022-11-10 09:42:30 error : Monitor timed out when connecting to server cmkprod02-mdb1[10.0.3.120:3306] : 'Lost connection to MySQL server during query' 2022-11-10 09:42:30 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_down. [Master, Running] -> [Down] 2022-11-10 09:42:30 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2022-11-10 09:42:30 error : (34474649) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:42:30 error : (34474649) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:42:30 error : (34474650) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:42:30 error : (34474650) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:42:30 error : (34474651) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:42:30 error : (34474651) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:42:30 error : (34474652) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:42:30 error : (34474652) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:42:30 error : (34474653) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:42:30 error : (34474653) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:42:30 error : (34474655) [readconnroute] Failed to create new routing session. Couldn't find eligible candidate server. Freeing allocated resources. 2022-11-10 09:42:30 error : (34474655) Failed to create new router session for service 'MasterOnlyService'. See previous errors for more details. 2022-11-10 09:42:31 notice : Server changed state: cmkprod02-mdb1[10.0.3.120:3306]: master_up. [Down] -> [Master, Running] 2022-11-10 09:42:41 warning: [mariadbmon] Error during monitor update of server 'cmkprod02-mdb1': Query 'SHOW ALL SLAVES STATUS;' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. 2022-11-10 09:43:13 error : [mariadbmon] Could not query events of 'cmkprod02-mdb1': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server at 'handshake: reading initial communication packet', system error: 110'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-10 11:32:11 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 11:32:11 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-10 11:32:11 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-10 11:32:11 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-10 11:32:11 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-10 11:32:11 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-10 11:32:11 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 11:32:11 notice : Server changed state: cmkprod02-mdb50[10.0.4.174:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 11:32:11 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-10 11:32:28 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 11:32:28 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-10 11:32:28 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-10 11:32:28 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-10 11:32:28 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-10 11:32:28 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-10 11:32:28 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 11:32:28 notice : Server changed state: cmkprod02-mdb52[10.0.3.43:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 11:32:28 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-10 11:32:44 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 11:32:44 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-10 11:32:44 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-10 11:32:44 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-10 11:32:44 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-10 11:32:44 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-10 11:32:44 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 11:32:44 notice : Server changed state: cmkprod02-mdb51[10.0.3.219:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 11:32:44 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-10 12:39:50 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 12:39:50 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-10 12:39:50 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-10 12:39:50 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-10 12:39:50 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-10 12:39:50 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-10 12:39:50 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 12:39:50 notice : Server changed state: cmkprod02-mdb60[10.0.4.128:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 12:39:50 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'. 2022-11-10 12:41:10 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 12:41:10 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-10 12:41:10 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-10 12:41:10 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-10 12:41:10 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-10 12:41:10 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-10 12:41:10 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 12:41:10 notice : Server changed state: cmkprod02-mdb61[10.0.3.117:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 12:41:10 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-10 17:11:30 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 17:11:30 notice : Added server 'cmkprod02-mdb76' to monitor 'DatabaseMonitor' 2022-11-10 17:11:30 notice : Added server 'cmkprod02-mdb76' to service 'MasterOnlyService' 2022-11-10 17:11:30 notice : Added server 'cmkprod02-mdb76' to service 'ReadOnlyService' 2022-11-10 17:11:30 notice : Added server 'cmkprod02-mdb76' to service 'SlaveOnlyService' 2022-11-10 17:11:30 notice : Server 'cmkprod02-mdb76' charset: utf8mb4 2022-11-10 17:11:30 notice : Server 'cmkprod02-mdb76' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 17:11:30 notice : Server changed state: cmkprod02-mdb76[10.0.4.5:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 17:11:30 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb76'. 2022-11-10 17:11:57 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 17:11:57 notice : Added server 'cmkprod02-mdb77' to monitor 'DatabaseMonitor' 2022-11-10 17:11:57 notice : Added server 'cmkprod02-mdb77' to service 'MasterOnlyService' 2022-11-10 17:11:57 notice : Added server 'cmkprod02-mdb77' to service 'ReadOnlyService' 2022-11-10 17:11:57 notice : Added server 'cmkprod02-mdb77' to service 'SlaveOnlyService' 2022-11-10 17:11:57 notice : Server 'cmkprod02-mdb77' charset: utf8mb4 2022-11-10 17:11:57 notice : Server 'cmkprod02-mdb77' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 17:11:57 notice : Server changed state: cmkprod02-mdb77[10.0.3.62:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 17:11:57 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb77'. 2022-11-10 17:12:06 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 17:12:06 notice : Added server 'cmkprod02-mdb78' to monitor 'DatabaseMonitor' 2022-11-10 17:12:06 notice : Added server 'cmkprod02-mdb78' to service 'MasterOnlyService' 2022-11-10 17:12:06 notice : Added server 'cmkprod02-mdb78' to service 'ReadOnlyService' 2022-11-10 17:12:06 notice : Added server 'cmkprod02-mdb78' to service 'SlaveOnlyService' 2022-11-10 17:12:09 error : Monitor timed out when connecting to server cmkprod02-mdb78[10.0.3.18:3306] : 'Can't connect to MySQL server on '10.0.3.18' (110)' 2022-11-10 17:12:09 notice : Server changed state: cmkprod02-mdb78[10.0.3.18:3306]: server_down. [Running] -> [Down] 2022-11-10 17:12:15 notice : Server 'cmkprod02-mdb78' charset: utf8mb4 2022-11-10 17:12:15 notice : Server 'cmkprod02-mdb78' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 17:12:16 notice : Server changed state: cmkprod02-mdb78[10.0.3.18:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-10 17:12:16 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb78'. 2022-11-10 17:12:16 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 17:12:16 notice : Added server 'cmkprod02-mdb75' to monitor 'DatabaseMonitor' 2022-11-10 17:12:16 notice : Added server 'cmkprod02-mdb75' to service 'MasterOnlyService' 2022-11-10 17:12:16 notice : Added server 'cmkprod02-mdb75' to service 'ReadOnlyService' 2022-11-10 17:12:16 notice : Added server 'cmkprod02-mdb75' to service 'SlaveOnlyService' 2022-11-10 17:12:16 notice : Server 'cmkprod02-mdb75' charset: utf8mb4 2022-11-10 17:12:16 notice : Server 'cmkprod02-mdb75' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 17:12:16 notice : Server changed state: cmkprod02-mdb75[10.0.4.82:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 17:12:16 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb75'. 2022-11-10 17:12:17 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-10 17:12:17 notice : Added server 'cmkprod02-mdb79' to monitor 'DatabaseMonitor' 2022-11-10 17:12:17 notice : Added server 'cmkprod02-mdb79' to service 'MasterOnlyService' 2022-11-10 17:12:17 notice : Added server 'cmkprod02-mdb79' to service 'ReadOnlyService' 2022-11-10 17:12:17 notice : Added server 'cmkprod02-mdb79' to service 'SlaveOnlyService' 2022-11-10 17:12:17 notice : Server 'cmkprod02-mdb79' charset: utf8mb4 2022-11-10 17:12:17 notice : Server 'cmkprod02-mdb79' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-10 17:12:18 notice : Server changed state: cmkprod02-mdb79[10.0.3.80:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-10 17:12:18 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb79'. 2022-11-10 20:52:59 error : [mariadbmon] Could not query events of 'cmkprod02-mdb60': Query 'SELECT Event_schema, Event_name FROM information_schema.EVENTS WHERE Status = 'ENABLED';' failed: 'Lost connection to MySQL server during query'. Event handling can be disabled by setting 'handle_events' to false. 2022-11-10 22:13:46 error : (38237378) [mariadbbackend] Invalid authentication message from backend 'cmkprod02-mdb1'. Error code: 1045, Msg : #28000Access denied for user 'dkeyes'@'10.0.3.60' (using password: YES) 2022-11-10 22:13:46 notice : (38237378) [MariaDBAuth] [MasterOnlyService] Loaded 89 MySQL users for listener 'MasterOnlyListener' from server 'cmkprod02-mdb1' with checksum 0x73e688d6. 2022-11-10 22:13:46 error : (38237378) [mariadbbackend] Unable to write to backend 'cmkprod02-mdb1' due to authentication failure. Server in state Master, Running. 2022-11-10 22:13:46 error : (38237378) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 22:13:46 error : (38237405) [mariadbbackend] Invalid authentication message from backend 'cmkprod02-mdb1'. Error code: 1045, Msg : #28000Access denied for user 'dkeyes'@'10.0.3.60' (using password: YES) 2022-11-10 22:13:47 error : (38237430) [mariadbbackend] Invalid authentication message from backend 'cmkprod02-mdb1'. Error code: 1045, Msg : #28000Access denied for user 'dkeyes'@'10.0.3.60' (using password: YES) 2022-11-10 22:13:47 error : (38237430) [mariadbbackend] Unable to write to backend 'cmkprod02-mdb1' due to authentication failure. Server in state Master, Running. 2022-11-10 22:13:47 error : (38237430) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 22:13:47 warning: (38237471) [MasterOnlyService] Refresh rate limit (once every 30 seconds) exceeded for load of users' table. 2022-11-10 22:13:47 warning: (38237471) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53221, authentication failed. Wrong password. 2022-11-10 22:13:47 error : (38237495) [mariadbbackend] Invalid authentication message from backend 'cmkprod02-mdb1'. Error code: 1045, Msg : #28000Access denied for user 'dkeyes'@'10.0.3.60' (using password: YES) 2022-11-10 22:13:47 error : (38237495) [mariadbbackend] Unable to write to backend 'cmkprod02-mdb1' due to authentication failure. Server in state Master, Running. 2022-11-10 22:13:47 error : (38237495) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-10 22:13:48 warning: (38237523) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53223, authentication failed. Wrong password. 2022-11-10 22:13:48 warning: (38237546) [MasterOnlyService] Refresh rate limit (once every 30 seconds) exceeded for load of users' table. 2022-11-10 22:13:48 warning: (38237546) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53224, authentication failed. Wrong password. 2022-11-10 22:13:52 warning: (38237922) [MasterOnlyService] Refresh rate limit (once every 30 seconds) exceeded for load of users' table. 2022-11-10 22:13:52 warning: (38237922) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53231, authentication failed. Wrong password. 2022-11-10 22:13:53 warning: (38237946) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53232, authentication failed. Wrong password. 2022-11-10 22:14:09 warning: (38239331) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53240, authentication failed. Wrong password. 2022-11-10 22:14:09 warning: (38239354) [MariaDBAuth] MasterOnlyService: login attempt for user 'dkeyes'@[::ffff:44.225.235.30]:53241, authentication failed. Wrong password. 2022-11-11 04:01:56 notice : Removed server 'cmkprod02-mdb50' from service 'MasterOnlyService' 2022-11-11 04:01:58 notice : Removed server 'cmkprod02-mdb50' from service 'SlaveOnlyService' 2022-11-11 04:02:00 notice : Removed server 'cmkprod02-mdb50' from service 'ReadOnlyService' 2022-11-11 04:02:01 notice : Removed server 'cmkprod02-mdb60' from service 'MasterOnlyService' 2022-11-11 04:02:03 notice : Removed server 'cmkprod02-mdb60' from service 'SlaveOnlyService' 2022-11-11 04:02:05 notice : Removed server 'cmkprod02-mdb60' from service 'ReadOnlyService' 2022-11-11 04:02:06 notice : Removed server 'cmkprod02-mdb75' from service 'MasterOnlyService' 2022-11-11 04:02:08 notice : Removed server 'cmkprod02-mdb75' from service 'SlaveOnlyService' 2022-11-11 04:02:09 notice : Removed server 'cmkprod02-mdb75' from service 'ReadOnlyService' 2022-11-11 04:02:11 notice : Removed server 'cmkprod02-mdb76' from service 'MasterOnlyService' 2022-11-11 04:02:13 notice : Removed server 'cmkprod02-mdb76' from service 'SlaveOnlyService' 2022-11-11 04:02:14 notice : Removed server 'cmkprod02-mdb76' from service 'ReadOnlyService' 2022-11-11 04:02:16 notice : Removed server 'cmkprod02-mdb61' from service 'MasterOnlyService' 2022-11-11 04:02:18 notice : Removed server 'cmkprod02-mdb61' from service 'SlaveOnlyService' 2022-11-11 04:02:19 notice : Removed server 'cmkprod02-mdb61' from service 'ReadOnlyService' 2022-11-11 04:02:21 notice : Removed server 'cmkprod02-mdb51' from service 'MasterOnlyService' 2022-11-11 04:02:22 notice : Removed server 'cmkprod02-mdb51' from service 'SlaveOnlyService' 2022-11-11 04:02:24 notice : Removed server 'cmkprod02-mdb51' from service 'ReadOnlyService' 2022-11-11 04:02:26 notice : Removed server 'cmkprod02-mdb52' from service 'MasterOnlyService' 2022-11-11 04:02:27 notice : Removed server 'cmkprod02-mdb52' from service 'SlaveOnlyService' 2022-11-11 04:02:29 notice : Removed server 'cmkprod02-mdb52' from service 'ReadOnlyService' 2022-11-11 04:02:30 notice : Removed server 'cmkprod02-mdb77' from service 'MasterOnlyService' 2022-11-11 04:02:32 notice : Removed server 'cmkprod02-mdb77' from service 'SlaveOnlyService' 2022-11-11 04:02:33 notice : Removed server 'cmkprod02-mdb77' from service 'ReadOnlyService' 2022-11-11 04:02:35 notice : Removed server 'cmkprod02-mdb78' from service 'MasterOnlyService' 2022-11-11 04:02:36 notice : Removed server 'cmkprod02-mdb78' from service 'SlaveOnlyService' 2022-11-11 04:02:38 notice : Removed server 'cmkprod02-mdb78' from service 'ReadOnlyService' 2022-11-11 04:02:39 notice : Removed server 'cmkprod02-mdb79' from service 'MasterOnlyService' 2022-11-11 04:02:41 notice : Removed server 'cmkprod02-mdb79' from service 'SlaveOnlyService' 2022-11-11 04:02:42 notice : Removed server 'cmkprod02-mdb79' from service 'ReadOnlyService' 2022-11-11 04:02:52 error : (39319579) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb51' is in maintenance. 2022-11-11 04:02:52 error : (39319579) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-11 04:02:55 error : (39329032) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb77' is in maintenance. 2022-11-11 04:02:55 error : (39329032) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-11 04:03:00 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:01 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:01 error : (39408561) [readconnroute] Failed to route MySQL command 3 to backend server. Server 'cmkprod02-mdb52' is in maintenance. 2022-11-11 04:03:01 error : (39408561) [mariadbclient] Routing the query failed. Session will be closed. 2022-11-11 04:03:03 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:04 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:06 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:10 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:11 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:13 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:14 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:16 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 04:03:17 notice : Destroyed server 'cmkprod02-mdb50' at 10.0.4.174:3306 2022-11-11 04:03:19 notice : Destroyed server 'cmkprod02-mdb60' at 10.0.4.128:3306 2022-11-11 04:03:20 notice : Destroyed server 'cmkprod02-mdb75' at 10.0.4.82:3306 2022-11-11 04:03:22 notice : Destroyed server 'cmkprod02-mdb76' at 10.0.4.5:3306 2022-11-11 04:03:24 notice : Destroyed server 'cmkprod02-mdb61' at 10.0.3.117:3306 2022-11-11 04:03:25 notice : Destroyed server 'cmkprod02-mdb51' at 10.0.3.219:3306 2022-11-11 04:03:27 notice : Destroyed server 'cmkprod02-mdb52' at 10.0.3.43:3306 2022-11-11 04:03:28 notice : Destroyed server 'cmkprod02-mdb77' at 10.0.3.62:3306 2022-11-11 04:03:30 notice : Destroyed server 'cmkprod02-mdb78' at 10.0.3.18:3306 2022-11-11 04:03:31 notice : Destroyed server 'cmkprod02-mdb79' at 10.0.3.80:3306 2022-11-11 11:22:48 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 11:22:48 notice : Added server 'cmkprod02-mdb51' to monitor 'DatabaseMonitor' 2022-11-11 11:22:48 notice : Added server 'cmkprod02-mdb51' to service 'MasterOnlyService' 2022-11-11 11:22:48 notice : Added server 'cmkprod02-mdb51' to service 'ReadOnlyService' 2022-11-11 11:22:48 notice : Added server 'cmkprod02-mdb51' to service 'SlaveOnlyService' 2022-11-11 11:22:48 notice : Server 'cmkprod02-mdb51' charset: utf8mb4 2022-11-11 11:22:48 notice : Server 'cmkprod02-mdb51' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-11 11:22:48 notice : Server changed state: cmkprod02-mdb51[10.0.3.215:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-11 11:22:48 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb51'. 2022-11-11 11:22:56 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 11:22:56 notice : Added server 'cmkprod02-mdb50' to monitor 'DatabaseMonitor' 2022-11-11 11:22:56 notice : Added server 'cmkprod02-mdb50' to service 'MasterOnlyService' 2022-11-11 11:22:56 notice : Added server 'cmkprod02-mdb50' to service 'ReadOnlyService' 2022-11-11 11:22:56 notice : Added server 'cmkprod02-mdb50' to service 'SlaveOnlyService' 2022-11-11 11:22:56 notice : Server 'cmkprod02-mdb50' charset: utf8mb4 2022-11-11 11:22:56 notice : Server 'cmkprod02-mdb50' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-11 11:22:56 notice : Server changed state: cmkprod02-mdb50[10.0.4.45:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-11 11:22:56 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb50'. 2022-11-11 11:23:00 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 11:23:00 notice : Added server 'cmkprod02-mdb52' to monitor 'DatabaseMonitor' 2022-11-11 11:23:00 notice : Added server 'cmkprod02-mdb52' to service 'MasterOnlyService' 2022-11-11 11:23:00 notice : Added server 'cmkprod02-mdb52' to service 'ReadOnlyService' 2022-11-11 11:23:00 notice : Added server 'cmkprod02-mdb52' to service 'SlaveOnlyService' 2022-11-11 11:23:03 error : Monitor timed out when connecting to server cmkprod02-mdb52[10.0.3.146:3306] : 'Can't connect to MySQL server on '10.0.3.146' (110)' 2022-11-11 11:23:03 notice : Server changed state: cmkprod02-mdb52[10.0.3.146:3306]: server_down. [Running] -> [Down] 2022-11-11 11:23:09 notice : Server 'cmkprod02-mdb52' charset: utf8mb4 2022-11-11 11:23:09 notice : Server 'cmkprod02-mdb52' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-11 11:23:09 notice : Server changed state: cmkprod02-mdb52[10.0.3.146:3306]: slave_up. [Down] -> [Slave, Running] 2022-11-11 11:23:09 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb52'. 2022-11-11 12:30:54 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 12:30:54 notice : Added server 'cmkprod02-mdb61' to monitor 'DatabaseMonitor' 2022-11-11 12:30:54 notice : Added server 'cmkprod02-mdb61' to service 'MasterOnlyService' 2022-11-11 12:30:54 notice : Added server 'cmkprod02-mdb61' to service 'ReadOnlyService' 2022-11-11 12:30:54 notice : Added server 'cmkprod02-mdb61' to service 'SlaveOnlyService' 2022-11-11 12:30:54 notice : Server 'cmkprod02-mdb61' charset: utf8mb4 2022-11-11 12:30:54 notice : Server 'cmkprod02-mdb61' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-11 12:30:54 notice : Server changed state: cmkprod02-mdb61[10.0.3.37:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-11 12:30:54 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb61'. 2022-11-11 12:31:53 notice : Loaded server states from journal file: /var/lib/maxscale/DatabaseMonitor/monitor.dat 2022-11-11 12:31:53 notice : Added server 'cmkprod02-mdb60' to monitor 'DatabaseMonitor' 2022-11-11 12:31:53 notice : Added server 'cmkprod02-mdb60' to service 'MasterOnlyService' 2022-11-11 12:31:53 notice : Added server 'cmkprod02-mdb60' to service 'ReadOnlyService' 2022-11-11 12:31:53 notice : Added server 'cmkprod02-mdb60' to service 'SlaveOnlyService' 2022-11-11 12:31:53 notice : Server 'cmkprod02-mdb60' charset: utf8mb4 2022-11-11 12:31:53 notice : Server 'cmkprod02-mdb60' version: 10.4.24-15-MariaDB-enterprise-log 2022-11-11 12:31:53 notice : Server changed state: cmkprod02-mdb60[10.0.4.23:3306]: new_slave. [Running] -> [Slave, Running] 2022-11-11 12:31:54 notice : [mariadbmon] read_only set to ON on 'cmkprod02-mdb60'.