MariaDB MaxScale /var/log/maxscale/maxscale.log Wed Oct 24 08:19:38 2018 ---------------------------------------------------------------------------- 2018-10-24 08:19:38 notice : Working directory: /var/log/maxscale 2018-10-24 08:19:38 notice : The collection of SQLite memory allocation statistics turned off. 2018-10-24 08:19:38 notice : Threading mode of SQLite set to Multi-thread. 2018-10-24 08:19:38 notice : MariaDB MaxScale 2.2.4 started 2018-10-24 08:19:38 notice : MaxScale is running in process 30995 2018-10-24 08:19:38 notice : Configuration file: /etc/maxscale.cnf 2018-10-24 08:19:38 notice : Log directory: /var/log/maxscale 2018-10-24 08:19:38 notice : Data directory: /var/lib/maxscale 2018-10-24 08:19:38 notice : Module directory: /usr/lib64/maxscale 2018-10-24 08:19:38 notice : Service cache: /var/cache/maxscale 2018-10-24 08:19:38 notice : Loading /etc/maxscale.cnf. 2018-10-24 08:19:38 notice : /etc/maxscale.cnf.d does not exist, not reading. 2018-10-24 08:19:38 notice : [mariadbmon] Initialise the MariaDB Monitor module. 2018-10-24 08:19:38 notice : Loaded module mariadbmon: V1.5.0 from /usr/lib64/maxscale/libmariadbmon.so 2018-10-24 08:19:38 notice : [cli] Initialise CLI router module 2018-10-24 08:19:38 notice : Loaded module cli: V1.0.0 from /usr/lib64/maxscale/libcli.so 2018-10-24 08:19:38 notice : [readwritesplit] Initializing statement-based read/write split router module. 2018-10-24 08:19:38 notice : Loaded module readwritesplit: V1.1.0 from /usr/lib64/maxscale/libreadwritesplit.so 2018-10-24 08:19:38 notice : [readconnroute] Initialise readconnroute router module. 2018-10-24 08:19:38 notice : Loaded module readconnroute: V1.1.0 from /usr/lib64/maxscale/libreadconnroute.so 2018-10-24 08:19:38 notice : Loaded module MariaDBBackend: V2.0.0 from /usr/lib64/maxscale/libmariadbbackend.so 2018-10-24 08:19:38 notice : Loaded module MySQLBackendAuth: V1.0.0 from /usr/lib64/maxscale/libmysqlbackendauth.so 2018-10-24 08:19:38 notice : Monitor 'ReplMonitor' is missing the 'journal_max_age' parameter, using default value of 28800 seconds. 2018-10-24 08:19:38 notice : Monitor 'ReplMonitor' is missing the 'script_timeout' parameter, using default value of 90 seconds. 2018-10-24 08:19:38 notice : Loaded module maxscaled: V2.0.0 from /usr/lib64/maxscale/libmaxscaled.so 2018-10-24 08:19:38 notice : Loaded module MaxAdminAuth: V2.1.0 from /usr/lib64/maxscale/libmaxadminauth.so 2018-10-24 08:19:38 notice : Loaded module MariaDBClient: V1.1.0 from /usr/lib64/maxscale/libmariadbclient.so 2018-10-24 08:19:38 notice : Loaded module MySQLAuth: V1.1.0 from /usr/lib64/maxscale/libmysqlauth.so 2018-10-24 08:19:38 notice : No query classifier specified, using default 'qc_sqlite'. 2018-10-24 08:19:38 notice : Loaded module qc_sqlite: V1.0.0 from /usr/lib64/maxscale/libqc_sqlite.so 2018-10-24 08:19:38 warning: Removing stale journal file for monitor 'ReplMonitor'. 2018-10-24 08:19:38 notice : Using encrypted passwords. Encryption key: '/var/lib/maxscale/.secrets'. 2018-10-24 08:19:38 notice : Starting a total of 3 services... 2018-10-24 08:19:38 notice : [MySQLAuth] [Read-Only-Service] Loaded 8 MySQL users for listener Read-Only-Listener. 2018-10-24 08:19:38 notice : Listening for connections at [::]:4008 with protocol MySQL 2018-10-24 08:19:38 notice : Service 'Read-Only-Service' started (1/3) 2018-10-24 08:19:38 notice : [MySQLAuth] [Read-Write-Service] Loaded 8 MySQL users for listener Read-Write-Listener. 2018-10-24 08:19:38 notice : Listening for connections at [::]:4006 with protocol MySQL 2018-10-24 08:19:38 notice : Service 'Read-Write-Service' started (2/3) 2018-10-24 08:19:38 notice : Listening for connections at [/tmp/maxadmin.sock]:0 with protocol MaxScale Admin 2018-10-24 08:19:38 notice : Service 'MaxAdmin-Service' started (3/3) 2018-10-24 08:19:38 notice : Started MaxScale log flusher. 2018-10-24 08:19:38 notice : Started REST API on [127.0.0.1]:8989 2018-10-24 08:19:38 notice : MaxScale started with 1 worker threads, each with a stack size of 8388608 bytes. 2018-10-24 08:19:38 notice : Server changed state: server1[INFDMYS001.example.gov.uk:3306]: new_master. [Running] -> [Master, Running] 2018-10-24 08:19:38 notice : Server changed state: server2[npedbmys001.example.gov.uk:3306]: new_slave. [Running] -> [Slave, Running] 2018-10-24 08:19:38 notice : [mariadbmon] A Master Server is now available: INFDMYS001.example.gov.uk:3306 2018-10-24 08:20:00 error : Monitor was unable to connect to server [INFDMYS001.example.gov.uk]:3306 : "Can't connect to MySQL server on 'INFDMYS001.example.gov.uk' (115)" 2018-10-24 08:20:00 notice : [mariadbmon] Server [INFDMYS001.example.gov.uk]:3306 lost the master status. 2018-10-24 08:20:00 notice : Server changed state: server1[INFDMYS001.example.gov.uk:3306]: master_down. [Master, Running] -> [Down] 2018-10-24 08:20:00 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2018-10-24 08:20:00 error : [mariadbmon] No Master can be determined. Last known was INFDMYS001.example.gov.uk:3306 2018-10-24 08:20:08 notice : [mariadbmon] Performing automatic failover to replace failed master 'server1'. 2018-10-24 08:20:08 notice : [mariadbmon] Promoting server 'server2' to master. 2018-10-24 08:20:08 notice : [mariadbmon] Redirecting slaves to new master. 2018-10-24 08:20:10 warning: [mariadbmon] Setting standalone master, server 'server2' is now the master. 2018-10-24 08:20:10 notice : Server changed state: server2[npedbmys001.example.gov.uk:3306]: new_master. [Slave, Running] -> [Master, Running] 2018-10-24 08:20:28 notice : Server changed state: server1[INFDMYS001.example.gov.uk:3306]: server_up. [Down] -> [Running] 2018-10-24 08:20:28 notice : [mariadbmon] Directing standalone server 'server1' to replicate from 'server2'. 2018-10-24 08:20:28 notice : [mariadbmon] 1 server(s) redirected or rejoined the cluster. 2018-10-24 08:20:30 notice : Server changed state: server1[INFDMYS001.example.gov.uk:3306]: new_slave. [Running] -> [Slave, Running] 2018-10-24 08:20:53 notice : (7) [mariadbmon] Stopped monitor ReplMonitor for the duration of failover. 2018-10-24 08:20:53 error : (7) [mariadbmon] Master server 'server2' is running. 2018-10-24 08:20:53 error : (7) [mariadbmon] Failover not allowed due to errors. 2018-10-24 08:20:53 notice : Loaded server states from journal file: /var/lib/maxscale/ReplMonitor/monitor.dat 2018-10-24 08:39:18 error : Monitor was unable to connect to server [npedbmys001.example.gov.uk]:3306 : "Can't connect to MySQL server on 'npedbmys001.example.gov.uk' (115)" 2018-10-24 08:39:18 notice : [mariadbmon] Server [npedbmys001.example.gov.uk]:3306 lost the master status. 2018-10-24 08:39:18 notice : Server changed state: server2[npedbmys001.example.gov.uk:3306]: master_down. [Master, Running] -> [Down] 2018-10-24 08:39:18 warning: [mariadbmon] Master has failed. If master status does not change in 4 monitor passes, failover begins. 2018-10-24 08:39:26 notice : [mariadbmon] Performing automatic failover to replace failed master 'server2'. 2018-10-24 08:39:26 notice : [mariadbmon] Promoting server 'server1' to master. 2018-10-24 08:39:26 notice : [mariadbmon] Redirecting slaves to new master. 2018-10-24 08:39:28 warning: [mariadbmon] Setting standalone master, server 'server1' is now the master. 2018-10-24 08:39:28 notice : Server changed state: server1[INFDMYS001.example.gov.uk:3306]: new_master. [Slave, Running] -> [Master, Running] 2018-10-24 08:39:52 notice : Server changed state: server2[npedbmys001.example.gov.uk:3306]: server_up. [Down] -> [Running] 2018-10-24 08:39:52 notice : [mariadbmon] Directing standalone server 'server2' to replicate from 'server1'. 2018-10-24 08:39:53 notice : [mariadbmon] 1 server(s) redirected or rejoined the cluster. 2018-10-24 08:39:55 notice : Server changed state: server2[npedbmys001.example.gov.uk:3306]: new_slave. [Running] -> [Slave, Running] 2018-10-24 08:48:26 notice : (13) [mariadbmon] Stopped monitor ReplMonitor for the duration of failover. 2018-10-24 08:48:26 error : (13) [mariadbmon] Master server 'server1' is running. 2018-10-24 08:48:26 error : (13) [mariadbmon] Failover not allowed due to errors. 2018-10-24 08:48:26 notice : Loaded server states from journal file: /var/lib/maxscale/ReplMonitor/monitor.dat 2018-10-24 08:56:14 notice : (15) [mariadbmon] Stopped the monitor ReplMonitor for the duration of switchover. 2018-10-24 08:56:14 notice : (15) [mariadbmon] Demoting server 'server1'. 2018-10-24 08:56:14 error : (15) [mariadbmon] Demotion failed due to a query error: 'Unknown error'. Query: 'FLUSH LOGS;'. 2018-10-24 08:56:14 error : (15) [mariadbmon] Switchover server1 -> server2 failed, failover has been disabled.. 2018-10-24 08:56:14 notice : Loaded server states from journal file: /var/lib/maxscale/ReplMonitor/monitor.dat