Details
-
Bug
-
Status: Closed (View Workflow)
-
Minor
-
Resolution: Fixed
-
2.1.13
-
None
-
ubuntu 14.04
Description
With long sessions applications, like zabbix, when master is going down, readwritesplit service correctly changes the backend server. But when master going back, readwritesplit service does not change master back and in log appears messages like:
error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
Example
Start server:
2018-01-16 12:05:43 notice : MaxScale started with 4 server threads. |
Shuting down master:
2018-01-16 12:14:23 error : Monitor was unable to connect to server [MASTER_IP_ADDRESS]:3306 : "Can't connect to MySQL server on 'MASTER_IP_ADDRESS' (107)" |
2018-01-16 12:14:23 notice : Server changed state: MASTER_SERVER[MASTER_IP_ADDRESS:3306]: master_down. [Master, Synced, Running] -> [Down] |
2018-01-16 12:14:23 notice : Server changed state: SLAVE_SERVER[SLAVE_IP_ADDRESS:3306]: new_master. [Slave, Synced, Running] -> [Master, Synced, Running] |
Master changed, no errors:
Start master:
2018-01-16 12:15:26 notice : Server changed state: MASTER_SERVER[MASTER_IP_ADDRESS:3306]: master_up. [Down] -> [Master, Synced, Running] |
2018-01-16 12:15:26 notice : Server changed state: SLAVE_SERVER[SLAVE_IP_ADDRESS:3306]: new_slave. [Master, Synced, Running] -> [Slave, Synced, Running] |
Errors in log appears:
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:26 error : (2831) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE (subsequent similar messages suppressed for 10000 milliseconds) |
2018-01-16 12:15:27 warning: (2595) [readwritesplit] [Splitter Service] Write query received from zabbix@::ffff:xxx.xxx.xxx.xxx. Could not find a valid master connection. Closing client connection. |
2018-01-16 12:15:27 error : (2595) [MySQLClient] Routing the query failed. Session will be closed. |
2018-01-16 12:15:30 warning: (2606) [readwritesplit] [Splitter Service] Write query received from zabbix@::ffff:xxx.xxx.xxx.xxx. Could not find a valid master connection. Closing client connection. |
2018-01-16 12:15:30 error : (2606) [MySQLClient] Routing the query failed. Session will be closed. |
2018-01-16 12:15:33 warning: (2599) [readwritesplit] [Splitter Service] Write query received from zabbix@::ffff:xxx.xxx.xxx.xxx. Could not find a valid master connection. Closing client connection. |
2018-01-16 12:15:33 error : (2599) [MySQLClient] Routing the query failed. Session will be closed. |
2018-01-16 12:15:38 error : (2675) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2573) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2573) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2600) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2600) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2585) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2585) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:39 error : (2585) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:40 error : (2605) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:40 error : (2571) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:40 error : (2571) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2648) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2626) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2648) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2626) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2648) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2626) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2648) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:41 error : (2626) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE (subsequent similar messages suppressed for 10000 milliseconds) |
2018-01-16 12:15:52 error : (2594) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2594) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2571) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2576) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2571) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2576) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2600) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:52 error : (2600) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
2018-01-16 12:15:53 error : (2600) [readwritesplit] Could not find master among the backend servers. Previous master's state : RUNNING SLAVE |
Аnd this errors continues to appear until the zabbix service reboots.
maxscale config:
[maxscale]
|
threads=4 |
|
|
[MASTER]
|
type=server
|
address=xxx.xxx.xxx.xxx
|
port=3306 |
protocol=MySQLBackend
|
priority=1 |
|
|
[SLAVE]
|
type=server
|
address=xxx.xxx.xxx.xxx
|
port=3306 |
protocol=MySQLBackend
|
priority=2 |
|
|
[SLAVE2]
|
type=server
|
address=xxx.xxx.xxx.xxx
|
port=3306 |
protocol=MySQLBackend
|
priority=3 |
|
|
[Galera Monitor]
|
type=monitor
|
module=galeramon
|
servers=MASTER,SLAVE,SLAVE2
|
user=xxx
|
passwd=yyy
|
monitor_interval=10000 |
available_when_donor=true |
use_priority=1 |
|
|
[Splitter Service]
|
type=service
|
router=readwritesplit
|
servers=MASTER, SLAVE2, SLAVE
|
user=xxx
|
passwd=yyy
|
max_slave_connections=1 |
router_options=master_accept_reads=true,slave_selection_criteria=LEAST_GLOBAL_CONNECTIONS |
use_sql_variables_in=master
|
|
|
[Splitter Listener]
|
type=listener
|
service=Splitter Service
|
protocol=MySQLClient
|
port=3306 |