Calling external scripts on monitoring event is a good first step for failover.
It need additional feature to make the failover a 2 steps procedure:
- Many users would flavor a 2 step scenario
- Can be deliver to clients as default setup
- Removing ownership for failover tricky decision
- State = dead_master is subject to many false positive.
First reason to proceed like that is that switchover is most requested feature from users
- Potentially more frequently triggered when comparing it to hardware failure
- To upgrade hardware and software
- For rolling blocking database operations
Switchover is not control by monitoring but it's a user decision.
I would suggest that our first monitoring external script just send an email that :
- Warn the user to double check the reason the master is un-joinable
- Describe maxadmin command to failover
- List the current state of all backend
- Propose the best candidate for failover
This been done we need to map maxadmin cmd to external scripts
SWITCHOVER TO <backend_name>
FAILOVER TO <backend_name>
Following command pick the best candidate for the command
We need to add options in maxscale config file
for each backend
condidate_master=0 means the server can not be elected as potential master