[#MXS-1262] Mantenance bit(s) should persist after maxscale restart

[MXS-1262] Mantenance bit(s) should persist after maxscale restart Created: 2017-05-10 Updated: 2017-08-16 Resolved: 2017-08-16
Status:	Closed
Project:	MariaDB MaxScale
Component/s:	maxadmin
Affects Version/s:	2.0.5, 2.1.2
Fix Version/s:	2.2.0

Type:

Bug

Priority:

Major

Reporter:

Andrii Nikitin (Inactive)

Assignee:

markus makela

Resolution:

Fixed

Votes:

0

Labels:

None

Sprint:

2017-39

Description

On practice it may be very unpleasant situation when maxscale service is restarted and server 'maintenance' bits are not preserved.
The restart may happen accidentally or because of some software/hardware problem or result of software bug or result of threat, etc

Thus server 'maintenance' flag must survive service restart.

MaxScale> list servers

Servers.

-------------------+-----------------+-------+-------------+--------------------

Server             | Address         | Port  | Connections | Status

-------------------+-----------------+-------+-------------+--------------------

server1            | 127.0.0.1       |  3307 |           0 | Maintenance, Master, Running

-------------------+-----------------+-------+-------------+--------------------

MaxScale> exit

$ killall maxscale

Shutting down MaxScale

$ bin/maxscale --basedir=$(pwd)

$ bin/maxadmin

MaxScale> list servers

Servers.

-------------------+-----------------+-------+-------------+--------------------

Server             | Address         | Port  | Connections | Status

-------------------+-----------------+-------+-------------+--------------------

server1            | 127.0.0.1       |  3307 |           0 | Master, Running

-------------------+-----------------+-------+-------------+--------------------

Comments

Comment by Dipti Joshi (Inactive) [ 2017-05-10 ]

anikitinPlease test this with 2.1.2 - With dynamic configuration features of 2.1 this should have been fixed there.

Comment by Andrii Nikitin (Inactive) [ 2017-05-10 ]

Thank you Dipti - I've tried 2.1.2 and it shows identical behavior.

2.1.2$ sudo bin/maxadmin -S /tmp/s1

Unable to connect to MaxScale at /tmp/s1: Connection refused

2.1.2$ bin/maxscale --version

MaxScale 2.1.2

2.1.2$ bin/maxscale --basedir=$(pwd)

2.1.2$ sudo bin/maxadmin -S /tmp/s1

MaxScale> list servers

Servers.

-------------------+-----------------+-------+-------------+--------------------

Server             | Address         | Port  | Connections | Status

-------------------+-----------------+-------+-------------+--------------------

server1            | 127.0.0.1       |  3307 |           0 | Master, Running

-------------------+-----------------+-------+-------------+--------------------

MaxScale> set server server1 maintenance

MaxScale> list servers

Servers.

-------------------+-----------------+-------+-------------+--------------------

Server             | Address         | Port  | Connections | Status

-------------------+-----------------+-------+-------------+--------------------

server1            | 127.0.0.1       |  3307 |           0 | Maintenance, Master, Running

-------------------+-----------------+-------+-------------+--------------------

MaxScale> exit

2.1.2$ ps auxw| grep maxscale

a         8852  0.0  0.0 296736  9652 ?        Ssl  20:39   0:01 bin/maxscale --basedir=/home/a/test1/mariadb-environs/_depot/s-tar/2.0.5

a         9795  0.0  0.0 303656 11136 ?        Ssl  21:39   0:00 bin/maxscale --basedir=/home/a/test1/mariadb-environs/_depot/s-tar/2.1.2

a         9808  0.0  0.0  14228   924 pts/22   S+   21:40   0:00 grep --color=auto maxscale

2.1.2$ kill 9795

Shutting down MaxScale

2.1.2$ bin/maxscale --basedir=$(pwd)

2.1.2$ sudo bin/maxadmin -S /tmp/s1

MaxScale> list servers

Servers.

-------------------+-----------------+-------+-------------+--------------------

Server             | Address         | Port  | Connections | Status

-------------------+-----------------+-------+-------------+--------------------

server1            | 127.0.0.1       |  3307 |           0 | Master, Running

-------------------+-----------------+-------+-------------+--------------------

Comment by markus makela [ 2017-05-11 ]

This has been partially fixed for 2.2 where the MySQL monitor module now stores a persistent copy of the server states on disk. This prevents the loss of information that happens when MaxScale is restarted. We'll probably extend this functionality to a generic monitor feature for 2.2.

Comment by Andrii Nikitin (Inactive) [ 2017-05-11 ]

This feature is very rarely needed, but considering amount of damage it may lead to: I'd give it top priority in every future 2.0 / 2.1 release where possible .

Also in my understanding 'Maintenance' status is conceptually different from other states like 'Master' and 'Running', etc. thus should be treated differently.

I.e. we cannot be fully sure that Server is still 'Running': even if we just checked that it was OK - we should be ready that some problem may occur any moment.
'Maintenance' is completely different: once Server is marked as 'in maintenance' - it must remain marked 'in maintenance' no matter what happens in system (until admin explicitly removes it).

Comment by markus makela [ 2017-05-11 ]

With 2.1, this can be emulated by removing the server from the service.

Comment by markus makela [ 2017-08-10 ]

Moved the code that persists the server state into the core and took it into use in all monitors.

Generated at Thu Feb 08 04:05:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.