Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-25

bugzillaId-656: MySQL Monitor: claims that Master is available after master failure

Details

    • Bug
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Fixed
    • 1.0.0
    • 1.1.1
    • mariadbmon, Plugin
    • None
    • Linux

    Description

      This is imported from bugzilla item: http://bugs.mariadb.com/show_bug.cgi?id=656

      Description Vilho Raatikka 2014-12-18 15:16:42 UTC
      How to reproduce:
      1. start rwsplit with a master and optionally with slaves
      2. prevent connections to master by executing
      iptables -I INPUT -p tcp --dport 3000 -j REJECT --reject-with icmp-admin-prohibited
      3. read the message log:

      (..errors..)
      2014-12-18 17:09:30 Server 127.0.0.1:3000 lost the master status.
      2014-12-18 17:09:30 Info: A Master Server is now available: 127.0.0.1:3000
      2014-12-18 17:10:35 Info: A Master Server is now available: 127.0.0.1:3000

      Attachments

        Activity

          This is comment history from bugzilla:

          Comment 1 Mark Riddoch 2015-02-13 10:28:46 UTC
          Add as test case and retest

          Comment 2 Timofey Turenko 2015-03-02 10:42:34 UTC
          Error log says:

          2015-03-02 12:36:31 Error : Monitor was unable to connect to server 192.168.122.141:3306 : "Can't connect to MySQL server on '192.168.122.141'
          (111)"

          Message log says:

          2015-03-02 12:36:31 Server 192.168.122.141:3306 lost the master status.
          2015-03-02 12:36:31 Info : A Master Server is now available: 192.168.122.141:3306

          MaxAdmin says:

           
          Server 0x26e71c0 (server1)
          	Server:				192.168.122.141
          	Status:               		Master, Down
          	Protocol:			MySQLBackend
          	Port:				3306
          	Server Version:			5.5.42-MariaDB-log
          	Node Id:			141
          	Master Id:			-1
          	Slave Ids:			142, 143 , 144 
          	Repl Depth:			0
          	Number of connections:		0
          	Current no. of conns:		0
          	Current no. of operations:	0

          So, the problem is visible only in the messages log. Is it a real problem or just logging issue?

          dshjoshi Dipti Joshi (Inactive) added a comment - This is comment history from bugzilla: Comment 1 Mark Riddoch 2015-02-13 10:28:46 UTC Add as test case and retest Comment 2 Timofey Turenko 2015-03-02 10:42:34 UTC Error log says: 2015-03-02 12:36:31 Error : Monitor was unable to connect to server 192.168.122.141:3306 : "Can't connect to MySQL server on '192.168.122.141' (111)" Message log says: 2015-03-02 12:36:31 Server 192.168.122.141:3306 lost the master status. 2015-03-02 12:36:31 Info : A Master Server is now available: 192.168.122.141:3306 MaxAdmin says:   Server 0x26e71c0 (server1) Server: 192.168.122.141 Status: Master, Down Protocol: MySQLBackend Port: 3306 Server Version: 5.5.42-MariaDB-log Node Id: 141 Master Id: -1 Slave Ids: 142, 143 , 144 Repl Depth: 0 Number of connections: 0 Current no. of conns: 0 Current no. of operations: 0 So, the problem is visible only in the messages log. Is it a real problem or just logging issue?
          markus makela markus makela added a comment -

          MySQL monitor wasn't checking if the server was running when generating the Info message about new master.

          markus makela markus makela added a comment - MySQL monitor wasn't checking if the server was running when generating the Info message about new master.

          checked manually:

          2015-05-06 15:46:26 MaxScale started with 8 server threads.
          2015-05-06 15:46:26 Info : A Master Server is now available: 192.168.122.161:3306
          2015-05-06 15:46:36 Info : A Master Server is now available: 192.168.122.161:3306
          2015-05-06 15:46:43 Loaded module MySQLBackend: V2.0.0 from /usr/local/mariadb-maxscale/modules/libMySQLBackend.so
          2015-05-06 15:46:43 Loaded module qlafilter: V1.1.1 from /usr/local/mariadb-maxscale/modules/libqlafilter.so
          2015-05-06 15:46:46 Info : A Master Server is now available: 192.168.122.161:3306
          2015-05-06 15:46:56 Info : A Master Server is now available: 192.168.122.161:3306
          2015-05-06 15:47:07 Info : A Master Server is now available: 192.168.122.161:3306

          block master:

          2015-05-06 15:47:17 Server 192.168.122.161:3306 lost the master status.
          2015-05-06 15:49:29 Server 192.168.122.161:3306 lost the master status.

          unblock master:
          2015-05-06 15:50:40 Info : A Master Server is now available: 192.168.122.161:3306
          2015-05-06 15:50:50 Info : A Master Server is now available: 192.168.122.161:3306

          block again: nothing in the lmessage log, but clear message in error log

          2015-05-06 15:51:00 Error : Monitor was unable to connect to server 192.168.122.161:3306 : "Can't connect to MySQL server on '192.168.122.161' (113)"

          I guees this can be closed now

          tturenko Timofey Turenko added a comment - checked manually: 2015-05-06 15:46:26 MaxScale started with 8 server threads. 2015-05-06 15:46:26 Info : A Master Server is now available: 192.168.122.161:3306 2015-05-06 15:46:36 Info : A Master Server is now available: 192.168.122.161:3306 2015-05-06 15:46:43 Loaded module MySQLBackend: V2.0.0 from /usr/local/mariadb-maxscale/modules/libMySQLBackend.so 2015-05-06 15:46:43 Loaded module qlafilter: V1.1.1 from /usr/local/mariadb-maxscale/modules/libqlafilter.so 2015-05-06 15:46:46 Info : A Master Server is now available: 192.168.122.161:3306 2015-05-06 15:46:56 Info : A Master Server is now available: 192.168.122.161:3306 2015-05-06 15:47:07 Info : A Master Server is now available: 192.168.122.161:3306 block master: 2015-05-06 15:47:17 Server 192.168.122.161:3306 lost the master status. 2015-05-06 15:49:29 Server 192.168.122.161:3306 lost the master status. unblock master: 2015-05-06 15:50:40 Info : A Master Server is now available: 192.168.122.161:3306 2015-05-06 15:50:50 Info : A Master Server is now available: 192.168.122.161:3306 block again: nothing in the lmessage log, but clear message in error log 2015-05-06 15:51:00 Error : Monitor was unable to connect to server 192.168.122.161:3306 : "Can't connect to MySQL server on '192.168.122.161' (113)" I guees this can be closed now

          verified by manual checking of error log when Master node is blocked by firewall

          tturenko Timofey Turenko added a comment - verified by manual checking of error log when Master node is blocked by firewall

          People

            markus makela markus makela
            tturenko Timofey Turenko
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.