Uploaded image for project: 'MariaDB MaxScale'
  1. MariaDB MaxScale
  2. MXS-3254

Monitor failover fails

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.5.5
    • Fix Version/s: Icebox
    • Component/s: mariadbmon
    • Labels:
      None

      Description

      The pinloki switchover test causes the monitor to fail as described below. Rare scenario, not likely to happen in the real world.

      11:38
      niclas The pinloki test in review revealed two monitor TODO:s. First, (which I think has come up before) the monitor deduces a replica is replicating from an "external" server by comparing IPs. So a server that is 127.0.0.1 can be external or internal depending on where the IP comes from, and how the monitor is configured. It should be consistent.
      11:40
      Second, if the sleep(5) in the test is replaced with test.maxscale().wait_monitor_ticks(5) the monitor ties itself in knots, and maxctrl becomes unresponsive.
      11:43
      esak The monitor gets stuck?
      11:45
      niclas Something goes awry and the monitor goes into a loop trying to STOP SLAVE, which fails.
      11:46
      I didn't look into it much, just noticing that something is messed up when the two scenarios play at the same time.
      11:47
      esak It's likely not an infinite loop, but depends on some timeout settings.
      11:47
      but why does "stop slave" fail?
      11:48
      niclas That's the part that needs to be dug into.
      11:50
      2020-10-22 10:49:20 warning: [mariadbmon] Query 'SET STATEMENT max_statement_time=3 FOR STOP SLAVE '';' failed on 'pinloki': 'Lost connection to MySQL server during query' (2013). Retrying with 86.9 seconds left.
      11:52
      esak could there be some weird deadlock where one thread cannot advance before the other? It's a bit weird since monitor runs in its own.
      11:52
      niclas I think it is something like that.

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            nantti Niclas Antti
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: