Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15933

Cannot resume Node SYNCED state when wsrep_desync is done after FTWRL

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.1, 10.2, 10.3.6
    • Fix Version/s: 10.1.36, 10.2.18, 10.3.10
    • Component/s: Galera
    • Labels:
      None
    • Environment:
      CentOS 7.4; Galera 2 Nodes

      Description

      Cannot resume Node SYNCED state when wsrep_desync is done after FTWRL

      1. Issue ' flush tables with read lock;' on Galera Node
      MariaDB [(none)]> flush tables with read lock;
      Query OK, 0 rows affected (0.000 sec)

      2. Set wsrep_desync – Command hangs and wsrep_desync=1 is not applied ,
      but afterwards cannot resume SYNCED state

      MariaDB [(none)]> set global wsrep_desync=1;
      ^CCtrl-C – query killed. Continuing normally.
      ^CCtrl-C – query killed. Continuing normally.
      ERROR 2013 (HY000): Lost connection to MySQL server during query
      MariaDB [(none)]> show processlist ;
      ERROR 2006 (HY000): MySQL server has gone away
      No connection. Trying to reconnect...
      Connection id: 81
      Current database: *** NONE ***

      ------------------------------------------------------------------------------------------------

      Id User Host db Command Time State Info Progress

      ------------------------------------------------------------------------------------------------

      1 system user   NULL Sleep 3331 WSREP aborter idle NULL 0.000
      2 system user   NULL Sleep 3331 NULL NULL 0.000
      4 system user   NULL Daemon NULL InnoDB purge coordinator NULL 0.000
      5 system user   NULL Daemon NULL InnoDB purge worker NULL 0.000
      3 system user   NULL Daemon NULL InnoDB purge worker NULL 0.000
      6 system user   NULL Daemon NULL InnoDB purge worker NULL 0.000
      7 system user   NULL Daemon NULL InnoDB shutdown handler NULL 0.000
      9 system user   NULL Sleep 3331 NULL NULL 0.000
      8 system user   NULL Sleep 3331 NULL NULL 0.000
      10 system user   NULL Sleep 3331 NULL NULL 0.000
      26 system user   NULL Sleep 3331 NULL NULL 0.000
      17 system user   NULL Sleep 3331 NULL NULL 0.000
      16 system user   NULL Sleep 3331 NULL NULL 0.000
      19 system user   NULL Sleep 3331 NULL NULL 0.000
      20 system user   NULL Sleep 3331 NULL NULL 0.000
      22 system user   NULL Sleep 3331 NULL NULL 0.000
      23 system user   NULL Sleep 3331 NULL NULL 0.000
      18 system user   NULL Sleep 3331 NULL NULL 0.000
      24 system user   NULL Sleep 3331 NULL NULL 0.000
      21 system user   NULL Sleep 3331 NULL NULL 0.000
      14 system user   NULL Sleep 3331 NULL NULL 0.000
      25 system user   NULL Sleep 3331 NULL NULL 0.000
      35 system user   NULL Sleep 3331 NULL NULL 0.000
      27 system user   NULL Sleep 3331 NULL NULL 0.000
      13 system user   NULL Sleep 3331 NULL NULL 0.000
      34 system user   NULL Sleep 3331 NULL NULL 0.000
      11 system user   NULL Sleep 3331 NULL NULL 0.000
      30 system user   NULL Sleep 3331 NULL NULL 0.000
      37 system user   NULL Sleep 3331 NULL NULL 0.000
      15 system user   NULL Sleep 3331 NULL NULL 0.000
      33 system user   NULL Sleep 3331 NULL NULL 0.000
      32 system user   NULL Sleep 3331 NULL NULL 0.000
      31 system user   NULL Sleep 3331 NULL NULL 0.000
      12 system user   NULL Sleep 3331 NULL NULL 0.000
      55 system user   NULL Sleep 3331 NULL NULL 0.000
      38 system user   NULL Sleep 3331 NULL NULL 0.000
      51 system user   NULL Sleep 3331 NULL NULL 0.000
      42 system user   NULL Sleep 3331 NULL NULL 0.000
      43 system user   NULL Sleep 3331 NULL NULL 0.000
      36 system user   NULL Sleep 3331 NULL NULL 0.000
      41 system user   NULL Sleep 3331 NULL NULL 0.000
      45 system user   NULL Sleep 3331 NULL NULL 0.000
      44 system user   NULL Sleep 3331 NULL NULL 0.000
      39 system user   NULL Sleep 3331 NULL NULL 0.000
      48 system user   NULL Sleep 3331 NULL NULL 0.000
      46 system user   NULL Sleep 3331 NULL NULL 0.000
      47 system user   NULL Sleep 3331 NULL NULL 0.000
      50 system user   NULL Sleep 3331 NULL NULL 0.000
      52 system user   NULL Sleep 3331 NULL NULL 0.000
      53 system user   NULL Sleep 3331 NULL NULL 0.000
      40 system user   NULL Sleep 3331 NULL NULL 0.000
      28 system user   NULL Sleep 3331 NULL NULL 0.000
      49 system user   NULL Sleep 3331 NULL NULL 0.000
      29 system user   NULL Sleep 3331 NULL NULL 0.000
      58 system user   NULL Sleep 3331 NULL NULL 0.000
      59 system user   NULL Sleep 3331 NULL NULL 0.000
      61 system user   NULL Sleep 3331 NULL NULL 0.000
      56 system user   NULL Sleep 3331 NULL NULL 0.000
      60 system user   NULL Sleep 3331 NULL NULL 0.000
      62 system user   NULL Sleep 3331 NULL NULL 0.000
      57 system user   NULL Sleep 3331 NULL NULL 0.000
      63 system user   NULL Sleep 3331 NULL NULL 0.000
      64 system user   NULL Sleep 3331 NULL NULL 0.000
      54 system user   NULL Sleep 3331 NULL NULL 0.000
      65 system user   NULL Sleep 3331 NULL NULL 0.000
      69 system user   NULL Sleep 3331 NULL NULL 0.000
      66 system user   NULL Sleep 3331 NULL NULL 0.000
      67 system user   NULL Sleep 3331 NULL NULL 0.000
      70 system user   NULL Sleep 3331 NULL NULL 0.000
      68 system user   NULL Sleep 3331 NULL NULL 0.000
      78 root localhost NULL Killed 30 NULL set global wsrep_desync=1 0.000
      81 root localhost NULL Query 0 Init show processlist 0.000

      ------------------------------------------------------------------------------------------------
      72 rows in set (0.001 sec)

      Node moved in DESYNCED state and cannot be recovered
      2018-04-19 17:34:17 0 [Note] WSREP: Member 0.0 (t4w6) desyncs itself from group
      2018-04-19 17:34:17 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 2)
      2018-04-19 17:34:17 78 [Note] WSREP: Provider paused at 9f23373a-3e6d-11e8-b625-0646494b946c:2 (31)

      MariaDB [(none)]> unlock tables ;
      Query OK, 0 rows affected (0.000 sec)

      MariaDB [(none)]> show variables like '%wsrep_desync%' ;
      --------------------+

      Variable_name Value

      --------------------+

      wsrep_desync OFF

      --------------------+
      1 row in set (0.002 sec)

      MariaDB [(none)]> show status like 'wsrep_local_state_comment' ;
      -----------------------------------------+

      Variable_name Value

      -----------------------------------------+

      wsrep_local_state_comment Donor/Desynced

      -----------------------------------------+
      1 row in set (0.001 sec)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jplindst Jan Lindström
              Reporter:
              winstone Zdravelina Sokolovska
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: