Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15933

Cannot resume Node SYNCED state when wsrep_desync is done after FTWRL

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 10.3.6, 10.1(EOL), 10.2(EOL)
    • 10.1.36, 10.2.18, 10.3.10
    • Galera
    • None
    • CentOS 7.4; Galera 2 Nodes

    Description

      Cannot resume Node SYNCED state when wsrep_desync is done after FTWRL

      1. Issue ' flush tables with read lock;' on Galera Node
      MariaDB [(none)]> flush tables with read lock;
      Query OK, 0 rows affected (0.000 sec)

      2. Set wsrep_desync – Command hangs and wsrep_desync=1 is not applied ,
      but afterwards cannot resume SYNCED state

      MariaDB [(none)]> set global wsrep_desync=1;
      ^CCtrl-C – query killed. Continuing normally.
      ^CCtrl-C – query killed. Continuing normally.
      ERROR 2013 (HY000): Lost connection to MySQL server during query
      MariaDB [(none)]> show processlist ;
      ERROR 2006 (HY000): MySQL server has gone away
      No connection. Trying to reconnect...
      Connection id: 81
      Current database: *** NONE ***

      ------------------------------------------------------------------------------------------------

      Id User Host db Command Time State Info Progress

      ------------------------------------------------------------------------------------------------

      1 system user   NULL Sleep 3331 WSREP aborter idle NULL 0.000
      2 system user   NULL Sleep 3331 NULL NULL 0.000
      4 system user   NULL Daemon NULL InnoDB purge coordinator NULL 0.000
      5 system user   NULL Daemon NULL InnoDB purge worker NULL 0.000
      3 system user   NULL Daemon NULL InnoDB purge worker NULL 0.000
      6 system user   NULL Daemon NULL InnoDB purge worker NULL 0.000
      7 system user   NULL Daemon NULL InnoDB shutdown handler NULL 0.000
      9 system user   NULL Sleep 3331 NULL NULL 0.000
      8 system user   NULL Sleep 3331 NULL NULL 0.000
      10 system user   NULL Sleep 3331 NULL NULL 0.000
      26 system user   NULL Sleep 3331 NULL NULL 0.000
      17 system user   NULL Sleep 3331 NULL NULL 0.000
      16 system user   NULL Sleep 3331 NULL NULL 0.000
      19 system user   NULL Sleep 3331 NULL NULL 0.000
      20 system user   NULL Sleep 3331 NULL NULL 0.000
      22 system user   NULL Sleep 3331 NULL NULL 0.000
      23 system user   NULL Sleep 3331 NULL NULL 0.000
      18 system user   NULL Sleep 3331 NULL NULL 0.000
      24 system user   NULL Sleep 3331 NULL NULL 0.000
      21 system user   NULL Sleep 3331 NULL NULL 0.000
      14 system user   NULL Sleep 3331 NULL NULL 0.000
      25 system user   NULL Sleep 3331 NULL NULL 0.000
      35 system user   NULL Sleep 3331 NULL NULL 0.000
      27 system user   NULL Sleep 3331 NULL NULL 0.000
      13 system user   NULL Sleep 3331 NULL NULL 0.000
      34 system user   NULL Sleep 3331 NULL NULL 0.000
      11 system user   NULL Sleep 3331 NULL NULL 0.000
      30 system user   NULL Sleep 3331 NULL NULL 0.000
      37 system user   NULL Sleep 3331 NULL NULL 0.000
      15 system user   NULL Sleep 3331 NULL NULL 0.000
      33 system user   NULL Sleep 3331 NULL NULL 0.000
      32 system user   NULL Sleep 3331 NULL NULL 0.000
      31 system user   NULL Sleep 3331 NULL NULL 0.000
      12 system user   NULL Sleep 3331 NULL NULL 0.000
      55 system user   NULL Sleep 3331 NULL NULL 0.000
      38 system user   NULL Sleep 3331 NULL NULL 0.000
      51 system user   NULL Sleep 3331 NULL NULL 0.000
      42 system user   NULL Sleep 3331 NULL NULL 0.000
      43 system user   NULL Sleep 3331 NULL NULL 0.000
      36 system user   NULL Sleep 3331 NULL NULL 0.000
      41 system user   NULL Sleep 3331 NULL NULL 0.000
      45 system user   NULL Sleep 3331 NULL NULL 0.000
      44 system user   NULL Sleep 3331 NULL NULL 0.000
      39 system user   NULL Sleep 3331 NULL NULL 0.000
      48 system user   NULL Sleep 3331 NULL NULL 0.000
      46 system user   NULL Sleep 3331 NULL NULL 0.000
      47 system user   NULL Sleep 3331 NULL NULL 0.000
      50 system user   NULL Sleep 3331 NULL NULL 0.000
      52 system user   NULL Sleep 3331 NULL NULL 0.000
      53 system user   NULL Sleep 3331 NULL NULL 0.000
      40 system user   NULL Sleep 3331 NULL NULL 0.000
      28 system user   NULL Sleep 3331 NULL NULL 0.000
      49 system user   NULL Sleep 3331 NULL NULL 0.000
      29 system user   NULL Sleep 3331 NULL NULL 0.000
      58 system user   NULL Sleep 3331 NULL NULL 0.000
      59 system user   NULL Sleep 3331 NULL NULL 0.000
      61 system user   NULL Sleep 3331 NULL NULL 0.000
      56 system user   NULL Sleep 3331 NULL NULL 0.000
      60 system user   NULL Sleep 3331 NULL NULL 0.000
      62 system user   NULL Sleep 3331 NULL NULL 0.000
      57 system user   NULL Sleep 3331 NULL NULL 0.000
      63 system user   NULL Sleep 3331 NULL NULL 0.000
      64 system user   NULL Sleep 3331 NULL NULL 0.000
      54 system user   NULL Sleep 3331 NULL NULL 0.000
      65 system user   NULL Sleep 3331 NULL NULL 0.000
      69 system user   NULL Sleep 3331 NULL NULL 0.000
      66 system user   NULL Sleep 3331 NULL NULL 0.000
      67 system user   NULL Sleep 3331 NULL NULL 0.000
      70 system user   NULL Sleep 3331 NULL NULL 0.000
      68 system user   NULL Sleep 3331 NULL NULL 0.000
      78 root localhost NULL Killed 30 NULL set global wsrep_desync=1 0.000
      81 root localhost NULL Query 0 Init show processlist 0.000

      ------------------------------------------------------------------------------------------------
      72 rows in set (0.001 sec)

      Node moved in DESYNCED state and cannot be recovered
      2018-04-19 17:34:17 0 [Note] WSREP: Member 0.0 (t4w6) desyncs itself from group
      2018-04-19 17:34:17 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 2)
      2018-04-19 17:34:17 78 [Note] WSREP: Provider paused at 9f23373a-3e6d-11e8-b625-0646494b946c:2 (31)

      MariaDB [(none)]> unlock tables ;
      Query OK, 0 rows affected (0.000 sec)

      MariaDB [(none)]> show variables like '%wsrep_desync%' ;
      --------------------+

      Variable_name Value

      --------------------+

      wsrep_desync OFF

      --------------------+
      1 row in set (0.002 sec)

      MariaDB [(none)]> show status like 'wsrep_local_state_comment' ;
      -----------------------------------------+

      Variable_name Value

      -----------------------------------------+

      wsrep_local_state_comment Donor/Desynced

      -----------------------------------------+
      1 row in set (0.001 sec)

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MDEV-15067 Node fails to rejoin mdb cluster after recovering with mariabackup with option --galera-info

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Transition Time In Source Status Execution Times
            Mario Karuza (Inactive) made transition -
            Open In Progress
            		112d 21h 20m 1
            Mario Karuza (Inactive) made transition -
            In Progress In Review
            		3d 20h 1
            Jan Lindström (Inactive) made transition -
            In Review Stalled
            		23h 4m 1
            Jan Lindström (Inactive) made transition -
            Stalled In Progress
            		4s 1
            Jan Lindström (Inactive) made transition -
            In Progress Closed
            		1m 15s 1

            People

              jplindst Jan Lindström (Inactive)
              winstone Zdravelina Sokolovska (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.