Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23958

All node ejected from cluster after a new member

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Stalled (View Workflow)
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 10.4.15
    • Fix Version/s: 10.4
    • Component/s: Galera, Galera SST
    • Labels:
      None
    • Environment:
      Ubuntu 18.04

      Description

      The goal was to join a node from dev to make a test to upgrade in 10.5 (get prod as master, to got activity).

      sst_auth was different (forgot to update)
      prod don't got right to go to 4444 of the node 4
      node 1~3 : 10.4.15
      node 4 : 10.4.14

      the node 4 try to join the cluster
      node3 as donor => failed to process IST, ... => New SELF-LEAVE
      node1 as donor => failed to process IST, ... => New SELF-LEAVE
      node2 as donor => alone => non primary

      on global status all this node was in status : "initialized"

      So if we know the name of the cluster and one node to join we can switchof the cluster.

      ===== Node 1 log =====

      2020-10-13 13:05:13 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 
      2020-10-13 13:05:14 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a84f tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:14 0 [Note] WSREP: declaring 68c69353-9bc1 at tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 stable
      2020-10-13 13:05:14 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable
      2020-10-13 13:05:14 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:05:14 0 [Note] WSREP: Node 68c69353-9bc1 state prim
      2020-10-13 13:05:14 0 [Note] WSREP: view(view_id(PRIM,68c69353-9bc1,156) memb {
      	68c69353-9bc1,0
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      	f7b801a1-a84f,0
      } joined {
      } left {
      } partitioned {
      })
      2020-10-13 13:05:14 0 [Note] WSREP: save pc into disk
      2020-10-13 13:05:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 4
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 0 (db1-c2m4p)
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 1 (db1-c2m2p)
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 2 (db1-c2m3p)
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 3 (db1-c2m2d)
      2020-10-13 13:05:14 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 65,
      	members    = 3/4 (joined/total),
      	act_id     = 3286865882,
      	last_appl. = 3286865813,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:05:14 0 [Note] WSREP: Flow-control interval: [32, 32]
      2020-10-13 13:05:14 2 [Note] WSREP: ####### processing CC 3286865883, local, ordered
      2020-10-13 13:05:14 2 [Note] WSREP: ####### My UUID: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf
      2020-10-13 13:05:14 2 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:05:14 2 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:05:14 2 [Note] WSREP: ####### Adjusting cert position: 3286865882 -> 3286865883
      2020-10-13 13:05:14 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:05:14 2 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 1
        members(4):
      	0: 68c69353-09a8-11eb-9bc1-3376b5cfc3ea, db1-c2m4p
      	1: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p
      	2: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	3: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:05:14 2 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865814
      2020-10-13 13:05:14 2 [Note] WSREP: Min available from gcache for CC from group: 3285368863
      2020-10-13 13:05:16 0 [Note] WSREP: Member 3.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m4p)(SYNCED) as donor.
      2020-10-13 13:05:16 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable
      2020-10-13 13:05:16 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:05:16 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567)
      2020-10-13 13:05:17 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:05:19 0 [Note] WSREP: Node b51bd9d6-8d21 state prim
      2020-10-13 13:05:19 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,157) memb {
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      	f7b801a1-a84f,0
      } joined {
      } left {
      } partitioned {
      	68c69353-9bc1,0
      })
      2020-10-13 13:05:19 0 [Note] WSREP: save pc into disk
      2020-10-13 13:05:19 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567)
      2020-10-13 13:05:19 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
      2020-10-13 13:05:19 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: sent state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 1 (db1-c2m3p)
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 0 (db1-c2m2p)
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 2 (db1-c2m2d)
      2020-10-13 13:05:19 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 66,
      	members    = 2/3 (joined/total),
      	act_id     = 3286866015,
      	last_appl. = 3286865942,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:05:19 0 [Note] WSREP: Flow-control interval: [28, 28]
      2020-10-13 13:05:19 2 [Note] WSREP: ####### processing CC 3286866016, local, ordered
      2020-10-13 13:05:19 2 [Note] WSREP: ####### My UUID: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf
      2020-10-13 13:05:19 2 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:05:19 2 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:05:19 2 [Note] WSREP: ####### Adjusting cert position: 3286866015 -> 3286866016
      2020-10-13 13:05:19 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:05:19 2 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866016
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 0
        members(3):
      	0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p
      	1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	2: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:05:19 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 
      2020-10-13 13:05:19 2 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943
      2020-10-13 13:05:19 2 [Note] WSREP: Min available from gcache for CC from group: 3285369050
      2020-10-13 13:05:20 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') reconnecting to f7b801a1-a84f (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0
      2020-10-13 13:05:21 0 [Note] WSREP:  cleaning up 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567)
      2020-10-13 13:05:40 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:40 0 [Note] WSREP: remote endpoint tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 changed identity f7b801a1-0d43-11eb-a84f-878d2f0020e7 -> f7b801a1-0d43-11eb-a850-878d2f0020e7
      2020-10-13 13:05:44 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(b51bd9d6-8d21, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspecting node: f7b801a1-a84f
      2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(b51bd9d6-8d21, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspected node without join message, declaring inactive
      2020-10-13 13:05:50 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable
      2020-10-13 13:05:50 0 [Note] WSREP: Node b51bd9d6-8d21 state prim
      2020-10-13 13:05:50 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,158) memb {
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a84f,0
      })
      2020-10-13 13:05:50 0 [Note] WSREP: save pc into disk
      2020-10-13 13:05:50 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
      2020-10-13 13:05:50 0 [Note] WSREP: forgetting f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567)
      2020-10-13 13:05:50 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 0dda9600-0d44-11eb-aaaf-3b6a45970173
      2020-10-13 13:05:51 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:51 0 [Warning] WSREP: discarding established (time wait) f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) 
      2020-10-13 13:05:53 0 [Note] WSREP:  cleaning up f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567)
      2020-10-13 13:05:53 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:53 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: 
      2020-10-13 13:05:56 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:05:58 0 [Warning] WSREP: Failed to report last committed 77d86ba4-c581-11e8-b013-173e23a1017d:3286866014, -110 (Connection timed out)
      2020-10-13 13:06:21 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable
      2020-10-13 13:06:21 0 [Note] WSREP: declaring f7b801a1-a850 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:06:21 0 [Note] WSREP: Node b51bd9d6-8d21 state prim
      2020-10-13 13:06:21 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,159) memb {
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      	f7b801a1-a850,0
      } joined {
      } left {
      } partitioned {
      })
      2020-10-13 13:06:21 0 [Note] WSREP: save pc into disk
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 0dda9600-0d44-11eb-aaaf-3b6a45970173
      2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
      2020-10-13 13:06:21 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 2058e43e-0d44-11eb-8787-1f99ecd795c6
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6
      2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored.
      2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored.
      2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored.
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 0 (db1-c2m2p)
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 2 (db1-c2m2d)
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 1 (db1-c2m3p)
      2020-10-13 13:06:21 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 67,
      	members    = 2/3 (joined/total),
      	act_id     = 3286866099,
      	last_appl. = 3286865942,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:06:21 0 [Note] WSREP: Flow-control interval: [28, 28]
      2020-10-13 13:06:21 2 [Note] WSREP: ####### processing CC 3286866100, local, ordered
      2020-10-13 13:06:21 2 [Note] WSREP: ####### My UUID: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf
      2020-10-13 13:06:21 2 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:06:21 2 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:06:21 2 [Note] WSREP: ####### Adjusting cert position: 3286866099 -> 3286866100
      2020-10-13 13:06:21 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:06:21 2 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 0
        members(3):
      	0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p
      	1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	2: f7b801a1-0d43-11eb-a850-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:06:22 2 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943
      2020-10-13 13:06:22 2 [Note] WSREP: Min available from gcache for CC from group: 3285369224
      2020-10-13 13:06:29 0 [Note] WSREP: Member 2.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m2p)(SYNCED) as donor.
      2020-10-13 13:06:29 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 3286866166)
      2020-10-13 13:06:29 2 [Note] WSREP: Detected STR version: 1, req_len: 155, req: STRv1
      2020-10-13 13:06:29 2 [Note] WSREP: Cert index preload: 3286865943 -> 3286866100
      2020-10-13 13:06:29 2 [ERROR] WSREP: Failed to process action STATE_REQUEST, g: 3286866166, l: 20899575, ptr: 0x7f222d992380, size: 155: IST sender, failed to connect 'tcp://db1-c2m2d.os.acsc2012:4568': connect: Connection refused: 111 (Connection refused)
      	 at galera/src/ist.cpp:Sender():735
      2020-10-13 13:06:29 2 [Note] WSREP: Closing send monitor...
      2020-10-13 13:06:29 2 [Note] WSREP: Closed send monitor.
      2020-10-13 13:06:29 2 [Note] WSREP: gcomm: terminating thread
      2020-10-13 13:06:29 2 [Note] WSREP: gcomm: joining thread
      2020-10-13 13:06:29 2 [Note] WSREP: gcomm: closing backend
      2020-10-13 13:06:29 2 [Note] WSREP: view(view_id(NON_PRIM,b51bd9d6-8d21,159) memb {
      	b51bd9d6-8d21,0
      } joined {
      } left {
      } partitioned {
      	c7ca3602-9def,0
      	f7b801a1-a850,0
      })
      2020-10-13 13:06:29 2 [Note] WSREP: PC protocol downgrade 1 -> 0
      2020-10-13 13:06:29 2 [Note] WSREP: view((empty))
      2020-10-13 13:06:29 2 [Note] WSREP: gcomm: closed
      2020-10-13 13:06:29 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      2020-10-13 13:06:29 0 [Note] WSREP: Flow-control interval: [16, 16]
      2020-10-13 13:06:29 0 [Note] WSREP: Received NON-PRIMARY.
      2020-10-13 13:06:29 0 [Note] WSREP: Shifting DONOR/DESYNCED -> OPEN (TO: 3286866279)
      2020-10-13 13:06:29 0 [Note] WSREP: New SELF-LEAVE.
      2020-10-13 13:06:29 0 [Note] WSREP: Flow-control interval: [0, 0]
      2020-10-13 13:06:29 0 [Note] WSREP: Received SELF-LEAVE. Closing connection.
      2020-10-13 13:06:29 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 3286866279)
      2020-10-13 13:06:29 0 [Note] WSREP: RECV thread exiting 0: Success
      2020-10-13 13:06:29 2 [Note] WSREP: recv_thread() joined.
      2020-10-13 13:06:29 2 [Note] WSREP: Closing replication queue.
      2020-10-13 13:06:29 2 [Note] WSREP: Closing slave action queue.
      2020-10-13 13:06:29 2 [Note] WSREP: ================================================
      View:
        id: 00000000-0000-0000-0000-000000000000:-1
        status: non-primary
        protocol_version: -1
        capabilities: 
        final: yes
        own_index: -1
        members(0):
      =================================================
      2020-10-13 13:06:29 2 [Note] WSREP: Non-primary view
      2020-10-13 13:06:29 2 [Note] WSREP: Server status change synced -> disconnected
      2020-10-13 13:06:30 2 [Note] WSREP: Applier thread exiting ret: 6 thd: 2
      2020-10-13 13:06:30 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication)
      2020-10-13 13:08:50 14167028 [Warning] Aborted connection 14167028 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167024 [Warning] Aborted connection 14167024 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167021 [Warning] Aborted connection 14167021 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167020 [Warning] Aborted connection 14167020 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167030 [Warning] Aborted connection 14167030 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167038 [Warning] Aborted connection 14167038 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167037 [Warning] Aborted connection 14167037 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167040 [Warning] Aborted connection 14167040 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167034 [Warning] Aborted connection 14167034 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167022 [Warning] Aborted connection 14167022 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167023 [Warning] Aborted connection 14167023 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167035 [Warning] Aborted connection 14167035 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167031 [Warning] Aborted connection 14167031 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167041 [Warning] Aborted connection 14167041 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:50 14167044 [Warning] Aborted connection 14167044 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:08:56 14167083 [Warning] Aborted connection 14167083 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:09:01 14166441 [Warning] Aborted connection 14166441 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:09:22 14167138 [Warning] Aborted connection 14167138 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:09:22 14167135 [Warning] Aborted connection 14167135 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:09:22 14167140 [Warning] Aborted connection 14167140 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:09:22 14167139 [Warning] Aborted connection 14167139 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:12:45 14117071 [Warning] Aborted connection 14117071 to db: 'sfr_pims' user: 'tolkien' host: 'pims-celery-c0m1p.os.pfpims.sfrpims.fr' (Got timeout reading communication packets)
      2020-10-13 13:19:47 14167305 [Warning] WSREP: Unknown parameter 'pc.bootstrap'
      2020-10-13 13:19:47 14167305 [ERROR] WSREP: Set options returned 7
      2020-10-13 13:20:03 14167305 [Warning] WSREP: Unknown parameter 'pc.bootstrap'
      2020-10-13 13:20:03 14167305 [ERROR] WSREP: Set options returned 7
      2020-10-13 13:21:11 14167305 [Warning] WSREP: Unknown parameter 'pc. bootstrap'
      2020-10-13 13:21:11 14167305 [ERROR] WSREP: Set options returned 7
      2020-10-13 13:21:28 14168200 [Warning] WSREP: Unknown parameter 'pc.bootstrap'
      2020-10-13 13:21:28 14168200 [ERROR] WSREP: Set options returned 7
      2020-10-13 13:23:04 0 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown
      2020-10-13 13:23:04 0 [Note] WSREP: Shutdown replication
      
      

      ===== Node 2 log =====

      2020-10-13 13:05:13 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a84f tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:13 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: 
      2020-10-13 13:05:14 0 [Note] WSREP: declaring 68c69353-9bc1 at tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 stable
      2020-10-13 13:05:14 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable
      2020-10-13 13:05:14 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:05:14 0 [Note] WSREP: Node 68c69353-9bc1 state prim
      2020-10-13 13:05:14 0 [Note] WSREP: view(view_id(PRIM,68c69353-9bc1,156) memb {
      	68c69353-9bc1,0
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      	f7b801a1-a84f,0
      } joined {
      } left {
      } partitioned {
      })
      2020-10-13 13:05:14 0 [Note] WSREP: save pc into disk
      2020-10-13 13:05:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 0 (db1-c2m4p)
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 1 (db1-c2m2p)
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 2 (db1-c2m3p)
      2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 3 (db1-c2m2d)
      2020-10-13 13:05:14 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 65,
      	members    = 3/4 (joined/total),
      	act_id     = 3286865882,
      	last_appl. = 3286865813,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:05:14 0 [Note] WSREP: Flow-control interval: [32, 32]
      2020-10-13 13:05:14 1 [Note] WSREP: ####### processing CC 3286865883, local, ordered
      2020-10-13 13:05:14 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44
      2020-10-13 13:05:14 1 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:05:14 1 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:05:14 1 [Note] WSREP: ####### Adjusting cert position: 3286865882 -> 3286865883
      2020-10-13 13:05:14 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:05:14 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 2
        members(4):
      	0: 68c69353-09a8-11eb-9bc1-3376b5cfc3ea, db1-c2m4p
      	1: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p
      	2: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	3: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:05:14 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865814
      2020-10-13 13:05:14 1 [Note] WSREP: Min available from gcache for CC from group: 3285368819
      2020-10-13 13:05:16 0 [Note] WSREP: Member 3.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m4p)(SYNCED) as donor.
      2020-10-13 13:05:16 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable
      2020-10-13 13:05:16 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:05:16 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567)
      2020-10-13 13:05:16 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:05:19 0 [Note] WSREP: Node b51bd9d6-8d21 state prim
      2020-10-13 13:05:19 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,157) memb {
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      	f7b801a1-a84f,0
      } joined {
      } left {
      } partitioned {
      	68c69353-9bc1,0
      })
      2020-10-13 13:05:19 0 [Note] WSREP: save pc into disk
      2020-10-13 13:05:19 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567)
      2020-10-13 13:05:19 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: sent state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 1 (db1-c2m3p)
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 0 (db1-c2m2p)
      2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 2 (db1-c2m2d)
      2020-10-13 13:05:19 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 66,
      	members    = 2/3 (joined/total),
      	act_id     = 3286866015,
      	last_appl. = 3286865942,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:05:19 0 [Note] WSREP: Flow-control interval: [28, 28]
      2020-10-13 13:05:19 1 [Note] WSREP: ####### processing CC 3286866016, local, ordered
      2020-10-13 13:05:19 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44
      2020-10-13 13:05:19 1 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:05:19 1 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:05:19 1 [Note] WSREP: ####### Adjusting cert position: 3286866015 -> 3286866016
      2020-10-13 13:05:19 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:05:19 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866016
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 1
        members(3):
      	0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p
      	1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	2: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:05:19 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943
      2020-10-13 13:05:19 1 [Note] WSREP: Min available from gcache for CC from group: 3285369041
      2020-10-13 13:05:19 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 
      2020-10-13 13:05:20 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a84f (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0
      2020-10-13 13:05:21 0 [Note] WSREP:  cleaning up 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567)
      2020-10-13 13:05:40 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:40 0 [Note] WSREP: remote endpoint tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 changed identity f7b801a1-0d43-11eb-a84f-878d2f0020e7 -> f7b801a1-0d43-11eb-a850-878d2f0020e7
      2020-10-13 13:05:43 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspecting node: f7b801a1-a84f
      2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspected node without join message, declaring inactive
      2020-10-13 13:05:50 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable
      2020-10-13 13:05:50 0 [Note] WSREP: Node b51bd9d6-8d21 state prim
      2020-10-13 13:05:50 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,158) memb {
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a84f,0
      })
      2020-10-13 13:05:50 0 [Note] WSREP: save pc into disk
      2020-10-13 13:05:50 0 [Note] WSREP: forgetting f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567)
      2020-10-13 13:05:51 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:51 0 [Warning] WSREP: discarding established (time wait) f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) 
      2020-10-13 13:05:53 0 [Note] WSREP:  cleaning up f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567)
      2020-10-13 13:05:53 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 
      2020-10-13 13:05:53 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:05:56 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:06:21 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable
      2020-10-13 13:06:21 0 [Note] WSREP: declaring f7b801a1-a850 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:06:21 0 [Note] WSREP: Node b51bd9d6-8d21 state prim
      2020-10-13 13:06:21 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,159) memb {
      	b51bd9d6-8d21,0
      	c7ca3602-9def,0
      	f7b801a1-a850,0
      } joined {
      } left {
      } partitioned {
      })
      2020-10-13 13:06:21 0 [Note] WSREP: save pc into disk
      2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 0dda9600-0d44-11eb-aaaf-3b6a45970173
      2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6
      2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored.
      2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored.
      2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored.
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 0 (db1-c2m2p)
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 2 (db1-c2m2d)
      2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 1 (db1-c2m3p)
      2020-10-13 13:06:21 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 67,
      	members    = 2/3 (joined/total),
      	act_id     = 3286866099,
      	last_appl. = 3286865942,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:06:21 0 [Note] WSREP: Flow-control interval: [28, 28]
      2020-10-13 13:06:21 1 [Note] WSREP: ####### processing CC 3286866100, local, ordered
      2020-10-13 13:06:21 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44
      2020-10-13 13:06:21 1 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:06:21 1 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:06:21 1 [Note] WSREP: ####### Adjusting cert position: 3286866099 -> 3286866100
      2020-10-13 13:06:29 0 [Warning] WSREP: Failed to report last committed 77d86ba4-c581-11e8-b013-173e23a1017d:3286866016, -110 (Connection timed out)
      2020-10-13 13:06:29 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:06:29 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 1
        members(3):
      	0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p
      	1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	2: f7b801a1-0d43-11eb-a850-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:06:29 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943
      2020-10-13 13:06:29 1 [Note] WSREP: Min available from gcache for CC from group: 3285369215
      2020-10-13 13:06:29 0 [Note] WSREP: Member 2.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m2p)(SYNCED) as donor.
      2020-10-13 13:06:29 0 [Note] WSREP: declaring f7b801a1-a850 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:06:29 0 [Note] WSREP: forgetting b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567)
      2020-10-13 13:06:29 0 [Note] WSREP: Node c7ca3602-9def state prim
      2020-10-13 13:06:29 0 [Note] WSREP: view(view_id(PRIM,c7ca3602-9def,160) memb {
      	c7ca3602-9def,0
      	f7b801a1-a850,0
      } joined {
      } left {
      } partitioned {
      	b51bd9d6-8d21,0
      })
      2020-10-13 13:06:29 0 [Note] WSREP: save pc into disk
      2020-10-13 13:06:29 0 [Note] WSREP: forgetting b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567)
      2020-10-13 13:06:29 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
      2020-10-13 13:06:29 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5
      2020-10-13 13:06:29 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5
      2020-10-13 13:06:29 0 [Note] WSREP: STATE EXCHANGE: got state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 from 0 (db1-c2m3p)
      2020-10-13 13:06:30 0 [Note] WSREP: STATE EXCHANGE: got state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 from 1 (db1-c2m2d)
      2020-10-13 13:06:30 0 [Note] WSREP: Quorum results:
      	version    = 6,
      	component  = PRIMARY,
      	conf_id    = 68,
      	members    = 1/2 (joined/total),
      	act_id     = 3286866279,
      	last_appl. = 3286865942,
      	protocols  = 2/10/4 (gcs/repl/appl),
      	vote policy= 0,
      	group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
      2020-10-13 13:06:30 0 [Note] WSREP: Flow-control interval: [23, 23]
      2020-10-13 13:06:30 1 [Note] WSREP: ####### processing CC 3286866280, local, ordered
      2020-10-13 13:06:30 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44
      2020-10-13 13:06:30 1 [Note] WSREP: Skipping cert index reset
      2020-10-13 13:06:30 1 [Note] WSREP: REPL Protocols: 10 (5)
      2020-10-13 13:06:30 1 [Note] WSREP: ####### Adjusting cert position: 3286866279 -> 3286866280
      2020-10-13 13:06:30 0 [Note] WSREP: Service thread queue flushed.
      2020-10-13 13:06:30 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280
        status: primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 0
        members(2):
      	0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	1: f7b801a1-0d43-11eb-a850-878d2f0020e7, db1-c2m2d
      =================================================
      2020-10-13 13:06:30 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943
      2020-10-13 13:06:30 1 [Note] WSREP: Min available from gcache for CC from group: 3285369349
      2020-10-13 13:06:31 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 
      2020-10-13 13:06:32 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0
      2020-10-13 13:06:35 0 [Note] WSREP:  cleaning up b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567)
      2020-10-13 13:06:46 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a851 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567
      2020-10-13 13:06:46 0 [Note] WSREP: remote endpoint tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 changed identity f7b801a1-0d43-11eb-a850-878d2f0020e7 -> f7b801a1-0d43-11eb-a851-878d2f0020e7
      2020-10-13 13:06:50 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off
      2020-10-13 13:07:00 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,160)) suspecting node: f7b801a1-a850
      2020-10-13 13:07:00 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,160)) suspected node without join message, declaring inactive
      2020-10-13 13:07:01 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,160) memb {
      	c7ca3602-9def,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a850,0
      })
      2020-10-13 13:07:01 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      2020-10-13 13:07:01 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,161) memb {
      	c7ca3602-9def,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a850,0
      })
      2020-10-13 13:07:01 0 [Note] WSREP: Flow-control interval: [16, 16]
      2020-10-13 13:07:01 0 [Note] WSREP: Received NON-PRIMARY.
      2020-10-13 13:07:01 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 3286866280)
      2020-10-13 13:07:01 0 [Warning] WSREP: FLOW message from member 0 in non-primary configuration. Ignored.
      2020-10-13 13:07:01 0 [Warning] WSREP: Last Applied Action message in non-primary configuration from member 0
      2020-10-13 13:07:01 0 [Warning] WSREP: FLOW message from member 0 in non-primary configuration. Ignored.
      2020-10-13 13:07:01 0 [Warning] WSREP: FLOW message from member 0 in non-primary configuration. Ignored.
      2020-10-13 13:07:01 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280
        status: non-primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 0
        members(1):
      	0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      =================================================
      2020-10-13 13:07:01 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      2020-10-13 13:07:01 1 [Note] WSREP: Non-primary view
      2020-10-13 13:07:01 0 [Note] WSREP: Flow-control interval: [16, 16]
      2020-10-13 13:07:01 1 [Note] WSREP: Server status change synced -> connected
      2020-10-13 13:07:01 0 [Note] WSREP: Received NON-PRIMARY.
      2020-10-13 13:07:01 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280
        status: non-primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 0
        members(1):
      	0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      =================================================
      2020-10-13 13:07:01 1 [Note] WSREP: Non-primary view
      2020-10-13 13:07:01 1 [Note] WSREP: Server status change connected -> connected
      2020-10-13 13:07:32 0 [Note] WSREP: declaring f7b801a1-a851 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable
      2020-10-13 13:07:32 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,162) memb {
      	c7ca3602-9def,0
      	f7b801a1-a851,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a850,0
      })
      2020-10-13 13:07:32 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 2
      2020-10-13 13:07:32 0 [Note] WSREP: Flow-control interval: [23, 23]
      2020-10-13 13:07:32 0 [Note] WSREP: Received NON-PRIMARY.
      2020-10-13 13:07:32 1 [Note] WSREP: ================================================
      View:
        id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280
        status: non-primary
        protocol_version: 4
        capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
        final: no
        own_index: 0
        members(2):
      	0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p
      	1: f7b801a1-0d43-11eb-a851-878d2f0020e7, unspecified
      =================================================
      2020-10-13 13:07:32 1 [Note] WSREP: Non-primary view
      2020-10-13 13:07:32 1 [Note] WSREP: Server status change connected -> connected
      2020-10-13 13:16:13 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 
      2020-10-13 13:16:14 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0
      2020-10-13 13:16:41 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,162)) suspecting node: f7b801a1-a851
      2020-10-13 13:16:41 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,162)) suspected node without join message, declaring inactive
      2020-10-13 13:16:42 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,162) memb {
      	c7ca3602-9def,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a850,0
      	f7b801a1-a851,0
      })
      2020-10-13 13:16:42 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,163) memb {
      	c7ca3602-9def,0
      } joined {
      } left {
      } partitioned {
      	f7b801a1-a850,0
      	f7b801a1-a851,0
      })
      2020-10-13 13:16:42 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      2020-10-13 13:16:42 0 [Note] WSREP: Flow-control interval: [16, 16]
      2020-10-13 13:16:42 0 [Note] WSREP: Received NON-PRIMARY.
      2020-10-13 13:16:42 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      2020-10-13 13