Details
-
Bug
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Not a Bug
-
10.4.15
-
None
-
Ubuntu 18.04
Description
The goal was to join a node from dev to make a test to upgrade in 10.5 (get prod as master, to got activity).
sst_auth was different (forgot to update)
prod don't got right to go to 4444 of the node 4
node 1~3 : 10.4.15
node 4 : 10.4.14
the node 4 try to join the cluster
node3 as donor => failed to process IST, ... => New SELF-LEAVE
node1 as donor => failed to process IST, ... => New SELF-LEAVE
node2 as donor => alone => non primary
on global status all this node was in status : "initialized"
So if we know the name of the cluster and one node to join we can switchof the cluster.
===== Node 1 log =====
2020-10-13 13:05:13 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a84f tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: declaring 68c69353-9bc1 at tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: Node 68c69353-9bc1 state prim |
2020-10-13 13:05:14 0 [Note] WSREP: view(view_id(PRIM,68c69353-9bc1,156) memb { |
68c69353-9bc1,0 |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:05:14 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 4 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 0 (db1-c2m4p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 1 (db1-c2m2p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 2 (db1-c2m3p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 3 (db1-c2m2d) |
2020-10-13 13:05:14 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 65, |
members = 3/4 (joined/total), |
act_id = 3286865882, |
last_appl. = 3286865813, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:14 0 [Note] WSREP: Flow-control interval: [32, 32] |
2020-10-13 13:05:14 2 [Note] WSREP: ####### processing CC 3286865883, local, ordered |
2020-10-13 13:05:14 2 [Note] WSREP: ####### My UUID: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf |
2020-10-13 13:05:14 2 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:05:14 2 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:05:14 2 [Note] WSREP: ####### Adjusting cert position: 3286865882 -> 3286865883 |
2020-10-13 13:05:14 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:14 2 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 1 |
members(4): |
0: 68c69353-09a8-11eb-9bc1-3376b5cfc3ea, db1-c2m4p |
1: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
2: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
3: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:05:14 2 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865814 |
2020-10-13 13:05:14 2 [Note] WSREP: Min available from gcache for CC from group: 3285368863 |
2020-10-13 13:05:16 0 [Note] WSREP: Member 3.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m4p)(SYNCED) as donor. |
2020-10-13 13:05:16 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:05:16 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:05:16 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:17 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:19 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:05:19 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,157) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
68c69353-9bc1,0 |
})
|
2020-10-13 13:05:19 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:19 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:19 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: sent state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 1 (db1-c2m3p) |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 0 (db1-c2m2p) |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 2 (db1-c2m2d) |
2020-10-13 13:05:19 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 66, |
members = 2/3 (joined/total), |
act_id = 3286866015, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:19 0 [Note] WSREP: Flow-control interval: [28, 28] |
2020-10-13 13:05:19 2 [Note] WSREP: ####### processing CC 3286866016, local, ordered |
2020-10-13 13:05:19 2 [Note] WSREP: ####### My UUID: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf |
2020-10-13 13:05:19 2 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:05:19 2 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:05:19 2 [Note] WSREP: ####### Adjusting cert position: 3286866015 -> 3286866016 |
2020-10-13 13:05:19 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:19 2 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866016 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(3): |
0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
2: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:05:19 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:19 2 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943 |
2020-10-13 13:05:19 2 [Note] WSREP: Min available from gcache for CC from group: 3285369050 |
2020-10-13 13:05:20 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') reconnecting to f7b801a1-a84f (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0 |
2020-10-13 13:05:21 0 [Note] WSREP: cleaning up 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:40 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:40 0 [Note] WSREP: remote endpoint tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 changed identity f7b801a1-0d43-11eb-a84f-878d2f0020e7 -> f7b801a1-0d43-11eb-a850-878d2f0020e7 |
2020-10-13 13:05:44 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(b51bd9d6-8d21, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspecting node: f7b801a1-a84f |
2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(b51bd9d6-8d21, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspected node without join message, declaring inactive |
2020-10-13 13:05:50 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:05:50 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:05:50 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,158) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a84f,0 |
})
|
2020-10-13 13:05:50 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:50 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2 |
2020-10-13 13:05:50 0 [Note] WSREP: forgetting f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) |
2020-10-13 13:05:50 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 0dda9600-0d44-11eb-aaaf-3b6a45970173 |
2020-10-13 13:05:51 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:51 0 [Warning] WSREP: discarding established (time wait) f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) |
2020-10-13 13:05:53 0 [Note] WSREP: cleaning up f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) |
2020-10-13 13:05:53 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:53 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: |
2020-10-13 13:05:56 0 [Note] WSREP: (b51bd9d6-8d21, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:58 0 [Warning] WSREP: Failed to report last committed 77d86ba4-c581-11e8-b013-173e23a1017d:3286866014, -110 (Connection timed out) |
2020-10-13 13:06:21 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:06:21 0 [Note] WSREP: declaring f7b801a1-a850 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:06:21 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:06:21 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,159) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a850,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:06:21 0 [Note] WSREP: save pc into disk |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 0dda9600-0d44-11eb-aaaf-3b6a45970173 |
2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3 |
2020-10-13 13:06:21 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 2058e43e-0d44-11eb-8787-1f99ecd795c6 |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 0 (db1-c2m2p) |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 2 (db1-c2m2d) |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 1 (db1-c2m3p) |
2020-10-13 13:06:21 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 67, |
members = 2/3 (joined/total), |
act_id = 3286866099, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:06:21 0 [Note] WSREP: Flow-control interval: [28, 28] |
2020-10-13 13:06:21 2 [Note] WSREP: ####### processing CC 3286866100, local, ordered |
2020-10-13 13:06:21 2 [Note] WSREP: ####### My UUID: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf |
2020-10-13 13:06:21 2 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:06:21 2 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:06:21 2 [Note] WSREP: ####### Adjusting cert position: 3286866099 -> 3286866100 |
2020-10-13 13:06:21 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:06:21 2 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(3): |
0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
2: f7b801a1-0d43-11eb-a850-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:06:22 2 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943 |
2020-10-13 13:06:22 2 [Note] WSREP: Min available from gcache for CC from group: 3285369224 |
2020-10-13 13:06:29 0 [Note] WSREP: Member 2.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m2p)(SYNCED) as donor. |
2020-10-13 13:06:29 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 3286866166) |
2020-10-13 13:06:29 2 [Note] WSREP: Detected STR version: 1, req_len: 155, req: STRv1 |
2020-10-13 13:06:29 2 [Note] WSREP: Cert index preload: 3286865943 -> 3286866100 |
2020-10-13 13:06:29 2 [ERROR] WSREP: Failed to process action STATE_REQUEST, g: 3286866166, l: 20899575, ptr: 0x7f222d992380, size: 155: IST sender, failed to connect 'tcp://db1-c2m2d.os.acsc2012:4568': connect: Connection refused: 111 (Connection refused) |
at galera/src/ist.cpp:Sender():735 |
2020-10-13 13:06:29 2 [Note] WSREP: Closing send monitor... |
2020-10-13 13:06:29 2 [Note] WSREP: Closed send monitor. |
2020-10-13 13:06:29 2 [Note] WSREP: gcomm: terminating thread |
2020-10-13 13:06:29 2 [Note] WSREP: gcomm: joining thread |
2020-10-13 13:06:29 2 [Note] WSREP: gcomm: closing backend |
2020-10-13 13:06:29 2 [Note] WSREP: view(view_id(NON_PRIM,b51bd9d6-8d21,159) memb { |
b51bd9d6-8d21,0 |
} joined {
|
} left {
|
} partitioned {
|
c7ca3602-9def,0 |
f7b801a1-a850,0 |
})
|
2020-10-13 13:06:29 2 [Note] WSREP: PC protocol downgrade 1 -> 0 |
2020-10-13 13:06:29 2 [Note] WSREP: view((empty)) |
2020-10-13 13:06:29 2 [Note] WSREP: gcomm: closed |
2020-10-13 13:06:29 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 |
2020-10-13 13:06:29 0 [Note] WSREP: Flow-control interval: [16, 16] |
2020-10-13 13:06:29 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:06:29 0 [Note] WSREP: Shifting DONOR/DESYNCED -> OPEN (TO: 3286866279) |
2020-10-13 13:06:29 0 [Note] WSREP: New SELF-LEAVE. |
2020-10-13 13:06:29 0 [Note] WSREP: Flow-control interval: [0, 0] |
2020-10-13 13:06:29 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. |
2020-10-13 13:06:29 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 3286866279) |
2020-10-13 13:06:29 0 [Note] WSREP: RECV thread exiting 0: Success |
2020-10-13 13:06:29 2 [Note] WSREP: recv_thread() joined. |
2020-10-13 13:06:29 2 [Note] WSREP: Closing replication queue. |
2020-10-13 13:06:29 2 [Note] WSREP: Closing slave action queue. |
2020-10-13 13:06:29 2 [Note] WSREP: ================================================ |
View:
|
id: 00000000-0000-0000-0000-000000000000:-1 |
status: non-primary
|
protocol_version: -1 |
capabilities:
|
final: yes |
own_index: -1 |
members(0): |
=================================================
|
2020-10-13 13:06:29 2 [Note] WSREP: Non-primary view |
2020-10-13 13:06:29 2 [Note] WSREP: Server status change synced -> disconnected |
2020-10-13 13:06:30 2 [Note] WSREP: Applier thread exiting ret: 6 thd: 2 |
2020-10-13 13:06:30 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) |
2020-10-13 13:08:50 14167028 [Warning] Aborted connection 14167028 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167024 [Warning] Aborted connection 14167024 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167021 [Warning] Aborted connection 14167021 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167020 [Warning] Aborted connection 14167020 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167030 [Warning] Aborted connection 14167030 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167038 [Warning] Aborted connection 14167038 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167037 [Warning] Aborted connection 14167037 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167040 [Warning] Aborted connection 14167040 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167034 [Warning] Aborted connection 14167034 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167022 [Warning] Aborted connection 14167022 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167023 [Warning] Aborted connection 14167023 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167035 [Warning] Aborted connection 14167035 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167031 [Warning] Aborted connection 14167031 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167041 [Warning] Aborted connection 14167041 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:50 14167044 [Warning] Aborted connection 14167044 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:08:56 14167083 [Warning] Aborted connection 14167083 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:09:01 14166441 [Warning] Aborted connection 14166441 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:09:22 14167138 [Warning] Aborted connection 14167138 to db: 'sfr_pims' user: 'tolkien' host: 'sva7.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:09:22 14167135 [Warning] Aborted connection 14167135 to db: 'sfr_pims' user: 'tolkien' host: 'sva8.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:09:22 14167140 [Warning] Aborted connection 14167140 to db: 'sfr_pims' user: 'tolkien' host: 'sva9.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:09:22 14167139 [Warning] Aborted connection 14167139 to db: 'sfr_pims' user: 'tolkien' host: 'sva6.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:12:45 14117071 [Warning] Aborted connection 14117071 to db: 'sfr_pims' user: 'tolkien' host: 'pims-celery-c0m1p.os.pfpims.sfrpims.fr' (Got timeout reading communication packets) |
2020-10-13 13:19:47 14167305 [Warning] WSREP: Unknown parameter 'pc.bootstrap' |
2020-10-13 13:19:47 14167305 [ERROR] WSREP: Set options returned 7 |
2020-10-13 13:20:03 14167305 [Warning] WSREP: Unknown parameter 'pc.bootstrap' |
2020-10-13 13:20:03 14167305 [ERROR] WSREP: Set options returned 7 |
2020-10-13 13:21:11 14167305 [Warning] WSREP: Unknown parameter 'pc. bootstrap' |
2020-10-13 13:21:11 14167305 [ERROR] WSREP: Set options returned 7 |
2020-10-13 13:21:28 14168200 [Warning] WSREP: Unknown parameter 'pc.bootstrap' |
2020-10-13 13:21:28 14168200 [ERROR] WSREP: Set options returned 7 |
2020-10-13 13:23:04 0 [Note] /usr/sbin/mysqld (initiated by: unknown): Normal shutdown |
2020-10-13 13:23:04 0 [Note] WSREP: Shutdown replication |
|
===== Node 2 log =====
2020-10-13 13:05:13 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a84f tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:13 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: |
2020-10-13 13:05:14 0 [Note] WSREP: declaring 68c69353-9bc1 at tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: Node 68c69353-9bc1 state prim |
2020-10-13 13:05:14 0 [Note] WSREP: view(view_id(PRIM,68c69353-9bc1,156) memb { |
68c69353-9bc1,0 |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:05:14 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 4 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 0 (db1-c2m4p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 1 (db1-c2m2p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 2 (db1-c2m3p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 3 (db1-c2m2d) |
2020-10-13 13:05:14 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 65, |
members = 3/4 (joined/total), |
act_id = 3286865882, |
last_appl. = 3286865813, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:14 0 [Note] WSREP: Flow-control interval: [32, 32] |
2020-10-13 13:05:14 1 [Note] WSREP: ####### processing CC 3286865883, local, ordered |
2020-10-13 13:05:14 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44 |
2020-10-13 13:05:14 1 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:05:14 1 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:05:14 1 [Note] WSREP: ####### Adjusting cert position: 3286865882 -> 3286865883 |
2020-10-13 13:05:14 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:14 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 2 |
members(4): |
0: 68c69353-09a8-11eb-9bc1-3376b5cfc3ea, db1-c2m4p |
1: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
2: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
3: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:05:14 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865814 |
2020-10-13 13:05:14 1 [Note] WSREP: Min available from gcache for CC from group: 3285368819 |
2020-10-13 13:05:16 0 [Note] WSREP: Member 3.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m4p)(SYNCED) as donor. |
2020-10-13 13:05:16 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:05:16 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:05:16 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:16 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:19 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:05:19 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,157) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
68c69353-9bc1,0 |
})
|
2020-10-13 13:05:19 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:19 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:19 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: sent state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 1 (db1-c2m3p) |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 0 (db1-c2m2p) |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 2 (db1-c2m2d) |
2020-10-13 13:05:19 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 66, |
members = 2/3 (joined/total), |
act_id = 3286866015, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:19 0 [Note] WSREP: Flow-control interval: [28, 28] |
2020-10-13 13:05:19 1 [Note] WSREP: ####### processing CC 3286866016, local, ordered |
2020-10-13 13:05:19 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44 |
2020-10-13 13:05:19 1 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:05:19 1 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:05:19 1 [Note] WSREP: ####### Adjusting cert position: 3286866015 -> 3286866016 |
2020-10-13 13:05:19 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:19 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866016 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 1 |
members(3): |
0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
2: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:05:19 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943 |
2020-10-13 13:05:19 1 [Note] WSREP: Min available from gcache for CC from group: 3285369041 |
2020-10-13 13:05:19 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:20 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a84f (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0 |
2020-10-13 13:05:21 0 [Note] WSREP: cleaning up 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:40 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:40 0 [Note] WSREP: remote endpoint tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 changed identity f7b801a1-0d43-11eb-a84f-878d2f0020e7 -> f7b801a1-0d43-11eb-a850-878d2f0020e7 |
2020-10-13 13:05:43 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspecting node: f7b801a1-a84f |
2020-10-13 13:05:49 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,b51bd9d6-8d21,157)) suspected node without join message, declaring inactive |
2020-10-13 13:05:50 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:05:50 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:05:50 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,158) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a84f,0 |
})
|
2020-10-13 13:05:50 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:50 0 [Note] WSREP: forgetting f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) |
2020-10-13 13:05:51 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:51 0 [Warning] WSREP: discarding established (time wait) f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) |
2020-10-13 13:05:53 0 [Note] WSREP: cleaning up f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567) |
2020-10-13 13:05:53 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:53 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a850 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:56 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:06:21 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:06:21 0 [Note] WSREP: declaring f7b801a1-a850 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:06:21 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:06:21 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,159) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a850,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:06:21 0 [Note] WSREP: save pc into disk |
2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2 |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 0dda9600-0d44-11eb-aaaf-3b6a45970173 |
2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3 |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 0 (db1-c2m2p) |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 2 (db1-c2m2d) |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 1 (db1-c2m3p) |
2020-10-13 13:06:21 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 67, |
members = 2/3 (joined/total), |
act_id = 3286866099, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:06:21 0 [Note] WSREP: Flow-control interval: [28, 28] |
2020-10-13 13:06:21 1 [Note] WSREP: ####### processing CC 3286866100, local, ordered |
2020-10-13 13:06:21 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44 |
2020-10-13 13:06:21 1 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:06:21 1 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:06:21 1 [Note] WSREP: ####### Adjusting cert position: 3286866099 -> 3286866100 |
2020-10-13 13:06:29 0 [Warning] WSREP: Failed to report last committed 77d86ba4-c581-11e8-b013-173e23a1017d:3286866016, -110 (Connection timed out) |
2020-10-13 13:06:29 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:06:29 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 1 |
members(3): |
0: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
1: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
2: f7b801a1-0d43-11eb-a850-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:06:29 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943 |
2020-10-13 13:06:29 1 [Note] WSREP: Min available from gcache for CC from group: 3285369215 |
2020-10-13 13:06:29 0 [Note] WSREP: Member 2.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m2p)(SYNCED) as donor. |
2020-10-13 13:06:29 0 [Note] WSREP: declaring f7b801a1-a850 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:06:29 0 [Note] WSREP: forgetting b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567) |
2020-10-13 13:06:29 0 [Note] WSREP: Node c7ca3602-9def state prim |
2020-10-13 13:06:29 0 [Note] WSREP: view(view_id(PRIM,c7ca3602-9def,160) memb { |
c7ca3602-9def,0 |
f7b801a1-a850,0 |
} joined {
|
} left {
|
} partitioned {
|
b51bd9d6-8d21,0 |
})
|
2020-10-13 13:06:29 0 [Note] WSREP: save pc into disk |
2020-10-13 13:06:29 0 [Note] WSREP: forgetting b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567) |
2020-10-13 13:06:29 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2 |
2020-10-13 13:06:29 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 |
2020-10-13 13:06:29 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 |
2020-10-13 13:06:29 0 [Note] WSREP: STATE EXCHANGE: got state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 from 0 (db1-c2m3p) |
2020-10-13 13:06:30 0 [Note] WSREP: STATE EXCHANGE: got state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 from 1 (db1-c2m2d) |
2020-10-13 13:06:30 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 68, |
members = 1/2 (joined/total), |
act_id = 3286866279, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:06:30 0 [Note] WSREP: Flow-control interval: [23, 23] |
2020-10-13 13:06:30 1 [Note] WSREP: ####### processing CC 3286866280, local, ordered |
2020-10-13 13:06:30 1 [Note] WSREP: ####### My UUID: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44 |
2020-10-13 13:06:30 1 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:06:30 1 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:06:30 1 [Note] WSREP: ####### Adjusting cert position: 3286866279 -> 3286866280 |
2020-10-13 13:06:30 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:06:30 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(2): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
1: f7b801a1-0d43-11eb-a850-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:06:30 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865943 |
2020-10-13 13:06:30 1 [Note] WSREP: Min available from gcache for CC from group: 3285369349 |
2020-10-13 13:06:31 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:06:32 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a850 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0 |
2020-10-13 13:06:35 0 [Note] WSREP: cleaning up b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567) |
2020-10-13 13:06:46 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') connection established to f7b801a1-a851 tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:06:46 0 [Note] WSREP: remote endpoint tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 changed identity f7b801a1-0d43-11eb-a850-878d2f0020e7 -> f7b801a1-0d43-11eb-a851-878d2f0020e7 |
2020-10-13 13:06:50 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:07:00 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,160)) suspecting node: f7b801a1-a850 |
2020-10-13 13:07:00 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,160)) suspected node without join message, declaring inactive |
2020-10-13 13:07:01 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,160) memb { |
c7ca3602-9def,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a850,0 |
})
|
2020-10-13 13:07:01 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 |
2020-10-13 13:07:01 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,161) memb { |
c7ca3602-9def,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a850,0 |
})
|
2020-10-13 13:07:01 0 [Note] WSREP: Flow-control interval: [16, 16] |
2020-10-13 13:07:01 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:07:01 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 3286866280) |
2020-10-13 13:07:01 0 [Warning] WSREP: FLOW message from member 0 in non-primary configuration. Ignored. |
2020-10-13 13:07:01 0 [Warning] WSREP: Last Applied Action message in non-primary configuration from member 0 |
2020-10-13 13:07:01 0 [Warning] WSREP: FLOW message from member 0 in non-primary configuration. Ignored. |
2020-10-13 13:07:01 0 [Warning] WSREP: FLOW message from member 0 in non-primary configuration. Ignored. |
2020-10-13 13:07:01 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: non-primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(1): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
=================================================
|
2020-10-13 13:07:01 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 |
2020-10-13 13:07:01 1 [Note] WSREP: Non-primary view |
2020-10-13 13:07:01 0 [Note] WSREP: Flow-control interval: [16, 16] |
2020-10-13 13:07:01 1 [Note] WSREP: Server status change synced -> connected |
2020-10-13 13:07:01 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:07:01 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: non-primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(1): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
=================================================
|
2020-10-13 13:07:01 1 [Note] WSREP: Non-primary view |
2020-10-13 13:07:01 1 [Note] WSREP: Server status change connected -> connected |
2020-10-13 13:07:32 0 [Note] WSREP: declaring f7b801a1-a851 at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:07:32 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,162) memb { |
c7ca3602-9def,0 |
f7b801a1-a851,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a850,0 |
})
|
2020-10-13 13:07:32 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 2 |
2020-10-13 13:07:32 0 [Note] WSREP: Flow-control interval: [23, 23] |
2020-10-13 13:07:32 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:07:32 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: non-primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(2): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
1: f7b801a1-0d43-11eb-a851-878d2f0020e7, unspecified |
=================================================
|
2020-10-13 13:07:32 1 [Note] WSREP: Non-primary view |
2020-10-13 13:07:32 1 [Note] WSREP: Server status change connected -> connected |
2020-10-13 13:16:13 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:16:14 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 0 |
2020-10-13 13:16:41 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,162)) suspecting node: f7b801a1-a851 |
2020-10-13 13:16:41 0 [Note] WSREP: evs::proto(c7ca3602-9def, OPERATIONAL, view_id(REG,c7ca3602-9def,162)) suspected node without join message, declaring inactive |
2020-10-13 13:16:42 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,162) memb { |
c7ca3602-9def,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a850,0 |
f7b801a1-a851,0 |
})
|
2020-10-13 13:16:42 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,163) memb { |
c7ca3602-9def,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a850,0 |
f7b801a1-a851,0 |
})
|
2020-10-13 13:16:42 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 |
2020-10-13 13:16:42 0 [Note] WSREP: Flow-control interval: [16, 16] |
2020-10-13 13:16:42 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:16:42 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 |
2020-10-13 13:16:42 0 [Note] WSREP: Flow-control interval: [16, 16] |
2020-10-13 13:16:42 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:16:42 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: non-primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(1): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
=================================================
|
2020-10-13 13:16:42 1 [Note] WSREP: Non-primary view |
2020-10-13 13:16:42 1 [Note] WSREP: Server status change connected -> connected |
2020-10-13 13:16:42 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: non-primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(1): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
=================================================
|
2020-10-13 13:16:42 1 [Note] WSREP: Non-primary view |
2020-10-13 13:16:42 1 [Note] WSREP: Server status change connected -> connected |
2020-10-13 13:16:59 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 30 |
2020-10-13 13:17:44 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 60 |
2020-10-13 13:18:29 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 90 |
2020-10-13 13:19:14 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 120 |
2020-10-13 13:19:59 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 150 |
2020-10-13 13:20:44 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 180 |
2020-10-13 13:21:29 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 210 |
2020-10-13 13:22:14 0 [Note] WSREP: (c7ca3602-9def, 'tcp://[::]:4567') reconnecting to f7b801a1-a851 (tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567), attempt 240 |
2020-10-13 13:22:24 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost []): Normal shutdown |
2020-10-13 13:22:24 0 [Note] WSREP: Shutdown replication |
2020-10-13 13:22:24 0 [Note] WSREP: Server status change connected -> disconnecting |
2020-10-13 13:22:24 0 [Note] WSREP: Closing send monitor... |
2020-10-13 13:22:24 0 [Note] WSREP: Closed send monitor. |
2020-10-13 13:22:24 0 [Note] WSREP: gcomm: terminating thread |
2020-10-13 13:22:24 0 [Note] WSREP: gcomm: joining thread |
2020-10-13 13:22:24 0 [Note] WSREP: gcomm: closing backend |
2020-10-13 13:22:24 0 [Note] WSREP: PC protocol downgrade 1 -> 0 |
2020-10-13 13:22:24 0 [Note] WSREP: view((empty)) |
2020-10-13 13:22:24 0 [Note] WSREP: gcomm: closed |
2020-10-13 13:22:24 0 [Note] WSREP: New SELF-LEAVE. |
2020-10-13 13:22:24 0 [Note] WSREP: Flow-control interval: [0, 0] |
2020-10-13 13:22:24 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. |
2020-10-13 13:22:24 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 3286866280) |
2020-10-13 13:22:24 0 [Note] WSREP: RECV thread exiting 0: Success |
2020-10-13 13:22:24 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866280 |
status: non-primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: yes |
own_index: -1 |
members(0): |
=================================================
|
2020-10-13 13:22:24 1 [Note] WSREP: Non-primary view |
2020-10-13 13:22:24 1 [Note] WSREP: Server status change disconnecting -> disconnected |
2020-10-13 13:22:24 0 [Note] WSREP: recv_thread() joined. |
2020-10-13 13:22:24 0 [Note] WSREP: Closing replication queue. |
2020-10-13 13:22:24 0 [Note] WSREP: Closing slave action queue. |
2020-10-13 13:22:24 1 [Note] WSREP: Applier thread exiting ret: 0 thd: 1 |
2020-10-13 13:22:24 1 [Warning] Aborted connection 1 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 20762 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 20679 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 20676 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 19373 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 19372 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 15229 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 12005 |
2020-10-13 13:22:24 0 [Note] WSREP: killing local connection: 32 |
2020-10-13 13:22:25 2 [Note] WSREP: rollbacker thread exiting 2 |
2020-10-13 13:22:25 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) |
2020-10-13 13:22:25 0 [Note] Event Scheduler: Purging the queue. 0 events |
2020-10-13 13:22:25 0 [Note] InnoDB: FTS optimize thread exiting. |
2020-10-13 13:22:25 4 [Note] InnoDB: to purge 5 transactions |
2020-10-13 13:22:25 0 [Note] WSREP: dtor state: CLOSED |
2020-10-13 13:22:25 0 [Note] WSREP: MemPool(TrxHandleSlave): hit ratio: 0.998545, misses: 4185, in use: 335, in pool: 1715 |
2020-10-13 13:22:25 0 [Note] WSREP: mon: entered 2876225 oooe fraction 0 oool fraction 0 |
2020-10-13 13:22:25 0 [Note] WSREP: mon: entered 2876225 oooe fraction 0 oool fraction 0 |
2020-10-13 13:22:25 0 [Note] WSREP: mon: entered 2917930 oooe fraction 0 oool fraction 3.42709e-07 |
2020-10-13 13:22:25 0 [Note] WSREP: cert index usage at exit 318 |
2020-10-13 13:22:25 0 [Note] WSREP: cert trx map usage at exit 335 |
2020-10-13 13:22:25 0 [Note] WSREP: deps set usage at exit 0 |
2020-10-13 13:22:25 0 [Note] WSREP: avg deps dist 33.5026 |
2020-10-13 13:22:25 0 [Note] WSREP: avg cert interval 51455.4 |
2020-10-13 13:22:25 0 [Note] WSREP: cert index size 318 |
2020-10-13 13:22:25 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:22:25 0 [Note] WSREP: wsdb trx map usage 0 conn query map usage 0 |
2020-10-13 13:22:25 0 [Note] WSREP: MemPool(LocalTrxHandle): hit ratio: 0, misses: 0, in use: 0, in pool: 0 |
2020-10-13 13:22:25 0 [Note] WSREP: Shifting CLOSED -> DESTROYED (TO: 3286866280) |
2020-10-13 13:22:25 0 [Note] WSREP: Flushing memory map to disk... |
2020-10-13 13:22:25 0 [Note] InnoDB: Starting shutdown... |
2020-10-13 13:22:25 0 [Note] InnoDB: Dumping buffer pool(s) to /opt/db1/var/lib/mysql/ib_buffer_pool |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 0, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 1, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 2, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 3, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 4, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 5, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 6, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Instance 7, restricted to 8023 pages due to innodb_buf_pool_dump_pct=25 |
2020-10-13 13:22:25 0 [Note] InnoDB: Buffer pool(s) dump completed at 201013 13:22:25 |
2020-10-13 13:22:27 0 [Note] InnoDB: Shutdown completed; log sequence number 13045936597626; transaction id 24834143897 |
2020-10-13 13:22:27 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" |
2020-10-13 13:22:27 0 [Note] /usr/sbin/mysqld: Shutdown complete |
|
===== Node 3 log =====
2020-10-13 13:05:13 0 [Note] WSREP: (68c69353-9bc1, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring f7b801a1-a84f at tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: (68c69353-9bc1, 'tcp://[::]:4567') connection established to f7b801a1-a84f tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: Node 68c69353-9bc1 state prim |
2020-10-13 13:05:14 0 [Note] WSREP: view(view_id(PRIM,68c69353-9bc1,156) memb { |
68c69353-9bc1,0 |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:05:14 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 4 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 0 (db1-c2m4p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 1 (db1-c2m2p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 2 (db1-c2m3p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 3 (db1-c2m2d) |
2020-10-13 13:05:14 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 65, |
members = 3/4 (joined/total), |
act_id = 3286865882, |
last_appl. = 3286865813, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:14 0 [Note] WSREP: Flow-control interval: [32, 32] |
2020-10-13 13:05:14 1 [Note] WSREP: ####### processing CC 3286865883, local, ordered |
2020-10-13 13:05:14 1 [Note] WSREP: ####### My UUID: 68c69353-09a8-11eb-9bc1-3376b5cfc3ea |
2020-10-13 13:05:14 1 [Note] WSREP: Skipping cert index reset |
2020-10-13 13:05:14 1 [Note] WSREP: REPL Protocols: 10 (5) |
2020-10-13 13:05:14 1 [Note] WSREP: ####### Adjusting cert position: 3286865882 -> 3286865883 |
2020-10-13 13:05:14 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:14 1 [Note] WSREP: ================================================ |
View:
|
id: 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883 |
status: primary
|
protocol_version: 4 |
capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO
|
final: no |
own_index: 0 |
members(4): |
0: 68c69353-09a8-11eb-9bc1-3376b5cfc3ea, db1-c2m4p |
1: b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf, db1-c2m2p |
2: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, db1-c2m3p |
3: f7b801a1-0d43-11eb-a84f-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:05:14 1 [Note] WSREP: Lowest cert index boundary for CC from group: 3286865814 |
2020-10-13 13:05:14 1 [Note] WSREP: Min available from gcache for CC from group: 3285368852 |
2020-10-13 13:05:16 0 [Note] WSREP: Member 3.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m4p)(SYNCED) as donor. |
2020-10-13 13:05:16 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 3286866009) |
2020-10-13 13:05:16 1 [Note] WSREP: Detected STR version: 1, req_len: 163, req: STRv1 |
2020-10-13 13:05:16 1 [Note] WSREP: IST request: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340-3286865883|tcp://db1-c2m2d.os.acsc2012:4568 |
2020-10-13 13:05:16 1 [Note] WSREP: IST first seqno 2893434341 not found from cache, falling back to SST |
2020-10-13 13:05:16 1 [Note] WSREP: Cert index preload: 3286865814 -> 3286865883 |
2020-10-13 13:05:16 1 [ERROR] WSREP: Failed to process action STATE_REQUEST, g: 3286866009, l: 20886837, ptr: 0x7f041096dac8, size: 163: IST sender, failed to connect 'tcp://db1-c2m2d.os.acsc2012:4568': connect: Connection refused: 111 (Connection refused) |
at galera/src/ist.cpp:Sender():735 |
2020-10-13 13:05:16 1 [Note] WSREP: Closing send monitor... |
2020-10-13 13:05:16 1 [Note] WSREP: Closed send monitor. |
2020-10-13 13:05:16 1 [Note] WSREP: gcomm: terminating thread |
2020-10-13 13:05:16 1 [Note] WSREP: gcomm: joining thread |
2020-10-13 13:05:16 1 [Note] WSREP: gcomm: closing backend |
2020-10-13 13:05:16 1 [Note] WSREP: view(view_id(NON_PRIM,68c69353-9bc1,156) memb { |
68c69353-9bc1,0 |
} joined {
|
} left {
|
} partitioned {
|
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
})
|
2020-10-13 13:05:16 1 [Note] WSREP: PC protocol downgrade 1 -> 0 |
2020-10-13 13:05:16 1 [Note] WSREP: view((empty)) |
2020-10-13 13:05:16 1 [Note] WSREP: gcomm: closed |
2020-10-13 13:05:16 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1 |
2020-10-13 13:05:16 0 [Note] WSREP: Flow-control interval: [16, 16] |
2020-10-13 13:05:16 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:05:16 0 [Note] WSREP: Shifting DONOR/DESYNCED -> OPEN (TO: 3286866011) |
2020-10-13 13:05:16 0 [Note] WSREP: New SELF-LEAVE. |
2020-10-13 13:05:16 0 [Note] WSREP: Flow-control interval: [0, 0] |
2020-10-13 13:05:16 0 [Note] WSREP: Received SELF-LEAVE. Closing connection. |
2020-10-13 13:05:16 0 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 3286866011) |
2020-10-13 13:05:16 0 [Note] WSREP: RECV thread exiting 0: Success |
2020-10-13 13:05:16 1 [Note] WSREP: recv_thread() joined. |
2020-10-13 13:05:16 1 [Note] WSREP: Closing replication queue. |
2020-10-13 13:05:16 1 [Note] WSREP: Closing slave action queue. |
2020-10-13 13:05:16 1 [Note] WSREP: ================================================ |
View:
|
id: 00000000-0000-0000-0000-000000000000:-1 |
status: non-primary
|
protocol_version: -1 |
capabilities:
|
final: yes |
own_index: -1 |
members(0): |
=================================================
|
2020-10-13 13:05:16 1 [Note] WSREP: Non-primary view |
2020-10-13 13:05:16 1 [Note] WSREP: Server status change synced -> disconnected |
2020-10-13 13:05:16 1 [Note] WSREP: Applier thread exiting ret: 6 thd: 1 |
2020-10-13 13:05:16 1 [Warning] Aborted connection 1 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) |
2020-10-13 13:22:29 0 [Note] /usr/sbin/mysqld (initiated by: root[root] @ localhost []): Normal shutdown |
2020-10-13 13:22:29 0 [Note] WSREP: Shutdown replication |
2020-10-13 13:22:29 0 [Note] WSREP: killing local connection: 43850 |
2020-10-13 13:22:29 0 [Note] WSREP: killing local connection: 43848 |
2020-10-13 13:22:29 0 [Note] WSREP: killing local connection: 42948 |
2020-10-13 13:22:29 0 [Note] WSREP: killing local connection: 38814 |
2020-10-13 13:22:30 2 [Note] WSREP: rollbacker thread exiting 2 |
2020-10-13 13:22:30 2 [Warning] Aborted connection 2 to db: 'unconnected' user: 'unauthenticated' host: '' (This connection closed normally without authentication) |
2020-10-13 13:22:30 0 [Note] Event Scheduler: Purging the queue. 0 events |
2020-10-13 13:22:30 0 [Note] InnoDB: FTS optimize thread exiting. |
2020-10-13 13:22:30 4 [Note] InnoDB: to purge 40 transactions |
2020-10-13 13:22:30 0 [Note] WSREP: dtor state: CLOSED |
2020-10-13 13:22:30 0 [Note] WSREP: MemPool(TrxHandleSlave): hit ratio: 0.999594, misses: 8359, in use: 67, in pool: 1983 |
2020-10-13 13:22:30 0 [Note] WSREP: mon: entered 20573805 oooe fraction 0 oool fraction 0 |
2020-10-13 13:22:30 0 [Note] WSREP: mon: entered 20573805 oooe fraction 0 oool fraction 0 |
2020-10-13 13:22:30 0 [Note] WSREP: mon: entered 20886836 oooe fraction 0 oool fraction 4.7877e-08 |
2020-10-13 13:22:30 0 [Note] WSREP: cert index usage at exit 90 |
2020-10-13 13:22:30 0 [Note] WSREP: cert trx map usage at exit 67 |
2020-10-13 13:22:30 0 [Note] WSREP: deps set usage at exit 0 |
2020-10-13 13:22:30 0 [Note] WSREP: avg deps dist 32.3305 |
2020-10-13 13:22:30 0 [Note] WSREP: avg cert interval 1.3031e+06 |
2020-10-13 13:22:30 0 [Note] WSREP: cert index size 90 |
2020-10-13 13:22:30 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:22:30 0 [Note] WSREP: wsdb trx map usage 0 conn query map usage 0 |
2020-10-13 13:22:30 0 [Note] WSREP: MemPool(LocalTrxHandle): hit ratio: 0, misses: 0, in use: 0, in pool: 0 |
2020-10-13 13:22:30 0 [Warning] WSREP: Waiting for 4 items to be fetched. |
2020-10-13 13:23:48 0 [Note] WSREP: Loading provider /usr/lib/libgalera_smm.so initial position: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866009 |
2020-10-13 13:23:48 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so' |
2020-10-13 13:23:48 0 [Note] WSREP: wsrep_load(): Galera 26.4.5(rb3764ab6) by Codership Oy <info@codership.com> loaded successfully. |
2020-10-13 13:23:48 0 [Note] WSREP: CRC-32C: using hardware acceleration. |
2020-10-13 13:23:48 0 [Note] WSREP: Found saved state: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866009, safe_to_bootstrap: 0 |
2020-10-13 13:23:48 0 [Note] WSREP: GCache DEBUG: opened preamble: |
Version: 2 |
UUID: 77d86ba4-c581-11e8-b013-173e23a1017d
|
Seqno: -1 - -1 |
Offset: -1 |
Synced: 0 |
2020-10-13 13:23:48 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: 77d86ba4-c581-11e8-b013-173e23a1017d, offset: -1 |
2020-10-13 13:23:48 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/2147483672 bytes) complete. |
2020-10-13 13:23:50 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (2147483672/2147483672 bytes) complete. |
2020-10-13 13:23:50 0 [Note] WSREP: Recovering GCache ring buffer: found gapless sequence 3285369036-3286866009 |
2020-10-13 13:23:50 0 [Note] WSREP: GCache::RingBuffer unused buffers scan... 0.0% ( 0/2147481200 bytes) complete. |
2020-10-13 13:23:50 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...100.0% (2147481200/2147481200 bytes) complete. |
2020-10-13 13:23:50 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): found 0/1496974 locked buffers |
2020-10-13 13:23:50 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): free space: 2448/2147483648 |
2020-10-13 13:23:50 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = db1-c2m4p.os; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT1M; evs.install_timeout = PT1M; evs.join_retrans_period = PT1S; evs.keepalive_period = PT3S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT30S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /opt/db1/tmp/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 2G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.listen_addr = tcp://[: |
2020-10-13 13:23:50 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:23:50 0 [Note] WSREP: ####### Assign initial position for certification: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866009, protocol version: -1 |
2020-10-13 13:23:50 0 [Note] WSREP: Start replication |
2020-10-13 13:23:50 0 [Note] WSREP: Connecting with bootstrap option: 0 |
2020-10-13 13:23:50 0 [Note] WSREP: Setting GCS initial position to 77d86ba4-c581-11e8-b013-173e23a1017d:3286866009 |
2020-10-13 13:23:50 0 [Note] WSREP: protonet asio version 0 |
2020-10-13 13:23:50 0 [Note] WSREP: Using CRC-32C for message checksums. |
2020-10-13 13:23:50 0 [Note] WSREP: backend: asio |
2020-10-13 13:23:50 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 |
2020-10-13 13:23:50 0 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) |
2020-10-13 13:23:50 0 [Note] WSREP: restore pc from disk failed |
2020-10-13 13:23:50 0 [Note] WSREP: GMCast version 0 |
2020-10-13 13:23:50 0 [Note] WSREP: (916e5530-991e, 'tcp://[::]:4567') listening at tcp://[::]:4567 |
2020-10-13 13:23:50 0 [Note] WSREP: (916e5530-991e, 'tcp://[::]:4567') multicast: , ttl: 1 |
2020-10-13 13:23:50 0 [Note] WSREP: EVS version 1 |
2020-10-13 13:23:50 0 [Note] WSREP: gcomm: connecting to group 'db1-c2mXp', peer 'db1-c2m4p.os.abv1:,db1-c2m3p.os.abv1:,db1-c2m2p.os.abv1:' |
2020-10-13 13:23:50 0 [Note] WSREP: (916e5530-991e, 'tcp://[::]:4567') Found matching local endpoint for a connection, blacklisting address tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 |
2020-10-13 13:23:53 0 [Note] WSREP: EVS version upgrade 0 -> 1 |
2020-10-13 13:23:53 0 [Note] WSREP: PC protocol upgrade 0 -> 1 |
2020-10-13 13:23:53 0 [Warning] WSREP: no nodes coming from prim view, prim not possible |
2020-10-13 13:23:53 0 [Note] WSREP: view(view_id(NON_PRIM,916e5530-991e,1) memb { |
916e5530-991e,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:23:53 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.50271S), skipping check |
2020-10-13 13:24:23 0 [Note] WSREP: PC protocol downgrade 1 -> 0 |
2020-10-13 13:24:23 0 [Note] WSREP: view((empty)) |
2020-10-13 13:24:23 0 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out) |
at gcomm/src/pc.cpp:connect():160 |
2020-10-13 13:24:23 0 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():220: Failed to open backend connection: -110 (Connection timed out) |
2020-10-13 13:24:23 0 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1632: Failed to open channel 'db1-c2mXp' at 'gcomm://db1-c2m4p.os.abv1,db1-c2m3p.os.abv1,db1-c2m2p.os.abv1': -110 (Connection timed out) |
2020-10-13 13:24:23 0 [ERROR] WSREP: gcs connect failed: Connection timed out |
2020-10-13 13:24:23 0 [ERROR] WSREP: wsrep::connect(gcomm://db1-c2m4p.os.abv1,db1-c2m3p.os.abv1,db1-c2m2p.os.abv1) failed: 7 |
2020-10-13 13:24:23 0 [ERROR] Aborting |
|
===== Node 4 log =====
1602587107 2020-10-13 13:05:07 /bin/bash [381/940/940] begin... |
2020-10-13 13:05:12 0 [Note] WSREP: Loading provider /usr/lib/libgalera_smm.so initial position: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340 |
2020-10-13 13:05:12 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so' |
2020-10-13 13:05:12 0 [Note] WSREP: wsrep_load(): Galera 26.4.5(rb3764ab6) by Codership Oy <info@codership.com> loaded successfully. |
2020-10-13 13:05:12 0 [Note] WSREP: CRC-32C: using hardware acceleration. |
2020-10-13 13:05:12 0 [Note] WSREP: Found saved state: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340, safe_to_bootstrap: 0 |
2020-10-13 13:05:12 0 [Note] WSREP: GCache DEBUG: opened preamble: |
Version: 2 |
UUID: 77d86ba4-c581-11e8-b013-173e23a1017d
|
Seqno: 2892554740 - 2893434340 |
Offset: 1280 |
Synced: 1 |
2020-10-13 13:05:12 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: 77d86ba4-c581-11e8-b013-173e23a1017d, offset: 1280 |
2020-10-13 13:05:12 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/2147483672 bytes) complete. |
2020-10-13 13:05:13 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (2147483672/2147483672 bytes) complete. |
2020-10-13 13:05:13 0 [Note] WSREP: Recovering GCache ring buffer: found gapless sequence 2892554740-2893434340 |
2020-10-13 13:05:13 0 [Note] WSREP: GCache::RingBuffer unused buffers scan... 0.0% ( 0/2063254672 bytes) complete. |
2020-10-13 13:05:13 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...100.0% (2063254672/2063254672 bytes) complete. |
2020-10-13 13:05:13 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): found 255/879856 locked buffers |
2020-10-13 13:05:13 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): free space: 84281592/2147483648 |
2020-10-13 13:05:13 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = db1-c2m2d.os.acsc2012; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT1M; evs.install_timeout = PT1M; evs.join_retrans_period = PT1S; evs.keepalive_period = PT3S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT30S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /opt/db1/tmp/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 2G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.listen_addr = |
2020-10-13 13:05:13 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:13 0 [Note] WSREP: ####### Assign initial position for certification: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340, protocol version: -1 |
2020-10-13 13:05:13 0 [Note] WSREP: Start replication |
2020-10-13 13:05:13 0 [Note] WSREP: Connecting with bootstrap option: 0 |
2020-10-13 13:05:13 0 [Note] WSREP: Setting GCS initial position to 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340 |
2020-10-13 13:05:13 0 [Note] WSREP: protonet asio version 0 |
2020-10-13 13:05:13 0 [Note] WSREP: Using CRC-32C for message checksums. |
2020-10-13 13:05:13 0 [Note] WSREP: backend: asio |
2020-10-13 13:05:13 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 |
2020-10-13 13:05:13 0 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory) |
2020-10-13 13:05:13 0 [Note] WSREP: restore pc from disk failed |
2020-10-13 13:05:13 0 [Note] WSREP: GMCast version 0 |
2020-10-13 13:05:13 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') listening at tcp://[::]:4567 |
2020-10-13 13:05:13 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') multicast: , ttl: 1 |
2020-10-13 13:05:13 0 [Note] WSREP: EVS version 1 |
2020-10-13 13:05:13 0 [Note] WSREP: gcomm: connecting to group 'db1-c2mXp', peer 'db1-c2m3p.os:' |
2020-10-13 13:05:13 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') connection established to c7ca3602-9def tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 |
2020-10-13 13:05:13 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') connection established to b51bd9d6-8d21 tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: EVS version upgrade 0 -> 1 |
2020-10-13 13:05:14 0 [Note] WSREP: declaring 68c69353-9bc1 at tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:05:14 0 [Note] WSREP: PC protocol upgrade 0 -> 1 |
2020-10-13 13:05:14 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') connection established to 68c69353-9bc1 tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567 |
2020-10-13 13:05:14 0 [Note] WSREP: Node 68c69353-9bc1 state prim |
2020-10-13 13:05:14 0 [Note] WSREP: view(view_id(PRIM,68c69353-9bc1,156) memb { |
68c69353-9bc1,0 |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:05:14 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:14 0 [Note] WSREP: gcomm: connected |
2020-10-13 13:05:14 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 |
2020-10-13 13:05:14 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) |
2020-10-13 13:05:14 0 [Note] WSREP: Opened channel 'db1-c2mXp' |
2020-10-13 13:05:14 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 3, memb_num = 4 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 0 (db1-c2m4p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 1 (db1-c2m2p) |
2020-10-13 13:05:14 1 [Note] WSREP: Starting rollbacker thread 1 |
2020-10-13 13:05:14 2 [Note] WSREP: Starting applier thread 2 |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 2 (db1-c2m3p) |
2020-10-13 13:05:14 0 [Note] WSREP: STATE EXCHANGE: got state msg: f80d5f91-0d43-11eb-8f13-1b5f9b2a5ec1 from 3 (db1-c2m2d) |
2020-10-13 13:05:14 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 65, |
members = 3/4 (joined/total), |
act_id = 3286865882, |
last_appl. = 3286865813, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:14 0 [Note] WSREP: Flow-control interval: [32, 32] |
2020-10-13 13:05:14 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 3286865883) |
2020-10-13 13:05:14 2 [Note] WSREP: ####### processing CC 3286865883, local, ordered |
2020-10-13 13:05:14 2 [Note] WSREP: Process first view: 77d86ba4-c581-11e8-b013-173e23a1017d my uuid: f7b801a1-0d43-11eb-a84f-878d2f0020e7 |
2020-10-13 13:05:14 2 [Note] WSREP: Server db1-c2m2d connected to cluster at position 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883 with ID f7b801a1-0d43-11eb-a84f-878d2f0020e7 |
2020-10-13 13:05:14 2 [Note] WSREP: Server status change disconnected -> connected |
2020-10-13 13:05:15 2 [Note] WSREP: ####### My UUID: f7b801a1-0d43-11eb-a84f-878d2f0020e7 |
2020-10-13 13:05:15 2 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes |
2020-10-13 13:05:15 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:05:15 2 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1 |
2020-10-13 13:05:15 2 [Note] WSREP: State transfer required: |
Group state: 77d86ba4-c581-11e8-b013-173e23a1017d:3286865883 |
Local state: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340 |
2020-10-13 13:05:15 2 [Note] WSREP: Server status change connected -> joiner |
2020-10-13 13:05:15 0 [Note] WSREP: Joiner monitor thread started to monitor |
2020-10-13 13:05:15 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address 'db1-c2m2d.os.acsc2012' --datadir '/var/lib/mysql/' --parent '31744' --binlog 'mysql-bin' --binlog-index 'mysql-bin.index' --mysqld-args --wsrep_start_position=77d86ba4-c581-11e8-b013-173e23a1017d:2893434340' |
WSREP_SST: [INFO] Logging all stderr of SST/Innobackupex to syslog (20201013 13:05:15.742) |
2020-10-13 13:05:16 2 [Note] WSREP: Prepared SST request: mariabackup|db1-c2m2d.os.acsc2012:4444/xtrabackup_sst//1 |
2020-10-13 13:05:16 2 [Note] WSREP: ####### IST uuid:77d86ba4-c581-11e8-b013-173e23a1017d f: 2893434341, l: 3286865883, STRv: 3 |
2020-10-13 13:05:16 2 [Note] WSREP: IST receiver addr using tcp://db1-c2m2d.os.acsc2012:4568 |
2020-10-13 13:05:16 2 [Note] WSREP: Prepared IST receiver for 2893434341-3286865883, listening at: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4568 |
2020-10-13 13:05:16 0 [Note] WSREP: Member 3.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m4p)(SYNCED) as donor. |
2020-10-13 13:05:16 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 3286866009) |
2020-10-13 13:05:16 2 [Note] WSREP: Requesting state transfer: success, donor: 0 |
2020-10-13 13:05:16 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:05:16 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:05:16 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:17 0 [Note] WSREP: (f7b801a1-a84f, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:19 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:05:19 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,157) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a84f,0 |
} joined {
|
} left {
|
} partitioned {
|
68c69353-9bc1,0 |
})
|
2020-10-13 13:05:19 0 [Note] WSREP: save pc into disk |
2020-10-13 13:05:19 0 [Note] WSREP: forgetting 68c69353-9bc1 (tcp://[****:8400:4:2c3:216:3eff:fe73:266e]:4567) |
2020-10-13 13:05:19 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: sent state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 1 (db1-c2m3p) |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 0 (db1-c2m2p) |
2020-10-13 13:05:19 0 [Note] WSREP: STATE EXCHANGE: got state msg: fb14cfe3-0d43-11eb-af77-078aeb4bcaf4 from 2 (db1-c2m2d) |
2020-10-13 13:05:19 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 66, |
members = 2/3 (joined/total), |
act_id = 3286866015, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:05:19 0 [Warning] WSREP: Donor 68c69353-09a8-11eb-9bc1-3376b5cfc3ea is no longer in the group. State transfer cannot be completed, need to abort. Aborting... |
2020-10-13 13:05:19 0 [Note] WSREP: /usr/sbin/mysqld: Terminated. |
2020-10-13 13:05:37 0 [Note] WSREP: Loading provider /usr/lib/libgalera_smm.so initial position: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340 |
2020-10-13 13:05:37 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so' |
2020-10-13 13:05:37 0 [Note] WSREP: wsrep_load(): Galera 26.4.5(rb3764ab6) by Codership Oy <info@codership.com> loaded successfully. |
2020-10-13 13:05:37 0 [Note] WSREP: CRC-32C: using hardware acceleration. |
2020-10-13 13:05:37 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 0 |
2020-10-13 13:05:37 0 [Note] WSREP: GCache DEBUG: opened preamble: |
Version: 2 |
UUID: 77d86ba4-c581-11e8-b013-173e23a1017d
|
Seqno: -1 - -1 |
Offset: -1 |
Synced: 0 |
2020-10-13 13:05:37 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: 77d86ba4-c581-11e8-b013-173e23a1017d, offset: -1 |
2020-10-13 13:05:37 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/2147483672 bytes) complete. |
2020-10-13 13:05:39 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (2147483672/2147483672 bytes) complete. |
2020-10-13 13:05:39 0 [Note] WSREP: Recovering GCache ring buffer: found gapless sequence 2892554740-2893434340 |
2020-10-13 13:05:39 0 [Note] WSREP: GCache::RingBuffer unused buffers scan... 0.0% ( 0/2146704032 bytes) complete. |
2020-10-13 13:05:39 0 [Note] WSREP: GCache::RingBuffer unused buffers scan...100.0% (2146704032/2146704032 bytes) complete. |
2020-10-13 13:05:39 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): found 513/880114 locked buffers |
2020-10-13 13:05:39 0 [Note] WSREP: GCache DEBUG: RingBuffer::recover(): free space: 84281592/2147483648 |
2020-10-13 13:05:40 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = db1-c2m2d.os.acsc2012; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT1M; evs.install_timeout = PT1M; evs.join_retrans_period = PT1S; evs.keepalive_period = PT3S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT30S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /opt/db1/tmp/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 2G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.listen_addr = |
2020-10-13 13:05:40 0 [Note] WSREP: Start replication |
2020-10-13 13:05:40 0 [Note] WSREP: Connecting with bootstrap option: 0 |
2020-10-13 13:05:40 0 [Note] WSREP: Setting GCS initial position to 00000000-0000-0000-0000-000000000000:-1 |
2020-10-13 13:05:40 0 [Note] WSREP: protonet asio version 0 |
2020-10-13 13:05:40 0 [Note] WSREP: Using CRC-32C for message checksums. |
2020-10-13 13:05:40 0 [Note] WSREP: backend: asio |
2020-10-13 13:05:40 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 |
2020-10-13 13:05:40 0 [Note] WSREP: restore pc from disk successfully |
2020-10-13 13:05:40 0 [Note] WSREP: GMCast version 0 |
2020-10-13 13:05:40 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') listening at tcp://[::]:4567 |
2020-10-13 13:05:40 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') multicast: , ttl: 1 |
2020-10-13 13:05:40 0 [Note] WSREP: EVS version 1 |
2020-10-13 13:05:40 0 [Note] WSREP: gcomm: connecting to group 'db1-c2mXp', peer 'db1-c2m3p.os:' |
2020-10-13 13:05:40 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') connection established to c7ca3602-9def tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 |
2020-10-13 13:05:40 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:05:40 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') connection established to b51bd9d6-8d21 tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:05:43 0 [Note] WSREP: gcomm: connected |
2020-10-13 13:05:43 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 |
2020-10-13 13:05:43 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) |
2020-10-13 13:05:43 0 [Note] WSREP: Opened channel 'db1-c2mXp' |
2020-10-13 13:05:43 2 [Note] WSREP: Starting rollbacker thread 2 |
2020-10-13 13:05:43 1 [Note] WSREP: Starting applier thread 1 |
2020-10-13 13:05:43 0 [Warning] WSREP: last inactive check more than PT1.5S ago (PT3.19859S), skipping check |
2020-10-13 13:05:44 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:05:50 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 |
2020-10-13 13:05:51 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') reconnecting to b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567), attempt 0 |
2020-10-13 13:05:51 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') reconnecting to c7ca3602-9def (tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567), attempt 0 |
2020-10-13 13:05:51 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') connection established to b51bd9d6-8d21 tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:05:51 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') connection established to c7ca3602-9def tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 |
2020-10-13 13:05:53 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') reconnecting to b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567), attempt 0 |
2020-10-13 13:05:53 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') reconnecting to c7ca3602-9def (tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567), attempt 0 |
2020-10-13 13:05:53 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') connection established to b51bd9d6-8d21 tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:05:53 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') connection established to c7ca3602-9def tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 |
2020-10-13 13:05:56 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:06:21 0 [Note] WSREP: EVS version upgrade 0 -> 1 |
2020-10-13 13:06:21 0 [Note] WSREP: declaring b51bd9d6-8d21 at tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 stable |
2020-10-13 13:06:21 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:06:21 0 [Note] WSREP: PC protocol upgrade 0 -> 1 |
2020-10-13 13:06:21 0 [Note] WSREP: Node b51bd9d6-8d21 state prim |
2020-10-13 13:06:21 0 [Note] WSREP: view(view_id(PRIM,b51bd9d6-8d21,159) memb { |
b51bd9d6-8d21,0 |
c7ca3602-9def,0 |
f7b801a1-a850,0 |
} joined {
|
} left {
|
} partitioned {
|
})
|
2020-10-13 13:06:21 0 [Note] WSREP: save pc into disk |
2020-10-13 13:06:21 0 [Note] WSREP: clear restored view |
2020-10-13 13:06:21 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 2, memb_num = 3 |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Warning] WSREP: FLOW message from member 1 in non-primary configuration. Ignored. |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 0 (db1-c2m2p) |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 2 (db1-c2m2d) |
2020-10-13 13:06:21 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2058e43e-0d44-11eb-8787-1f99ecd795c6 from 1 (db1-c2m3p) |
2020-10-13 13:06:21 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 67, |
members = 2/3 (joined/total), |
act_id = 3286866099, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:06:21 0 [Note] WSREP: Flow-control interval: [28, 28] |
2020-10-13 13:06:21 0 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 3286866100) |
2020-10-13 13:06:21 1 [Note] WSREP: ####### processing CC 3286866100, local, ordered |
2020-10-13 13:06:21 1 [Note] WSREP: Process first view: 77d86ba4-c581-11e8-b013-173e23a1017d my uuid: f7b801a1-0d43-11eb-a850-878d2f0020e7 |
2020-10-13 13:06:21 1 [Note] WSREP: Server db1-c2m2d connected to cluster at position 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100 with ID f7b801a1-0d43-11eb-a850-878d2f0020e7 |
2020-10-13 13:06:21 1 [Note] WSREP: Server status change disconnected -> connected |
2020-10-13 13:06:22 1 [Note] WSREP: ####### My UUID: f7b801a1-0d43-11eb-a850-878d2f0020e7 |
2020-10-13 13:06:22 1 [Note] WSREP: Cert index reset to 00000000-0000-0000-0000-000000000000:-1 (proto: 10), state transfer needed: yes |
2020-10-13 13:06:22 0 [Note] WSREP: Service thread queue flushed. |
2020-10-13 13:06:22 1 [Note] WSREP: ####### Assign initial position for certification: 00000000-0000-0000-0000-000000000000:-1, protocol version: -1 |
2020-10-13 13:06:22 1 [Note] WSREP: State transfer required: |
Group state: 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100 |
Local state: 00000000-0000-0000-0000-000000000000:-1 |
2020-10-13 13:06:22 1 [Note] WSREP: Server status change connected -> joiner |
2020-10-13 13:06:22 0 [Note] WSREP: Joiner monitor thread started to monitor |
2020-10-13 13:06:22 0 [Note] WSREP: Running: 'wsrep_sst_mariabackup --role 'joiner' --address 'db1-c2m2d.os.acsc2012' --datadir '/var/lib/mysql/' --parent '32460' --binlog 'mysql-bin' --binlog-index 'mysql-bin.index' --mysqld-args --wsrep_start_position=77d86ba4-c581-11e8-b013-173e23a1017d:2893434340' |
WSREP_SST: [INFO] Logging all stderr of SST/Innobackupex to syslog (20201013 13:06:22.960) |
2020-10-13 13:06:23 1 [Note] WSREP: Prepared SST request: mariabackup|db1-c2m2d.os.acsc2012:4444/xtrabackup_sst//1 |
2020-10-13 13:06:23 1 [Note] WSREP: ####### IST uuid:00000000-0000-0000-0000-000000000000 f: 0, l: 3286866100, STRv: 3 |
2020-10-13 13:06:23 1 [Note] WSREP: IST receiver addr using tcp://db1-c2m2d.os.acsc2012:4568 |
2020-10-13 13:06:23 1 [Note] WSREP: Prepared IST receiver for 0-3286866100, listening at: tcp://[****:1d64:e1cb:2e4:216:3eff:fe7d:6501]:4568 |
2020-10-13 13:06:29 0 [Note] WSREP: Member 2.0 (db1-c2m2d) requested state transfer from '*any*'. Selected 0.0 (db1-c2m2p)(SYNCED) as donor. |
2020-10-13 13:06:29 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 3286866166) |
2020-10-13 13:06:29 1 [Note] WSREP: Requesting state transfer: success, donor: 0 |
2020-10-13 13:06:29 1 [Note] WSREP: Resetting GCache seqno map due to different histories. |
2020-10-13 13:06:29 1 [Note] WSREP: GCache history reset: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340 -> 77d86ba4-c581-11e8-b013-173e23a1017d:3286866100 |
2020-10-13 13:06:29 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567 |
2020-10-13 13:06:29 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:06:29 0 [Note] WSREP: forgetting b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567) |
2020-10-13 13:06:29 0 [Note] WSREP: (f7b801a1-a850, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:06:29 0 [Note] WSREP: Node c7ca3602-9def state prim |
2020-10-13 13:06:29 0 [Note] WSREP: view(view_id(PRIM,c7ca3602-9def,160) memb { |
c7ca3602-9def,0 |
f7b801a1-a850,0 |
} joined {
|
} left {
|
} partitioned {
|
b51bd9d6-8d21,0 |
})
|
2020-10-13 13:06:29 0 [Note] WSREP: save pc into disk |
2020-10-13 13:06:29 0 [Note] WSREP: forgetting b51bd9d6-8d21 (tcp://[****:8400:4:2c3:216:3eff:fe7d:6424]:4567) |
2020-10-13 13:06:30 1 [Note] WSREP: GCache DEBUG: RingBuffer::seqno_reset(): discarded 2063201848 bytes |
2020-10-13 13:06:30 1 [Note] WSREP: GCache DEBUG: RingBuffer::seqno_reset(): found 1/68 locked buffers |
2020-10-13 13:06:30 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2 |
2020-10-13 13:06:30 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. |
2020-10-13 13:06:30 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 |
2020-10-13 13:06:30 0 [Note] WSREP: STATE EXCHANGE: got state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 from 0 (db1-c2m3p) |
2020-10-13 13:06:30 0 [Note] WSREP: STATE EXCHANGE: got state msg: 253a3cb3-0d44-11eb-bf81-b6c941f0a1d5 from 1 (db1-c2m2d) |
2020-10-13 13:06:30 0 [Note] WSREP: Quorum results: |
version = 6, |
component = PRIMARY,
|
conf_id = 68, |
members = 1/2 (joined/total), |
act_id = 3286866279, |
last_appl. = 3286865942, |
protocols = 2/10/4 (gcs/repl/appl), |
vote policy= 0, |
group UUID = 77d86ba4-c581-11e8-b013-173e23a1017d
|
2020-10-13 13:06:30 0 [Warning] WSREP: Donor b51bd9d6-09a7-11eb-8d21-9aa8b7de5bcf is no longer in the group. State transfer cannot be completed, need to abort. Aborting... |
2020-10-13 13:06:30 0 [Note] WSREP: /usr/sbin/mysqld: Terminated. |
2020-10-13 13:06:46 0 [Note] WSREP: Loading provider /usr/lib/libgalera_smm.so initial position: 77d86ba4-c581-11e8-b013-173e23a1017d:2893434340 |
2020-10-13 13:06:46 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so' |
2020-10-13 13:06:46 0 [Note] WSREP: wsrep_load(): Galera 26.4.5(rb3764ab6) by Codership Oy <info@codership.com> loaded successfully. |
2020-10-13 13:06:46 0 [Note] WSREP: CRC-32C: using hardware acceleration. |
2020-10-13 13:06:46 0 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1, safe_to_bootstrap: 0 |
2020-10-13 13:06:46 0 [Note] WSREP: GCache DEBUG: opened preamble: |
Version: 2 |
UUID: 77d86ba4-c581-11e8-b013-173e23a1017d
|
Seqno: -1 - -1 |
Offset: -1 |
Synced: 0 |
2020-10-13 13:06:46 0 [Note] WSREP: Recovering GCache ring buffer: version: 2, UUID: 77d86ba4-c581-11e8-b013-173e23a1017d, offset: -1 |
2020-10-13 13:06:46 0 [Note] WSREP: GCache::RingBuffer initial scan... 0.0% ( 0/2147483672 bytes) complete. |
2020-10-13 13:06:46 0 [Note] WSREP: GCache::RingBuffer initial scan...100.0% (2147483672/2147483672 bytes) complete. |
2020-10-13 13:06:46 0 [Note] WSREP: Recovering GCache ring buffer: didn't recover any events. |
2020-10-13 13:06:46 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = db1-c2m2d.os.acsc2012; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT1M; evs.install_timeout = PT1M; evs.join_retrans_period = PT1S; evs.keepalive_period = PT3S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT30S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /opt/db1/tmp/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = galera.cache; gcache.page_size = 128M; gcache.recover = yes; gcache.size = 2G; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.listen_addr = |
2020-10-13 13:06:46 0 [Note] WSREP: Start replication |
2020-10-13 13:06:46 0 [Note] WSREP: Connecting with bootstrap option: 0 |
2020-10-13 13:06:46 0 [Note] WSREP: Setting GCS initial position to 00000000-0000-0000-0000-000000000000:-1 |
2020-10-13 13:06:46 0 [Note] WSREP: protonet asio version 0 |
2020-10-13 13:06:46 0 [Note] WSREP: Using CRC-32C for message checksums. |
2020-10-13 13:06:46 0 [Note] WSREP: backend: asio |
2020-10-13 13:06:46 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 |
2020-10-13 13:06:46 0 [Note] WSREP: restore pc from disk successfully |
2020-10-13 13:06:46 0 [Note] WSREP: GMCast version 0 |
2020-10-13 13:06:46 0 [Note] WSREP: (f7b801a1-a851, 'tcp://[::]:4567') listening at tcp://[::]:4567 |
2020-10-13 13:06:46 0 [Note] WSREP: (f7b801a1-a851, 'tcp://[::]:4567') multicast: , ttl: 1 |
2020-10-13 13:06:46 0 [Note] WSREP: EVS version 1 |
2020-10-13 13:06:46 0 [Note] WSREP: gcomm: connecting to group 'db1-c2mXp', peer 'db1-c2m3p.os:' |
2020-10-13 13:06:46 0 [Note] WSREP: (f7b801a1-a851, 'tcp://[::]:4567') connection established to c7ca3602-9def tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 |
2020-10-13 13:06:46 0 [Note] WSREP: (f7b801a1-a851, 'tcp://[::]:4567') turning message relay requesting on, nonlive peers: |
2020-10-13 13:06:47 0 [Note] WSREP: gcomm: connected |
2020-10-13 13:06:47 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636 |
2020-10-13 13:06:47 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0) |
2020-10-13 13:06:47 0 [Note] WSREP: Opened channel 'db1-c2mXp' |
2020-10-13 13:06:47 1 [Note] WSREP: Starting rollbacker thread 1 |
2020-10-13 13:06:47 2 [Note] WSREP: Starting applier thread 2 |
2020-10-13 13:06:50 0 [Note] WSREP: (f7b801a1-a851, 'tcp://[::]:4567') turning message relay requesting off |
2020-10-13 13:07:32 0 [Note] WSREP: EVS version upgrade 0 -> 1 |
2020-10-13 13:07:32 0 [Note] WSREP: declaring c7ca3602-9def at tcp://[****:8400:4:2c3:216:3eff:fef0:d954]:4567 stable |
2020-10-13 13:07:32 0 [Note] WSREP: PC protocol upgrade 0 -> 1 |
2020-10-13 13:07:32 0 [Note] WSREP: view(view_id(NON_PRIM,c7ca3602-9def,162) memb { |
c7ca3602-9def,0 |
f7b801a1-a851,0 |
} joined {
|
} left {
|
} partitioned {
|
f7b801a1-a850,0 |
})
|
2020-10-13 13:07:32 0 [Warning] WSREP: node uuid: c7ca3602-9def last_prim(type: 3, uuid: c7ca3602-9def) is inconsistent to restored view(type: V_NON_PRIM, uuid: c7ca3602-9def |
2020-10-13 13:07:32 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 1, memb_num = 2 |
2020-10-13 13:07:32 0 [Note] WSREP: Flow-control interval: [23, 23] |
2020-10-13 13:07:32 0 [Note] WSREP: Received NON-PRIMARY. |
2020-10-13 13:07:32 2 [Note] WSREP: ================================================ |
View:
|
id: 00000000-0000-0000-0000-000000000000:-1 |
status: non-primary
|
protocol_version: -1 |
capabilities:
|
final: no |
own_index: 1 |
members(2): |
0: c7ca3602-0cc5-11eb-9def-3a8a6cbcfe44, unspecified |
1: f7b801a1-0d43-11eb-a851-878d2f0020e7, db1-c2m2d |
=================================================
|
2020-10-13 13:07:32 2 [Note] WSREP: Non-primary view |
2020-10-13 13:07:32 2 [Note] WSREP: Server status change disconnected -> connected |
|
Attachments
Issue Links
- relates to
-
-
- Open
-