Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
11.4.0
-
None
-
None
-
None
-
Debian GNU/Linux 12 (bookworm)
Description
We are running a 3 node cluster, with wsrep_slave_threads=4 set.
After an index change in the database we see this error: "WSREP: MDL BF-BF conflict". The Cluster is falling apart. The Cluster Status is this afterwards:
node-1: wesrep_cluster_size = 1
node-2: wesrep_cluster_size = 2
node-3: wesrep_cluster_size = 2
But only node-1 is the real one, the other two has no connection to node-1 anymore.
"cinder-cinder-db-2","2025-02-25T08:22:11.158576901Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: declaring 3f24b45d-a4d0 at ssl://XXX.XXX.XXX.XXX:4567 stable"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.15859764Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: forgetting 55a20c81-a890 (ssl://10XXX.XXX.XXX.XXX:4567)"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.159083462Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: Node 3f24b45d-a4d0 state prim"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.16014208Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: view(view_id(PRIM,3f24b45d-a4d0,1444) memb {"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160162177Z stderr F 3f24b45d-a4d0,0"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160168279Z stderr F 6a7bfa18-9efe,0"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160173138Z stderr F } joined {"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160177175Z stderr F } left {"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160181564Z stderr F } partitioned {"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160185351Z stderr F 55a20c81-a890,0"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160189088Z stderr F })"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160193035Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: save pc into disk"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160654241Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: forgetting 55a20c81-a890 (ssl://XXX.XXX.XXX.XXX:4567)"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160676904Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.160685109Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID."
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161071815Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 9c476a1c-f351-11ef-ad85-dabb0dc0cc85"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.16144705Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: STATE EXCHANGE: got state msg: 9c476a1c-f351-11ef-ad85-dabb0dc0cc85 from 0 (db-0.XXX)"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161460215Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: STATE EXCHANGE: got state msg: 9c476a1c-f351-11ef-ad85-dabb0dc0cc85 from 1 (db-2.XXX)"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161464653Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: Quorum results:"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161470234Z stderr F version = 6,"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161476065Z stderr F component = PRIMARY,"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161480553Z stderr F conf_id = 1437,"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161485803Z stderr F members = 2/2 (joined/total),"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161489359Z stderr F act_id = 181296661,"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161493137Z stderr F last_appl. = 181296600,"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161497034Z stderr F protocols = 5/11/4 (gcs/repl/appl),"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161500571Z stderr F vote policy= 0,"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161504157Z stderr F group UUID = 4cff59e7-272e-11ef-acfd-2fa7271b71c2"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161517422Z stderr F 2025-02-25 8:22:11 0 [Note] WSREP: Flow-control interval: [181, 226]"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.161547849Z stderr F 2025-02-25 8:22:11 6 [Note] WSREP: ####### processing CC 181296662, local, ordered"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.172985115Z stderr F 2025-02-25 8:22:11 9 [Note] WSREP: MDL BF-BF conflict"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.173001996Z stderr F schema: cinder"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.173017395Z stderr F request: (9 seqno 181296659 wsrep (toi, exec, committed) cmd 0 2 CREATE INDEX volumes_deleted_project_id_idx ON volumes (deleted, project_id))"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.173028336Z stderr F granted: (7 seqno 181296660 wsrep (high priority, exec, committing) cmd 0 161 (null))"
|
"cinder-cinder-db-2","2025-02-25T08:22:11.173067659Z stderr F 2025-02-25 8:22:11 9 [ERROR] Aborting"
|
"cinder-cinder-db-2","2025-02-25T08:47:00.034222022Z stderr F 2025-02-25 8:47:00 70223 [Warning] WSREP: Node desync failed: File descriptor in bad state"
|
"cinder-cinder-db-2","2025-02-25T08:47:00.034266646Z stderr F at /bitnami/blacksmith-sandox/libgalera-26.4.21/galera/src/replicator_smm.cpp:desync():3164"
|
"cinder-cinder-db-2","2025-02-25T08:47:00.034275372Z stderr F ERROR 1396 (HY000) at line 1: Operation 'desync' failed for SET GLOBAL wsrep_desync = ON"
|
"cinder-cinder-db-2","2025-02-25T08:47:00.034282986Z stderr F 2025-02-25 8:47:00 70223 [Warning] WSREP: SET desync failed 1 for schema: (null), query: SET GLOBAL wsrep_desync = ON"
|
we could resolve it, by setting wsrep_slave_threads=1.
Attachments
Issue Links
- duplicates
-
MDEV-28452 wsrep_ready: OFF after MDL BF-BF conflict
-
- Closed
-
- relates to
-
MDEV-36123 WSREP: MDL BF-BF conflict
-
- Open
-