[MDEV-8619] flooding conflict messages logged Created: 2015-08-14  Updated: 2015-11-20  Resolved: 2015-11-20

Status: Closed
Project: MariaDB Server
Component/s: Galera, wsrep
Affects Version/s: 10.0.14-galera
Fix Version/s: 10.0.22-galera

Type: Bug Priority: Major
Reporter: aftab khan Assignee: Nirbhay Choubey (Inactive)
Resolution: Fixed Votes: 0
Labels: None


 Description   

Our error log is filled with following warnings:

150808  0:00:07 [Warning] WSREP: thd 428267 seqno 33550061: conflict state 7 after post commit
150808  0:00:21 [Warning] WSREP: thd 428267 seqno 33550062: conflict state 7 after post commit
150808  0:05:21 [Warning] WSREP: thd 428267 seqno 33550407: conflict state 7 after post commit
150808  0:08:20 [Warning] WSREP: thd 428267 seqno 33550660: conflict state 7 after post commit
150808  0:10:21 [Warning] WSREP: thd 428267 seqno 33550810: conflict state 7 after post commit
150808  0:15:22 [Warning] WSREP: thd 428267 seqno 33551109: conflict state 7 after post commit
 

and it seems it gets logged per every single transaction. Then we saw replication stopped with following error:

 150813  4:31:09 [Note] WSREP: New cluster view: global state:
 3e1a0462-2948-11e5-bea6-a2ee7ea12324:40509703, view# -1: non-Primary, 
 number of nodes: 1, my index: 0, protocol version 3
 150813  4:31:09 [Note] WSREP: wsrep_notify_cmd is not defined, 
 skipping notification.
 150813  4:31:09 [ERROR] Slave SQL: Error 'Deadlock found when trying 
 to get lock; try restarting transaction' on query. Default database: 'PRODUCTION'.
 Query: 'COMMIT', Gtid 0-1861481499-843451992, Internal MariaDB error code:
 1213
 150813  4:31:09 [ERROR] Slave SQL: Error 'WSREP has not yet prepared 
 node for application use' on query. Default database: 'PRODUCTION'. Query:
 'COMMIT', Gtid 0-1861481499-843451992, Internal MariaDB error code: 
 1047
 150813  4:31:09 [Warning] Slave: WSREP has not yet prepared node for 
 application use Error_code: 1047
 150813  4:31:09 [ERROR] Error running query, slave SQL thread aborted. 
 Fix the problem, and restart the slave SQL thread with "SLAVE START". 
 We stopped at log 'mariadb-bin.000498' position 849071251
 150813  4:31:09 [Note] WSREP: (1def0146-2a01-11e5-be80-ba43957a95cc,
 'tcp://0.0.0.0:4567') address 'tcp://10.16.10.80:4567' pointing to 
 uuid 1def0146-2a01-11e5-be80-ba43957a95cc is blacklisted, skipping
 150813  4:31:09 [Note] WSREP: (1def0146-2a01-11e5-be80-ba43957a95cc,
 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
 tcp://10.16.10.81:4567 tcp://10.16.10.82:4567

However, some of the such warnings are related to when we had network outage due to network component upgrade (e.g. switches and cables). Please ignore warnings between:

150807 21:55:26 [Warning] WSREP: thd 428267 seqno 33541593: conflict state 7 after post commit
..
150807 23:57:27 [Warning] WSREP: thd 428267 seqno 33549911: conflict state 7 after post commit

I have enclosed error log from all three nodes. Currently we do not write to this cluster, it only receives updates from replication master (standalone server).



 Comments   
Comment by aftab khan [ 2015-08-14 ]

compressed error log file size is 23mb, therefore it cannot be attached here due to max size limit (10mb)

Comment by aftab khan [ 2015-09-02 ]

This issue is reproduceable:

https://github.com/codership/mysql-wsrep/issues/181

Comment by Nirbhay Choubey (Inactive) [ 2015-11-20 ]

The fixes have been merged from Codership branch. Marking it a resolved.

Generated at Thu Feb 08 07:28:30 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.