Details
-
Bug
-
Status: Closed (View Workflow)
-
Major
-
Resolution: Incomplete
-
10.2.22
-
centos 7
Description
since upgrade of our galera servers to 10.2.x, we experience global crash of nodes due to database inconsistency propagated through galera. This happens about 3 times a month and put our cluster in a very unstable situation as it requires a full SST that is time and IO consuming.
It always has the same scenario:
a client on one node (here nodesdb2) is inserting data that triggers unique key constraint failure (duplicate), leading to client rollbacking it's operations.
However this insert is propagated to other node (here nodesdb1) as if it was approved, and the secondary node crashes due to inconsistency of content.
After this, mariadb stops on the affected node and, on restart, triggers a full SST.
Note that we currently do all insert / select / delete operation on a single node (nodesdb2) and the other nodes (nodesdb1, the crashing node) is currently used only for redundancy.
May 11 04:40:39 nodesdb1.myhost.com mysqld[412786]: 2019-05-11 4:40:39 139660305315584 [Warning] WSREP: Failed to report last committed 205740288, -110 (Connection timed out)
|
May 11 04:40:41 nodesdb1.myhost.com mysqld[412786]: 2019-05-11 4:40:41 139660305315584 [Warning] WSREP: Failed to report last committed 205740332, -110 (Connection timed out)
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error co
|
de: 1062
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: Failed to apply app buffer: seqno: 209185646, status: 1
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: at galera/src/trx_handle.cpp:apply():353
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: Retrying 2th time
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: Failed to apply app buffer: seqno: 209185646, status: 1
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: at galera/src/trx_handle.cpp:apply():353
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: Retrying 3th time
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: Failed to apply app buffer: seqno: 209185646, status: 1
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: at galera/src/trx_handle.cpp:apply():353
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: Retrying 4th time
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] WSREP: Failed to apply trx: source: e5c9c2b5-400f-11e9-ba3d-8f8054a48006 version: 4 local: 0 state: APPLYING flags: 1 conn_id: 10193487 trx_id: 20898181434 seqnos (l: 34565981, g: 209185646, s: 209185645, d: 209185626, ts: 14759991015976877)
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] WSREP: Failed to apply trx 209185646 4 times
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] WSREP: Node consistency compromised, aborting...
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Note] WSREP: Closing send monitor...
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Note] WSREP: Closed send monitor.
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Note] WSREP: gcomm: terminating thread
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Note] WSREP: gcomm: joining thread
|
May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Note] WSREP: gcomm: closing backend
|
This always happen on the same table "tags_history" which is our biggest table: about 1 445 158 186 rows, 500G storage size and has an estimated 1200 rows inserted and deleted per seconds .
journalctl -u mariadb | grep Write_rows
|
......
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
|
ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
|
e: 1062
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
|
ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
|
e: 1062
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
|
ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
|
e: 1062
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
|
ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
|
e: 1062
|
Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989
|
Apr 15 20:16:41 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:16:41 139793670989568 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '279006-2019-04-15 20:37:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
|
de: 1062
|
Apr 15 20:16:41 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:16:41 139793670989568 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123048539
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
|
de: 1062
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
|
de: 1062
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
|
de: 1062
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
|
de: 1062
|
Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error c
|
ode: 1062
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error c
|
ode: 1062
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error c
|
ode: 1062
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
|
try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error code: 1062
|
Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844
|
Apr 23 13:04:24 nodesdb1.talk2m.com mysqld[192756]: 2019-04-23 13:04:24 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '759823-2019-04-23 11:03:50' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 2231, Internal MariaDB error code: 1062
|
Apr 23 13:04:24 nodesdb1.talk2m.com mysqld[192756]: 2019-04-23 13:04:24 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 147848347
|
Apr 25 02:36:07 nodesdb1.talk2m.com mysqld[192756]: 2019-04-25 2:36:07 140163747960576 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '1101784-2019-04-25 02:34:25' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3502, Internal MariaDB error code: 1062
|
Apr 25 02:36:07 nodesdb1.talk2m.com mysqld[192756]: 2019-04-25 2:36:07 140163747960576 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 152665180
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
|
May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02 8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335
|
May 04 09:57:27 nodesdb1.talk2m.com mysqld[412786]: 2019-05-04 9:57:27 139606033254144 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '999712-2019-05-04 09:51:45' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 72003, Internal MariaDB error code: 1062
|
May 04 09:57:27 nodesdb1.talk2m.com mysqld[412786]: 2019-05-04 9:57:27 139606033254144 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 183403214
|
May 05 13:44:55 nodesdb1.talk2m.com mysqld[412786]: 2019-05-05 13:44:55 139533897619200 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '355512-2019-05-05 06:56:08' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
|
May 05 13:44:55 nodesdb1.talk2m.com mysqld[412786]: 2019-05-05 13:44:55 139533897619200 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 187523527
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
|
May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12 7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
|
Attached you will find a full log of donor and joiner at time of error.
The failing node show no other message prior to crash, the previous message in logs was days ago.