Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-19448

mariadb node replication crash on HA_ERR_FOUND_DUPP_KEY

    XMLWordPrintable

Details

    Description

      since upgrade of our galera servers to 10.2.x, we experience global crash of nodes due to database inconsistency propagated through galera. This happens about 3 times a month and put our cluster in a very unstable situation as it requires a full SST that is time and IO consuming.

      It always has the same scenario:
      a client on one node (here nodesdb2) is inserting data that triggers unique key constraint failure (duplicate), leading to client rollbacking it's operations.
      However this insert is propagated to other node (here nodesdb1) as if it was approved, and the secondary node crashes due to inconsistency of content.

      After this, mariadb stops on the affected node and, on restart, triggers a full SST.

      Note that we currently do all insert / select / delete operation on a single node (nodesdb2) and the other nodes (nodesdb1, the crashing node) is currently used only for redundancy.

      May 11 04:40:39 nodesdb1.myhost.com mysqld[412786]: 2019-05-11  4:40:39 139660305315584 [Warning] WSREP: Failed to report last committed 205740288, -110 (Connection timed out)
      May 11 04:40:41 nodesdb1.myhost.com mysqld[412786]: 2019-05-11  4:40:41 139660305315584 [Warning] WSREP: Failed to report last committed 205740332, -110 (Connection timed out)
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error co
      de: 1062
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: Failed to apply app buffer: seqno: 209185646, status: 1
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: at galera/src/trx_handle.cpp:apply():353
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: Retrying 2th time
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: Failed to apply app buffer: seqno: 209185646, status: 1
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: at galera/src/trx_handle.cpp:apply():353
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: Retrying 3th time
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: Failed to apply app buffer: seqno: 209185646, status: 1
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: at galera/src/trx_handle.cpp:apply():353
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: Retrying 4th time
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] WSREP: Failed to apply trx: source: e5c9c2b5-400f-11e9-ba3d-8f8054a48006 version: 4 local: 0 state: APPLYING flags: 1 conn_id: 10193487 trx_id: 20898181434 seqnos (l: 34565981, g: 209185646, s: 209185645, d: 209185626, ts: 14759991015976877)
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] WSREP: Failed to apply trx 209185646 4 times
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] WSREP: Node consistency compromised, aborting...
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Note] WSREP: Closing send monitor...
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Note] WSREP: Closed send monitor.
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Note] WSREP: gcomm: terminating thread
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Note] WSREP: gcomm: joining thread
      May 12 07:32:42 nodesdb1.myhost.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Note] WSREP: gcomm: closing backend
      

      This always happen on the same table "tags_history" which is our biggest table: about 1 445 158 186 rows, 500G storage size and has an estimated 1200 rows inserted and deleted per seconds .

      journalctl -u mariadb | grep Write_rows 
      ......
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
      ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
      e: 1062                                                                                                                                                                                        
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989                                 
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
      ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
      e: 1062                                                                                                                                                                                        
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989                                 
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
      ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
      e: 1062                                                                                                                                                                                        
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989                                 
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate ent
      ry '276267-2019-04-04 20:46:58' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3615, Internal MariaDB error cod
      e: 1062                                                                                                                                                                                        
      Apr 04 20:35:36 nodesdb1.talk2m.com mysqld[45259]: 2019-04-04 20:35:36 140556698482432 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 88177989                                 
      Apr 15 20:16:41 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:16:41 139793670989568 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '279006-2019-04-15 20:37:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
      de: 1062                                                                                                                                                                                       
      Apr 15 20:16:41 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:16:41 139793670989568 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123048539                               
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
      de: 1062                                                                                                                                                                                       
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718                               
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
      de: 1062                                                                                                                                                                                       
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718                               
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
      de: 1062                                                                                                                                                                                       
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718                               
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '279006-2019-04-15 20:46:26' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3579, Internal MariaDB error co
      de: 1062                                                                                                                                                                                       
      Apr 15 20:25:42 nodesdb1.talk2m.com mysqld[104042]: 2019-04-15 20:25:42 139793670682368 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 123067718                               
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error c
      ode: 1062                                                                                                                                                                                      
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844                               
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error c
      ode: 1062                                                                                                                                                                                      
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844                               
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error c
      ode: 1062                                                                                                                                                                                      
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844                               
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate en
      try '1058484-2019-04-19 11:09:09' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 6349, Internal MariaDB error code: 1062
      Apr 19 12:14:27 nodesdb1.talk2m.com mysqld[331647]: 2019-04-19 12:14:27 140351886702336 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 134929844                              
      Apr 23 13:04:24 nodesdb1.talk2m.com mysqld[192756]: 2019-04-23 13:04:24 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '759823-2019-04-23 11:03:50' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 2231, Internal MariaDB error code: 1062
      Apr 23 13:04:24 nodesdb1.talk2m.com mysqld[192756]: 2019-04-23 13:04:24 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 147848347                              
      Apr 25 02:36:07 nodesdb1.talk2m.com mysqld[192756]: 2019-04-25  2:36:07 140163747960576 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '1101784-2019-04-25 02:34:25' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 3502, Internal MariaDB error code: 1062
      Apr 25 02:36:07 nodesdb1.talk2m.com mysqld[192756]: 2019-04-25  2:36:07 140163747960576 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 152665180                              
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335                              
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335                              
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335                              
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '620709-2019-05-02 07:34:01' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
      May 02 08:04:00 nodesdb1.talk2m.com mysqld[192756]: 2019-05-02  8:04:00 140163747346176 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 176310335                              
      May 04 09:57:27 nodesdb1.talk2m.com mysqld[412786]: 2019-05-04  9:57:27 139606033254144 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '999712-2019-05-04 09:51:45' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 72003, Internal MariaDB error code: 1062
      May 04 09:57:27 nodesdb1.talk2m.com mysqld[412786]: 2019-05-04  9:57:27 139606033254144 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 183403214                              
      May 05 13:44:55 nodesdb1.talk2m.com mysqld[412786]: 2019-05-05 13:44:55 139533897619200 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '355512-2019-05-05 06:56:08' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 1007, Internal MariaDB error code: 1062
      May 05 13:44:55 nodesdb1.talk2m.com mysqld[412786]: 2019-05-05 13:44:55 139533897619200 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 187523527                              
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646                              
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646                              
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646                              
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table MyHostData.tags_history; Duplicate entry '276267-2019-05-12 07:38:56' for key 'tagshistory_unique', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log FIRST, end_log_pos 9801, Internal MariaDB error code: 1062
      May 12 07:32:42 nodesdb1.talk2m.com mysqld[412786]: 2019-05-12  7:32:42 139606033868544 [Warning] WSREP: RBR event 6 Write_rows_v1 apply warning: 121, 209185646 
      

      Attached you will find a full log of donor and joiner at time of error.

      The failing node show no other message prior to crash, the previous message in logs was days ago.

      Attachments

        Activity

          People

            jplindst Jan Lindström (Inactive)
            dade David Delbecq
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Git Integration

                Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.