[MDEV-33350] Bulk InnoDB LOAD DATA breaks replication upon hitting duplicate key Created: 2024-02-01  Updated: 2024-02-03

Status: Confirmed
Project: MariaDB Server
Component/s: Replication, Storage Engine - InnoDB
Affects Version/s: 10.7, 10.11, 11.0, 11.1, 11.2, 11.3, 11.4
Fix Version/s: 10.11, 11.0, 11.1, 11.2, 11.3

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Andrei Elkin
Resolution: Unresolved Votes: 0
Labels: affects-tests

Issue Links:
Relates
relates to MDEV-24622 Replication does not support bulk ins... Open
relates to MDEV-33179 Duplicate values upon InnoDB bulk ins... Open

 Description   

The issue looks logically related to MDEV-33179, in the sense that here LOAD DATA is also silently rolled back on the master; but it is written in the binary log, attempted on the slave and fails there.
However, since there is no clarity how (and whether) MDEV-33179 is going to be fixed, it's not guaranteed that the fix will help with the replication problem, so I'm filing it separately. If after MDEV-33179 is fixed the fix makes the failure described here disappear, it should be closed as a duplicate.

--source include/have_innodb.inc
--source include/have_binlog_format_row.inc
--source include/master-slave.inc
 
--write_file $MYSQL_TMP_DIR/1.txt
1
2
1
EOF
 
CREATE TABLE t (pk INT PRIMARY KEY) ENGINE=InnoDB;
SET FOREIGN_KEY_CHECKS=0, UNIQUE_CHECKS=0;
eval LOAD DATA INFILE '$MYSQL_TMP_DIR/1.txt' INTO TABLE t;
SET FOREIGN_KEY_CHECKS=1, UNIQUE_CHECKS=1;
SELECT * FROM t;
 
--sync_slave_with_master
 
# Cleanup
--connection master
DROP TABLE t;
--source include/rpl_end.inc

10.11 d0c80c211c1fe3370b68be540bb9113028c6746f

2024-02-01 11:54:35 7 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table test.t; Duplicate entry '1' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log master-bin.000001, end_log_pos 731, Gtid 0-1-2, Internal MariaDB error code: 1062
2024-02-01 11:54:35 7 [Warning] Slave: Duplicate entry '1' for key 'PRIMARY' Error_code: 1062



 Comments   
Comment by Roel Van de Paar [ 2024-02-02 ]

I also ran into this, in another way:

# Requires standard m/s setup
SET sql_mode='',unique_checks=0,foreign_key_checks=0,autocommit=0;
CREATE TABLE t1 (c1 INT UNIQUE KEY) ENGINE=InnoDB;
CREATE TABLE t2 (c1 BINARY (0),c2 INT UNIQUE KEY) ENGINE=InnoDB;
INSERT INTO t2 VALUES (0,0);
INSERT INTO t1 VALUES (0,0);
CREATE TEMPORARY TABLE t1 (c INT);
INSERT INTO t2 VALUES (0,0);

Leads to:

11.4.0 8bf9f21855606ddeb0fe90fb1e70165adbec6746 (Debug)

2024-02-03  9:48:58 6 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table test.t2; Duplicate entry '0' for key 'c2', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log binlog.000001, end_log_pos 0, Gtid 0-1-6, Internal MariaDB error code: 1062
2024-02-03  9:48:58 6 [Warning] Slave: Duplicate entry '0' for key 'c2' Error_code: 1062
2024-02-03  9:48:58 6 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'binlog.000001' position 1307; GTID position '0-1-5'

11.4.0 8bf9f21855606ddeb0fe90fb1e70165adbec6746 (Optimized)

2024-02-03  9:47:55 6 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table test.t2; Duplicate entry '0' for key 'c2', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log binlog.000001, end_log_pos 0, Gtid 0-1-6, Internal MariaDB error code: 1062
2024-02-03  9:47:55 6 [Warning] Slave: Duplicate entry '0' for key 'c2' Error_code: 1062
2024-02-03  9:47:55 6 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'binlog.000001' position 1307; GTID position '0-1-5'

Comment by Roel Van de Paar [ 2024-02-03 ]

Yet another way (XA):

# Requires standard m/s setup
CREATE TABLE ti (a INT,b INT UNSIGNED,c CHAR,d BINARY,e BINARY,f CHAR,g BLOB,h BLOB,id INT,KEY(b),KEY(e),PRIMARY KEY(id));
XA START 'a';
INSERT INTO ti VALUES (+1,+1,0,0,0,0,0,0,3);
SET foreign_key_checks=0,unique_checks=0;
CREATE TEMPORARY TABLE t2 (a INT,b INT,PRIMARY KEY(a));
INSERT INTO t2 VALUES (0,6),(0,7);
INSERT INTO ti VALUES (+1,1,0,0,0,0,0,0,3);

Leads to:

11.4.0 8bf9f21855606ddeb0fe90fb1e70165adbec6746 (Debug)

2024-02-03 14:46:17 6 [ERROR] Slave SQL: Could not execute Write_rows_v1 event on table test.ti; Duplicate entry '3' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log binlog.000001, end_log_pos 0, Gtid 0-1-5, Internal MariaDB error code: 1062
2024-02-03 14:46:17 6 [Warning] Slave: Duplicate entry '3' for key 'PRIMARY' Error_code: 1062
2024-02-03 14:46:17 6 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'binlog.000001' position 1198; GTID position '0-1-4'

Comment by Roel Van de Paar [ 2024-02-03 ]

This can also be seen as a warning when using --slave_skip_errors=ALL:

11.4.0 8bf9f21855606ddeb0fe90fb1e70165adbec6746 (Debug)

2024-01-02 20:10:44 12 [Warning] Slave SQL: Could not execute Write_rows_v1 event on table test.ti; Duplicate entry '12' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log binlog.000003, end_log_pos 10498, Gtid 0-1-162, Internal MariaDB error code: 1062

Generated at Thu Feb 08 10:38:15 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.