[MDEV-25477] Auto-create breaks replication when triggering event was not replicated Created: 2021-04-21  Updated: 2022-05-06  Resolved: 2022-04-23

Status: Closed
Project: MariaDB Server
Component/s: Documentation, Partitioning, Replication, Versioned Tables
Affects Version/s: None
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: Elena Stepanova Assignee: Aleksey Midenkov
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by MDEV-25478 Auto-create: Auto partitioning is uns... Closed
Problem/Incident
is caused by MDEV-17554 Auto-create history partitions for sy... Closed
Relates

 Description   

Partitioning auto-creation cannot be enabled in a replication topology, unless only non-transactional engines are used (which is hardly realistic these days).

The reason is that when a partition is auto-created within a transaction, but the transaction is later rolled back, there will be nothing written in the binary log. Thus, the master will have the new partition, but the slave will not. This discrepancy will almost inevitably cause a replication abort some time later.

If it's intentional, at the very least it has to be documented, as it is a pretty big limitation.

Example:

--source include/have_partition.inc
--source include/have_innodb.inc
--source include/master-slave.inc
 
create table t (a int) engine=InnoDB with system versioning
  partition by system_time interval 1 hour auto;
 
insert into t values (1),(2);
set @@timestamp= @@timestamp+3601;
 
start transaction;
delete from t;
rollback;
 
show create table t;
 
--sync_slave_with_master
 
show create table t;
 
--connection master
alter table t drop partition p0;
 
--sync_slave_with_master

After rollback, the master has 2 historical partitions, while the slave has only one:

bb-10.6-midenok-MDEV-17554 edbc73ac4

show create table t;
Table	Create Table
t	CREATE TABLE `t` (
  `a` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 WITH SYSTEM VERSIONING
 PARTITION BY SYSTEM_TIME INTERVAL 1 HOUR STARTS TIMESTAMP'2021-04-21 23:00:00' AUTO
PARTITIONS 3
connection slave;
show create table t;
Table	Create Table
t	CREATE TABLE `t` (
  `a` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 WITH SYSTEM VERSIONING
 PARTITION BY SYSTEM_TIME INTERVAL 1 HOUR STARTS TIMESTAMP'2021-04-21 23:00:00' AUTO
PARTITIONS 2

So, the master is allowed to drop partition p0, but the slave is not:

Last_Errno	4128
Last_Error	Error 'Wrong partitions for `t`: must have at least one HISTORY and exactly one last CURRENT' on query. Default database: 'test'. Query: 'alter table t drop partition p0'

Also test case from MDEV-25478 must be considered.



 Comments   
Comment by Aleksey Midenkov [ 2021-04-22 ]

This doesn't matter as auto-creation also works on slave. Discrepancy in names of history partitions is acceptable but this should be documented somewhere. I don't believe this is a big limitation as neither current data nor history get lost or made inconsistent across the nodes.

Comment by Elena Stepanova [ 2021-04-22 ]

You can see from the description above that auto-creation does not work on slave – it can't work on slave because there is no DML.

Discrepancy in names and in the number of partitions is not acceptable for replication, because it will always break replication at the end. Either auto-creation works with replication, and then the table structure consistency must be maintained, or it doesn't work with replication.

And yes, incompatibility with replication is a very big limitation.

Comment by Aleksey Midenkov [ 2021-04-22 ]

We should resurrect MDEV-25347 fix as it logs ALTER TABLE ADD PARTITION on auto-create. serg what do you think?

Comment by Sergei Golubchik [ 2021-12-29 ]

This seems to be rather trivial to fix? On auto-create you can set "non transactional tables were modified" flag, and then the transaction will be written into binlog all right, with ROLLBACK at the end. Or you set a flag OPTION_KEEP_LOG. See trans_cannot_safely_rollback() — you need to trick it to return true.

Comment by Ralf Gebhardt [ 2022-01-26 ]

MDEV-17554 was moved to 10.9, so I am changing fixVersion for this bug to 10.9. I am also removing the 10.8V1 support case ID label

Comment by Aleksey Midenkov [ 2022-02-08 ]

AFAICS this bug blocks MDEV-17554.
Please review bb-10.8-MDEV-17554-auto-create-partition

Comment by Aleksey Midenkov [ 2022-04-23 ]

Pushed to bb-10.9-MDEV-17554-auto-create-partition

Generated at Thu Feb 08 09:37:59 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.