[MDEV-29674] History modification inserts records into a wrong partition without warning Created: 2022-09-30  Updated: 2022-10-21

Status: Stalled
Project: MariaDB Server
Component/s: Partitioning, Versioned Tables
Affects Version/s: 10.11.1
Fix Version/s: 10.11

Type: Bug Priority: Major
Reporter: Elena Stepanova Assignee: Sergei Golubchik
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Problem/Incident
is caused by MDEV-16546 System versioning setting to allow hi... Closed
Relates

 Description   

If a record is inserted into a versioned partitioned table with row end outside existing history partitions, no warning is produced, and the record is inserted into the last history partition. If the table is auto-partitioned, no partition is created.

When the same operation is performed via timestamp modification, a warning is thrown, or auto-partitions are created.

--source include/have_partition.inc
 
create table t (a int) with system versioning
partition by system_time interval 1 day
starts timestamp'2022-09-30 00:00:00' auto partitions 3;
 
set system_versioning_insert_history=on;
insert into t (a,row_start,row_end) values (1,'2022-09-30','2022-10-10');
 
show create table t;
select * from t partition (p1);
 
# Cleanup
drop table t;

bb-10.11-MDEV-16546 2b1d324294

insert into t (a,row_start,row_end) values (1,'2022-09-30','2022-10-10');
show create table t;
Table	Create Table
t	CREATE TABLE `t` (
  `a` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1 WITH SYSTEM VERSIONING
 PARTITION BY SYSTEM_TIME INTERVAL 1 DAY STARTS TIMESTAMP'2022-09-30 00:00:00' AUTO
PARTITIONS 3
select * from t partition (p1);
a
1

With timestamp modification instead of insert_history:

--source include/have_partition.inc
 
create table t (a int) with system versioning
partition by system_time interval 1 day
starts timestamp'2022-09-30 00:00:00' auto partitions 3;
 
set @@timestamp= unix_timestamp('2022-09-30');
insert into t values (1);
set @@timestamp= unix_timestamp('2022-10-10');
delete from t;
 
show create table t;
select * from t partition (p1);
select * from t partition (p10);
 
# Cleanup
drop table t;

show create table t;
Table	Create Table
t	CREATE TABLE `t` (
  `a` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1 WITH SYSTEM VERSIONING
 PARTITION BY SYSTEM_TIME INTERVAL 1 DAY STARTS TIMESTAMP'2022-09-30 00:00:00' AUTO
PARTITIONS 12
select * from t partition (p1);
a
select * from t partition (p10);
a
1

For "doesn't produce/produces a warning" part, the test cases can be adjusted by removing AUTO clause from the table description.



 Comments   
Comment by Sergei Golubchik [ 2022-10-03 ]

Unfortunately, this is where the logic "exactly the same as setting the timestamp" breaks. We cannot create new partitions (DDL) in the middle of an INSERT (DML). Note that a multi-row insert might need to create new partitions for every row,

The row must go into the correct partition, if it exists, that's clear.
But when it doesn't — should it be a warning (as it there was no AUTO at all), an error, or nothing?

The main use case — mysqldump — creates tables with the correct number of partitions, so it won't be affected by that.

Comment by Elena Stepanova [ 2022-10-09 ]

FWIW, mysqldump isn't great at it either. Yes, it will create the correct number of partitions, but for partitioning by LIMIT (and if mysqldump runs with default extended insert), it will distribute the records between the partitions with rather little regard to the actual LIMIT condition. It will put into each partition at least as many records as a single INSERT contains (based on net-buffer-length), which can be a lot more than LIMIT, depending on circumstances; or, if LIMIT is greater than INSERT number, up to <LIMIT + INSERT> records.

And since historical records in the dump aren't sorted by ROW END, it will put them into partitions pretty much randomly comparing to the order they were originally added.

But this is of course to be expected.

Comment by Elena Stepanova [ 2022-10-16 ]

Another consequence:

--source include/have_partition.inc
 
CREATE TABLE t (a INT) WITH SYSTEM VERSIONING PARTITION BY system_time INTERVAL 1 YEAR (PARTITION p1 HISTORY, PARTITION ver_pn CURRENT);
SET SYSTEM_VERSIONING_INSERT_HISTORY= ON;
INSERT INTO t (a, row_start, row_end) VALUES (NULL, '2000-01-01', '2036-01-01');
ALTER TABLE t ADD PARTITION (PARTITION p2 HISTORY);
CHECK TABLE t;
 
# Cleanup
DROP TABLE t;

bb-10.11-MDEV-16546 32090722c

CHECK TABLE t;
Table	Op	Msg_type	Msg_text
test.t	check	error	Found a misplaced row
test.t	check	error	Partition p1 returned error
test.t	check	error	Upgrade required. Please do "REPAIR TABLE `t`" or dump/reload to fix it!

Comment by Elena Stepanova [ 2022-10-20 ]

or, if LIMIT is greater than INSERT number, up to <LIMIT + INSERT> records.

I was too optimistic in this part. In fact, all inserted historical records are put into the first LIMIT partition, regardless its size and the amount/size of inserts. For mysqldump it means that even if it's run with --skip-extended-insert, it still puts all records into the first partition.

Even if partition to insert into is specified explicitly, it still refuses to use any other than the first partition:

MariaDB [db3]> create or replace table t (a int) with system versioning partition by system_time limit 10 partitions 3;
Query OK, 0 rows affected (0.182 sec)
 
MariaDB [db3]> insert into t partition (p1) (a,row_start,row_end) select seq,'2022-02-02','2023-03-03' from seq_1_to_20;
ERROR 1748 (HY000): Found a row not matching the given partition set
MariaDB [db3]> insert into t partition (p0) (a,row_start,row_end) select seq,'2022-02-02','2023-03-03' from seq_1_to_20;
Query OK, 20 rows affected (0.026 sec)
Records: 20  Duplicates: 0  Warnings: 0
 
MariaDB [db3]> insert into t partition (p1) (a,row_start,row_end) select seq,'2022-02-02','2023-03-03' from seq_1_to_20;
ERROR 1748 (HY000): Found a row not matching the given partition set

Generated at Thu Feb 08 10:10:25 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.