[MDEV-22247] History partition overflow leads to wrong SELECT result Created: 2020-04-15  Updated: 2022-07-18  Resolved: 2021-07-06

Status: Closed
Project: MariaDB Server
Component/s: Partitioning, Versioned Tables
Affects Version/s: 10.3, 10.4, 10.5
Fix Version/s: 10.3.31, 10.4.21, 10.5.12, 10.6.4

Type: Bug Priority: Major
Reporter: Aleksey Midenkov Assignee: Aleksey Midenkov
Resolution: Fixed Votes: 0
Labels: partition-pruning

Issue Links:
Relates
relates to MDEV-17554 Auto-create history partitions for sy... Closed
relates to MDEV-18501 Partition pruning doesn't work for hi... Closed
relates to MDEV-25529 Auto-create: Pre-existing historical ... Stalled
relates to MDEV-29114 Pruning depends on current timestamp ... Stalled

 Description   

Reproduce

-- source include/have_partition.inc
 
set timestamp= unix_timestamp('2000-01-01 00:00:00');
create or replace table t1 (x int) with system versioning
partition by system_time interval 1 hour;
 
insert into t1 values (0);
update t1 set x= x + 1;
 
set timestamp= unix_timestamp('2000-01-01 01:00:01');
update t1 set x= x + 1;
 
select *, row_start, row_end from t1 for system_time as of '2000-01-01 01:00:00';
--replace_column 10 #
explain partitions select * from t1 for system_time as of '2000-01-01 01:00:00';
 
drop tables t1;

Result

set timestamp= unix_timestamp('2000-01-01 00:00:00');
create or replace table t1 (x int) with system versioning
partition by system_time interval 1 hour;
insert into t1 values (0);
update t1 set x= x + 1;
set timestamp= unix_timestamp('2000-01-01 01:00:01');
update t1 set x= x + 1;
Warnings:
Warning 4114    Versioned table `test`.`t1`: last HISTORY partition (`p0`) is out of INTERVAL, need more HISTORY partitions
select *, row_start, row_end from t1 for system_time as of '2000-01-01 01:00:00';
x       row_start       row_end
explain partitions select * from t1 for system_time as of '2000-01-01 01:00:00';
id      select_type     table   partitions      type    possible_keys   key     key_len ref     rows    Extra
1       SIMPLE  NULL    NULL    NULL    NULL    NULL    NULL    NULL    #       Impossible WHERE noticed after reading const tables
drop tables t1;

Expected

SELECT displays "1" record.
Compare with result from 3 partitions:

set timestamp= unix_timestamp('2000-01-01 00:00:00');
create or replace table t1 (x int) with system versioning
partition by system_time interval 1 hour partitions 3;
insert into t1 values (0);
update t1 set x= x + 1;
set timestamp= unix_timestamp('2000-01-01 01:00:01');
update t1 set x= x + 1;
select *, row_start, row_end from t1 for system_time as of '2000-01-01 01:00:00';
x       row_start       row_end
1       2000-01-01 00:00:00.000000      2000-01-01 01:00:01.000000
explain partitions select * from t1 for system_time as of '2000-01-01 01:00:00';
id      select_type     table   partitions      type    possible_keys   key     key_len ref     rows    Extra
1       SIMPLE  t1      p1,pn   ALL     NULL    NULL    NULL    NULL    #       Using where
drop tables t1;

Proposed fix

1. When overflow warning occurs touch file t1.no_pruning.
2. When share open sees that file it sets no_pruning property to TABLE_SHARE.
3. Pruning sees that property, selects all partitions and issues warning that pruning is off.
4. After partitioning rebuild this file is removed.



 Comments   
Comment by Sergei Golubchik [ 2020-04-30 ]

What about this approach: for the purpose of pruning always assume that the last history partition has no end time. That is, if the requested timestamp is after the last history partition — don't prune it anyway. All other partitions can be pruned normally.

Comment by Nikita Malyavin [ 2021-06-04 ]

ok to push

Generated at Thu Feb 08 09:13:18 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.