[MDEV-22935] Erroneous Aria Index / Optimizer behaviour Created: 2020-06-18  Updated: 2023-04-27

Status: Confirmed
Project: MariaDB Server
Component/s: Optimizer, Storage Engine - Aria
Affects Version/s: 10.1, 10.1.44, 10.3.22, 10.4.13, 10.2, 10.3, 10.4, 10.5
Fix Version/s: 10.4, 10.5

Type: Bug Priority: Major
Reporter: Frank Maas Assignee: Michael Widenius
Resolution: Unresolved Votes: 0
Labels: aria, index, optimizer
Environment:

Linux CentOS several versions


Attachments: File indextest.pl     File rt_sum_fm23.sql.gz    

 Description   

We stumbled upon a strange problem which seems to be related to either an index or an optimizer error which is repeatable but not straightforward to reproduce. For purpose of this bugreport we were able to bring the case down to a simple table:

CREATE TABLE `rt_sum_fm23` (
`route` varchar(20) NOT NULL DEFAULT '',
`timestamp` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`route`,`timestamp`)
) ENGINE=Aria DEFAULT CHARSET=latin1 PAGE_CHECKSUM=1 TRANSACTIONAL=1

which contains a little short of 1.900.000 records. This is relevant as the experienced behaviour seems related to having a certain number of records. This table contains records with 364 distinct entries for 'route' and per route a number of timestamped records. Now see this:

> select route, min(timestamp), max(timestamp) from rt_sum_fm23 where route = 'N201__N205_ZAV';

route min(timestamp) max(timestamp)
N201__N205_ZAV 2020-06-07 17:59:00 2020-06-12 23:59:00

So there is (at least) one record with a timestamp on June 7th, which is supported by the following:

> select * from rt_sum_fm23 where route = 'N201__N205_ZAV' order by timestamp asc limit 5;

route timestamp
N201__N205_ZAV 2020-06-07 17:59:00
N201__N205_ZAV 2020-06-07 18:00:00
N201__N205_ZAV 2020-06-07 18:01:00
N201__N205_ZAV 2020-06-07 18:02:00
N201__N205_ZAV 2020-06-07 18:03:00

> explain select * from rt_sum_fm23 where route = 'N201__N205_ZAV' order by timestamp asc limit 5;

id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE rt_sum_fm23 ref PRIMARY PRIMARY 22 const 7290 Using where; Using index

However - and this is the strange behaviour - this is what happens if we add a condition to that where:

> select * from rt_sum_fm23 where route = 'N201__N205_ZAV' and timestamp < 20200608000000 order by timestamp asc limit 5;
Empty set (0.00 sec)

> explain select * from rt_sum_fm23 where route = 'N201__N205_ZAV' and timestamp < 20200608000000 order by timestamp asc limit 5;

id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE NULL NULL NULL NULL NULL NULL NULL Impossible WHERE noticed after reading const tables

Repairing the table results in either the problem remaining the same,, or it shifts (other 'routes' experience the same issue). We have made an export (dump to SQL) and imported that on two different hosts what resulted in the same experience (even the same route). The SQL dump is attached for your benefit.

We have done numerous tests and the behavior is repeatable.

edit: removed some incorrect markup from the text, only editorial changes



 Comments   
Comment by Frank Maas [ 2020-06-24 ]

Unfortunately this bug remains unassigned... Yet not retrieving results that are in a database is quite an issue...

Comment by Willem van Pelt [ 2020-06-29 ]

With the attached SQL file, I have been able to reproduce the behaviour on:
Fedora 31 / MariaDB 10.3.22
Fedora 32 / MariaDB 10.4.13
Ubuntu 20.4 LTS / MariaDB 10.5.4

The problem seems to be related to the Aria engine; I haven't been able to reproduce it on MyISAM or InnoDB. Further, it seems to only occur when using a primary or unique key.

Attached is a simple Perl script to check for faulty results.
indextest.pl

Comment by Elena Stepanova [ 2020-08-04 ]

Thanks for the report and test case, reproducible as described.

REPAIR shows odd output:

+------------------+--------+----------+----------------------------------------------+
| Table            | Op     | Msg_type | Msg_text                                     |
+------------------+--------+----------+----------------------------------------------+
| test.rt_sum_fm23 | repair | warning  | Number of rows changed from 25664 to 1881297 |
| test.rt_sum_fm23 | repair | status   | OK                                           |
+------------------+--------+----------+----------------------------------------------+

and the error log says

2020-08-04 18:59:20 140367821108992 [Note] Found 1881297 of 25664 rows when repairing './test/rt_sum_fm23'

If REPAIR doesn't help, you can also try ALTER TABLE rt_sum_fm23 FORCE for a workaround, maybe it will be more efficient.

Generated at Thu Feb 08 09:18:35 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.