[MDEV-27385] Adding constraint sometimes (non-repeatably) checks historical rows Created: 2021-12-30 Updated: 2024-01-24 Resolved: 2024-01-23 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Versioned Tables |
| Affects Version/s: | 10.6.4 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Major |
| Reporter: | Trevor Gross | Assignee: | Unassigned |
| Resolution: | Not a Bug | Votes: | 0 |
| Labels: | UNIQUE, innodb, versioned-table | ||
| Environment: |
10.6.4-MariaDB-1:10.6.4+maria~focal-log |
||
| Description |
|
There is an error that I am running across but am unable to repeat on a simple schema. If anyone can point me in the right direction to collect more details, I would be happy to look. The problem arises when attempting to add a unique key on two columns with the following:
This results in a duplicate entry error: In an effort to find the problem, I searched for everything that matched the problematic key:
Results of the command:
Note that none of these rows are active, so there should be no violation. This leads me to think that adding the unique constraint is running a check on historical rows, which it shouldn't. Setting `check_constraint_checks` to 0 also has no effect, and adding the key still fails. As mentioned, I would be happy to dig deeper to try to find the cause since I can't seem to replicate this issue on table from scratch. |
| Comments |
| Comment by Marko Mäkelä [ 2024-01-23 ] | ||||||||||||||||||
|
This does not look like a problem of the InnoDB storage engine, but rather a problem with the implementation of versioned tables in MariaDB. I would expect the ROW_START column to be appended to the PRIMARY KEY (I will assume it is id) and therefore to each index. So, we would have PRIMARY KEY(id,ROW_START). If the SQL layer (including the versioning related changes) told InnoDB that the secondary index definition is UNIQUE KEY manufacturer_pn_UN(manufacturer,manufacturer_pn), then the internal representation of the UNIQUE KEY manufacturer_pn_UN in InnoDB should be (manufacturer, manufacturer_pn, id, ROW_START). Because the dict_index_t::n_uniq would be 2, InnoDB would flag a uniqueness violations. InnoDB only tolerates ‘duplicates’ for NULL values, and in that case the internal uniqueness will be guaranteed by the PRIMARY KEY values. Example:
Here, in the index tree of UNIQUE b(b,a) we would have the values (NULL,1),(NULL,2). I don’t know if this really is a code bug or a user error. A possible work-around could be to explicitly include the ROW_START column in the index:
In that way, the internal representation would be (manufacturer,manufacturer_pn,ROW_START,id), and the uniqueness violation should disappear. tgross35, can you try this? I did not try it myself, because your SQL example is not self-contained. | ||||||||||||||||||
| Comment by Sergei Golubchik [ 2024-01-23 ] | ||||||||||||||||||
|
ROW_START shouldn't be part of a PRIMARY KEY, why would it? It'd mean you can do
but ROW_END is, indeed, implicitly appended to unique keys. See first two rows, at (or just before) 2021-12-16 00:13:34 you had two rows with the value Yageo-RC0805FR-071ML. As documentation says, table metadata is not versioned, after ALTER TABLE the new table looks as if it had new structure from the very beginning, only the data is versioned. And because you had conflicting rows earlier, you cannot add a UNIQUE constraint that was violated somewhere in the table's past. | ||||||||||||||||||
| Comment by Trevor Gross [ 2024-01-24 ] | ||||||||||||||||||
|
I have since upgraded this server to 11.1, but the problem still seems to exist. @marko that query claims ROW_START does not exist, probably because it is anonymous. But I think Sergei found the issue, I didn't even realize that two of those rows must have gotten inserted in the same transaction. That makes sense, this is easy enough to repeat knowing that:
It threw me off that the time isn't included in the error message, I couldn't understand why adding other keys with duplicate history worked but this specific one failed. Thanks all for the help. Wish that `DELETE HISTORY` took a where clause | ||||||||||||||||||
| Comment by Marko Mäkelä [ 2024-01-24 ] | ||||||||||||||||||
|
I stand corrected on how the PRIMARY KEY is being padded with version-related metadata. I think that for InnoDB and MyRocks, ROW_START would have been a better choice than ROW_END. In heap-organized storage engines it might matter less. A DELETE or UPDATE operation would have to update ROW_END of the original record. In InnoDB, updating a PRIMARY KEY column will require delete-marking the record in all indexes, inserting a new record in all indexes, and eventually, purging the delete-marked record. In MyRocks, I understand that this operation would involve inserting a tombstone record for the record that may have carried an infinite ROW_END, followed by inserting a new record with the ROW_END set to the current time. As far as I can tell, ROW_START is ‘immutable’ after the record has been created, and therefore it would have been a better choice in this regard. That is, a DELETE (or the DELETE part of a versioned UPDATE) would only update a non-indexed column in the existing record. |