[MDEV-29159] Patch for MDEV-28918 introduces more inconsistency than it solves, breaks usability Created: 2022-07-23 Updated: 2023-11-23 Resolved: 2022-08-06 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Data types |
| Affects Version/s: | 10.7, 10.8, 10.9, 10.10 |
| Fix Version/s: | 10.7.5, 10.8.4, 10.9.2, 10.10.0 |
| Type: | Bug | Priority: | Blocker |
| Reporter: | Elena Stepanova | Assignee: | Alexander Barkov |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||||||
| Description |
|
There has been a recent change in behavior in regard to data type conversion and incompatibility, introduced in I think the approach is problematic in many ways and strongly suggest to reconsider before it has been released. According to JIRA and the commit comment, what it was trying to solve sounds like relatively minor inconsistencies. It does solve them, but at the cost of introducing bigger ones, adding unnecessary limitations, and hurting usability. Let's take the test case from The complaint there was that under the same circumstances, UPDATE and ALTER produced identical warnings but the automatic cast ended up with different inet6 values, :: vs NULL. The achievement is that now they behave identically. LimitationsNow both UPDATE and ALTER ignore non-strict SQL_MODE and IGNORE modifier in ALTER and UPDATE completely, they don't change the behavior anymore, even though they should and still do for most data types.
Usability problem* (in addition to the limitation above, which is also a usability problem)The error/warning message has changed from a quite useful, containing the problematic value, table name, column name, (and row number, which may be not always correct, but it was at least trying) to an incomprehensible and useless generic complaint:
I personally couldn't understand what it meant until I removed all irrelevant values and columns, got to inserting only one wrong row in one column, and only then, judging by what I was doing, I could guess what the message was trying to say. I cannot imagine how users can understand the problem in a real setup with many rows and many columns. More inconsistencyNow conversion into INET6 and alike from INT and some other types behaves differently comparing to string types/literals, which is a much more visible inconsistency than the one that was being solved:
That is, it has no problem converting an integer passed as a string (or any string, for that matter), but has a problem converting the integer itself. Same with other types, e.g. temporal. DiscussionSo far, two arguments were briefly discussed on Slack:
This is true; however, geometry types have always been somewhat special and notoriously weird, their behavior in this sense appears to be a limitation, more or less adequately explained by the error message itself (which btw has also been damaged by
Well, if it really cannot, then it cannot. There is no reason to extend the legacy deviations onto new data types unless absolutely necessary. We know that INET6 or UUID have no problem casting to a valid value, they did before. Which leads to the 2nd mentioned point:
"Natural" is a subjective term. We use 0000-00-00 as a default value for dates, which is anything but natural. In any case, both INET6 and UUID have a "hard default" value, and still do even after the change, so it's not really an excuse:
And with INET6 it is even more naturally ::. Btw, it is a weak excuse even for geometry, as it's capable of inserting an empty string when it wants to, but it wants to so inconsistently that it would not be helping my case. |
| Comments |
| Comment by Alexander Barkov [ 2022-07-26 ] | |||||||||||||||||
|
The idea of the patch for The same thing happens in this scenario:
There is no conversion between INT and ROW. This statement aborts:
After | |||||||||||||||||
| Comment by Alexander Barkov [ 2022-07-26 ] | |||||||||||||||||
|
The GEOMETRY data type was also fixed to work the same way with ROW. This statement:
now aborts at the preparation time. There is no conversion from INT to GEOMETRY. The patch for Before the patch such a mixture of INT and GEOMETRY failed, on the first row. It could only succeed on an empty table. | |||||||||||||||||
| Comment by Elena Stepanova [ 2022-07-26 ] | |||||||||||||||||
|
The idea of this report, on the other hand, is that it's an arbitrary limitation which doesn't solve anything other than a special case, and brings a lot of new inconsistencies and inconveniences instead. To summarize the effect of the change on the users (even aside inconsistencies, as they are always subjective): When users care about the exact result of the conversion and prefer getting an error and dealing with it, they use strict modes and normal INSERT / ALTER operations. Thus they are already getting an error, even without the change. After the patch, they'll continue getting an error, only instead of the old informative one with column names, values etc., they'll be getting the non-informative generic "illegal parameter data types". When users don't care about the exact result and are willing to accept an implicit possibly lossy conversion in order to have the operation succeed, they use non-strict modes and/or IGNORE clauses. Now they can do so, get somewhat informative warnings and possibly act upon them post-factum. I failed to find a scenario where the change will actually be beneficial for the user. So, I am assigning it to serg as an architect. I have written most if what I think about this change, and am not going to fight it any further, but if you really want to have it in the future, please write up a standard official answer which we will be giving the users who will reasonably complain about it and require an explanation why we did it, which should preferably be something better than "the idea of the limitation is that it limits". | |||||||||||||||||
| Comment by Sergei Golubchik [ 2022-08-03 ] | |||||||||||||||||
|
in bb-10.7-bar- | |||||||||||||||||
| Comment by Sergei Golubchik [ 2022-08-05 ] | |||||||||||||||||
|
3ebbfd88a03 is ok to push | |||||||||||||||||
| Comment by Alexander Barkov [ 2022-08-06 ] | |||||||||||||||||
|
Pushed to 10.7. | |||||||||||||||||
| Comment by Alexander Barkov [ 2022-08-06 ] | |||||||||||||||||
|
Change summary: 1. Changing the error/warning test as follows: -ERROR HY000: Illegal parameter data types inet6 and int for operation 'SET' so in case of a big table it's easier to see which column has the problem. 2. Store assignment failures on incompatible data types now raise errors and abort the statement if:
e.g.:
Otherwise, only a warning is raised and the statement continues, then a warning is issued per row.
| |||||||||||||||||
| Comment by Elena Stepanova [ 2022-08-06 ] | |||||||||||||||||
|
For the record (as we will likely face questions about it sooner or later), in layman's terms, the new limitation introduced by E.g. this is prohibited:
– a column cannot be converted between incompatible data types even if the table is empty. And this is prohibited too:
– insert/select between incompatible types fails even if all values in question are NULLs and the target column is NULL-able. The patch above doesn't change this, but it ensures that IGNORE clause and non-strict SQL modes relax the limitation and make the operation possible when necessary (and improves the error message so it's easier to understand what is happening). | |||||||||||||||||
| Comment by Elena Stepanova [ 2022-08-06 ] | |||||||||||||||||
|
Also important to mention, as customary with any new restrictions in new versions, any affected operation will break replication from an old master to a new slave. User must be aware. |