[MDEV-29159] Patch for MDEV-28918 introduces more inconsistency than it solves, breaks usability Created: 2022-07-23  Updated: 2023-11-23  Resolved: 2022-08-06

Status: Closed
Project: MariaDB Server
Component/s: Data types
Affects Version/s: 10.7, 10.8, 10.9, 10.10
Fix Version/s: 10.7.5, 10.8.4, 10.9.2, 10.10.0

Type: Bug Priority: Blocker
Reporter: Elena Stepanova Assignee: Alexander Barkov
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Problem/Incident
causes MDEV-29356 Assertion `0' failed in Type_handler_... Closed
is caused by MDEV-28918 Implicit cast from INET6 UNSIGNED wor... Closed

 Description   

There has been a recent change in behavior in regard to data type conversion and incompatibility, introduced in MDEV-28918 (patch).

I think the approach is problematic in many ways and strongly suggest to reconsider before it has been released.

According to JIRA and the commit comment, what it was trying to solve sounds like relatively minor inconsistencies. It does solve them, but at the cost of introducing bigger ones, adding unnecessary limitations, and hurting usability.

Let's take the test case from MDEV-28918 description as a basis.

The complaint there was that under the same circumstances, UPDATE and ALTER produced identical warnings but the automatic cast ended up with different inet6 values, :: vs NULL.

The achievement is that now they behave identically.
However:

Limitations

Now both UPDATE and ALTER ignore non-strict SQL_MODE and IGNORE modifier in ALTER and UPDATE completely, they don't change the behavior anymore, even though they should and still do for most data types.

10.7.3, OLD behavior

MariaDB [test]> UPDATE t1 SET a=b;
ERROR 1292 (22007): Incorrect inet6 value: '97' for column `test`.`t1`.`a` at row 1
MariaDB [test]> UPDATE IGNORE t1 SET a=b;
Query OK, 0 rows affected, 1 warning (0.000 sec)
Rows matched: 1  Changed: 0  Warnings: 1
 
MariaDB [test]> show warnings;
+---------+------+-----------------------------------------------------------------+
| Level   | Code | Message                                                         |
+---------+------+-----------------------------------------------------------------+
| Warning | 1292 | Incorrect inet6 value: '97' for column `test`.`t1`.`a` at row 1 |
+---------+------+-----------------------------------------------------------------+
1 row in set (0.000 sec)

10.7 current, NEW behavior

MariaDB [test]> UPDATE t1 SET a=b;
ERROR 4078 (HY000): Illegal parameter data types inet6 and int unsigned for operation 'SET'
MariaDB [test]> UPDATE IGNORE t1 SET a=b;
ERROR 4078 (HY000): Illegal parameter data types inet6 and int unsigned for operation 'SET'

Usability problem* (in addition to the limitation above, which is also a usability problem)

The error/warning message has changed from a quite useful, containing the problematic value, table name, column name, (and row number, which may be not always correct, but it was at least trying) to an incomprehensible and useless generic complaint:

OLD

ERROR 1292 (22007): Incorrect inet6 value: '97' for column `test`.`t1`.`a` at row 1

ERROR 4078 (HY000): Illegal parameter data types inet6 and int unsigned for operation 'SET'

I personally couldn't understand what it meant until I removed all irrelevant values and columns, got to inserting only one wrong row in one column, and only then, judging by what I was doing, I could guess what the message was trying to say. I cannot imagine how users can understand the problem in a real setup with many rows and many columns.

More inconsistency

Now conversion into INET6 and alike from INT and some other types behaves differently comparing to string types/literals, which is a much more visible inconsistency than the one that was being solved:

NEW

MariaDB [test]> insert into t values (1);
ERROR 4078 (HY000): Illegal parameter data types inet6 and int for operation 'SET'
MariaDB [test]> insert into t values ('1');
ERROR 1292 (22007): Incorrect inet6 value: '1' for column `test`.`t`.`a` at row 1
MariaDB [test]> insert ignore into t values (1);
ERROR 4078 (HY000): Illegal parameter data types inet6 and int for operation 'SET'
MariaDB [test]> insert ignore into t values ('1');
Query OK, 1 row affected, 1 warning (0.012 sec)

That is, it has no problem converting an integer passed as a string (or any string, for that matter), but has a problem converting the integer itself. Same with other types, e.g. temporal.


Discussion

So far, two arguments were briefly discussed on Slack:

this isn't completely unheard of. GEOMETRY does the same.

This is true; however, geometry types have always been somewhat special and notoriously weird, their behavior in this sense appears to be a limitation, more or less adequately explained by the error message itself (which btw has also been damaged by MDEV-28918):

ERROR 1416 (22003): Cannot get geometry object from data you send to the GEOMETRY field

Well, if it really cannot, then it cannot. There is no reason to extend the legacy deviations onto new data types unless absolutely necessary. We know that INET6 or UUID have no problem casting to a valid value, they did before. Which leads to the 2nd mentioned point:

It's what you can expect from a data type that doesn't have a natural "default" value. Although, one can use all-zeros as a natural ipv6 value, so may be it shouldn't have GEOMETRY behavior. uuid doesn't have a natural default though

"Natural" is a subjective term. We use 0000-00-00 as a default value for dates, which is anything but natural. In any case, both INET6 and UUID have a "hard default" value, and still do even after the change, so it's not really an excuse:

NEW

create or replace table t (a uuid not null);
insert ignore into t values (DEFAULT),('x');
show warnings;
select * from t;
 
MariaDB [test]> show warnings;
+---------+------+--------------------------------------------------------------+
| Level   | Code | Message                                                      |
+---------+------+--------------------------------------------------------------+
| Warning | 1364 | Field 'a' doesn't have a default value                       |
| Warning | 1292 | Incorrect uuid value: 'x' for column `test`.`t`.`a` at row 2 |
+---------+------+--------------------------------------------------------------+
2 rows in set (0.000 sec)
 
MariaDB [test]> select * from t;
+--------------------------------------+
| a                                    |
+--------------------------------------+
| 00000000-0000-0000-0000-000000000000 |
| 00000000-0000-0000-0000-000000000000 |
+--------------------------------------+
2 rows in set (0.000 sec)

And with INET6 it is even more naturally ::.

Btw, it is a weak excuse even for geometry, as it's capable of inserting an empty string when it wants to, but it wants to so inconsistently that it would not be helping my case.



 Comments   
Comment by Alexander Barkov [ 2022-07-26 ]

The idea of the patch for MDEV-28918 is that it consistently disallows cast between INT and INET6 at the statement preparation time.

The same thing happens in this scenario:

CREATE OR REPLACE TABLE t1 (a INT);
INSERT INTO t1 VALUES (ROW(1,2));

ERROR 1241 (21000): Operand should contain 1 column(s)

There is no conversion between INT and ROW. This statement aborts:

  • at the preparation time, before starting iterating rows
  • no matter what the current sql_mode is.

After MDEV-28918, exactly the same thing happens on conversion between INT and INET6. There is no such conversion. The statement also aborts at the preparation time, independently from sql_mode.

Comment by Alexander Barkov [ 2022-07-26 ]

The GEOMETRY data type was also fixed to work the same way with ROW. This statement:

insert into t1 (col_geom) values (1);

now aborts at the preparation time.

There is no conversion from INT to GEOMETRY.

The patch for MDEV-28918 makes things more consistent across different data types, and provides more reasonable behavior:
There is no sense to start iterating rows if it's clear by the data type combination that the operation would fail anyway.

Before the patch such a mixture of INT and GEOMETRY failed, on the first row. It could only succeed on an empty table.

Comment by Elena Stepanova [ 2022-07-26 ]

The idea of this report, on the other hand, is that it's an arbitrary limitation which doesn't solve anything other than a special case, and brings a lot of new inconsistencies and inconveniences instead.

To summarize the effect of the change on the users (even aside inconsistencies, as they are always subjective):

When users care about the exact result of the conversion and prefer getting an error and dealing with it, they use strict modes and normal INSERT / ALTER operations. Thus they are already getting an error, even without the change. After the patch, they'll continue getting an error, only instead of the old informative one with column names, values etc., they'll be getting the non-informative generic "illegal parameter data types".

When users don't care about the exact result and are willing to accept an implicit possibly lossy conversion in order to have the operation succeed, they use non-strict modes and/or IGNORE clauses. Now they can do so, get somewhat informative warnings and possibly act upon them post-factum.
After the patch, they won't be able to perform the operation regardless non-strict mode or IGNORE clause, they will be getting generic non-informative errors instead.

I failed to find a scenario where the change will actually be beneficial for the user.

So, I am assigning it to serg as an architect. I have written most if what I think about this change, and am not going to fight it any further, but if you really want to have it in the future, please write up a standard official answer which we will be giving the users who will reasonably complain about it and require an explanation why we did it, which should preferably be something better than "the idea of the limitation is that it limits".

Comment by Sergei Golubchik [ 2022-08-03 ]

in bb-10.7-bar-MDEV-29159

Comment by Sergei Golubchik [ 2022-08-05 ]

3ebbfd88a03 is ok to push

Comment by Alexander Barkov [ 2022-08-06 ]

Pushed to 10.7.

Comment by Alexander Barkov [ 2022-08-06 ]

Change summary:

1. Changing the error/warning test as follows:

-ERROR HY000: Illegal parameter data types inet6 and int for operation 'SET'
+ERROR HY000: Cannot cast 'int' as 'inet6' in assignment of `db`.`t`.`col`

so in case of a big table it's easier to see which column has the problem.
The new error text is also applied to SP variables.

2. Store assignment failures on incompatible data types now raise errors and abort the statement if:

  • STRICT_ALL_TABLES or STRICT_TRANS_TABLES sql_mode is used, and
  • IGNORE is not used

e.g.:

ERROR HY000: Cannot cast 'point' as 'double' in assignment of `test`.`t1`.`col1`

Otherwise, only a warning is raised and the statement continues, then a warning is issued per row.

Warning  4078  Cannot cast 'double' as 'inet6' in assignment of `test`.`t1`.`col1`
Warning  1292  Incorrect inet6 value: '0' for column `test`.`t1`.`col1` at row 1
Warning  1292  Incorrect inet6 value: '0' for column `test`.`t1`.`col1` at row 2

Comment by Elena Stepanova [ 2022-08-06 ]

For the record (as we will likely face questions about it sooner or later), in layman's terms, the new limitation introduced by MDEV-29159 applies to column types, not to values. The operations are prohibited regardless of what is actually stored or not stored in the columns in question.

E.g. this is prohibited:

MariaDB [test]> create table t (a int);
Query OK, 0 rows affected (0.074 sec)
 
MariaDB [test]> alter table t modify a inet6;
ERROR 4078 (HY000): Cannot cast 'int' as 'inet6' in assignment of `test`.`t`.`a`

– a column cannot be converted between incompatible data types even if the table is empty.

And this is prohibited too:

MariaDB [test]> create table t1 (a int);
Query OK, 0 rows affected (0.033 sec)
 
MariaDB [test]> insert into t1 values (null),(null);
Query OK, 2 rows affected (0.012 sec)
Records: 2  Duplicates: 0  Warnings: 0
 
MariaDB [test]> create table t2 (b inet6);
Query OK, 0 rows affected (0.026 sec)
 
MariaDB [test]> insert into t2 select * from t1;
ERROR 4078 (HY000): Cannot cast 'int' as 'inet6' in assignment of `test`.`t2`.`b`

– insert/select between incompatible types fails even if all values in question are NULLs and the target column is NULL-able.

The patch above doesn't change this, but it ensures that IGNORE clause and non-strict SQL modes relax the limitation and make the operation possible when necessary (and improves the error message so it's easier to understand what is happening).

Comment by Elena Stepanova [ 2022-08-06 ]

Also important to mention, as customary with any new restrictions in new versions, any affected operation will break replication from an old master to a new slave. User must be aware.

Generated at Thu Feb 08 10:06:21 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.