[MDEV-8921] Wrong result for CAST(AVG(double_column) AS SIGNED) Created: 2015-10-08  Updated: 2023-04-27

Status: Open
Project: MariaDB Server
Component/s: Data types
Affects Version/s: 10.0, 10.1
Fix Version/s: 10.4

Type: Bug Priority: Major
Reporter: Alexander Barkov Assignee: Alexander Barkov
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Relates
relates to MDEV-8918 Wrong result for CAST(AVG(bigint_colu... Closed

 Description   

It's similar to MDEV-8919, but now for a DOUBLE rather than BIGINT column.

DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (id INT, a DOUBLE);
INSERT INTO t1 VALUES (1,0x7FFFFFFFFFFFFFFF),(2,0x7FFFFFFFFFFFFFFF);
SELECT id, AVG(a) AS avg, CAST(MIN(a) AS SIGNED) AS cast_min,CAST(AVG(a) AS SIGNED) AS cast_avg FROM t1 GROUP BY id HAVING avg!=123 ORDER BY id;

returns

+------+----------------------+---------------------+----------------------+
| id   | avg                  | cast_min            | cast_avg             |
+------+----------------------+---------------------+----------------------+
|    1 | 9.223372036854776e18 | 9223372036854775807 | -9223372036854775808 |
|    2 | 9.223372036854776e18 | 9223372036854775807 | -9223372036854775808 |
+------+----------------------+---------------------+----------------------+

The values for the cast_avg column are wrong.

A similar problem:

DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (a DOUBLE);
INSERT INTO t1 VALUES (0x7FFFFFFFFFFFFFFF);
SELECT MIN(a), SUM(a), CAST(SUM(a) AS SIGNED), CAST(AVG(a) AS SIGNED) FROM t1;

returns

+----------------------+----------------------+------------------------+------------------------+
| MIN(a)               | SUM(a)               | CAST(SUM(a) AS SIGNED) | CAST(AVG(a) AS SIGNED) |
+----------------------+----------------------+------------------------+------------------------+
| 9.223372036854776e18 | 9.223372036854776e18 |   -9223372036854775808 |   -9223372036854775808 |
+----------------------+----------------------+------------------------+------------------------+

The value for the third and the fourth columns are wrong.


Generated at Thu Feb 08 07:30:47 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.