Details
-
Bug
-
Status: Open (View Workflow)
-
Major
-
Resolution: Unresolved
-
5.5.1, 5.6.1, 6.1.1
-
None
Description
Because of using double as a temporary storage for decimal in Func_mod::getDecimalVal, the MOD operator looses precision on huge narrow decimal values:
DROP TABLE IF EXISTS t1; |
CREATE TABLE t1 (a DECIMAL(18,0)) ENGINE=ColumnStore; |
INSERT INTO t1 VALUES (999999999999999999); |
SELECT a MOD 10000 FROM t1; |
+-------------+
|
| a MOD 10000 |
|
+-------------+
|
| 0 |
|
+-------------+
|
Looks wrong. The expected result is:
DROP TABLE IF EXISTS t1; |
CREATE TABLE t1 (a DECIMAL(18,0)) ENGINE=InnoDB; |
INSERT INTO t1 VALUES (999999999999999999); |
SELECT a MOD 10000 FROM t1; |
+-------------+
|
| a MOD 10000 |
|
+-------------+
|
| 9999 |
|
+-------------+
|
The same problem presents in the template Func_mod::doDecimal:
DROP TABLE IF EXISTS t1; |
CREATE TABLE t1 (a DECIMAL(18,0)) ENGINE=ColumnStore; |
INSERT INTO t1 VALUES (999999999999999999); |
SELECT RAND(9999), RAND(a MOD 10000) FROM t1; |
+--------------------+---------------------+
|
| RAND(9999) | RAND(a MOD 10000) |
|
+--------------------+---------------------+
|
| 0.7361292641015065 | 0.15522042769493574 |
|
+--------------------+---------------------+
|
The result for the second column looks wrong.
The expected result is:
DROP TABLE IF EXISTS t1; |
CREATE TABLE t1 (a DECIMAL(18,0)) ENGINE=InnoDB; |
INSERT INTO t1 VALUES (999999999999999999); |
SELECT RAND(9999), RAND(a MOD 10000) FROM t1; |
+--------------------+--------------------+
|
| RAND(9999) | RAND(a MOD 10000) |
|
+--------------------+--------------------+
|
| 0.7361292641015065 | 0.7361292641015065 |
|
+--------------------+--------------------+
|
