|
The patch for MCOL-4633 removed duplicate code by introducing TDecimal64::toSInt64Round() and reusing it in a few places:
- Func_cast_signed::getIntVal
- Func_char::getStrVal
- Func_elt::getStrVal
- makedate
- Func_maketime::getStrVal
The patch for MCOL-4633 did not change the behavior: narrow DECIMAL to INT conversion in TDecimal64::toSInt64Round() is still performed in double format by multiplying to pow(10,scale),
like in all duplicate chunks before the patch. This causes precision loss on huge narrow decimal values.
Note, the patch for MCOL-4633 intentionally did not change the behavior because the underlying code is very essential and fragile. Let's do one small thing at a time, to commit small and easy to review changes.
Under terms of this task we'll do the next step - fix TDecimal64::toSInt64Round() not to use DOUBLE arithmetic, to fix precision loss in the following examples:
DROP TABLE IF EXISTS t1;
|
CREATE TABLE t1 (a DECIMAL(17,1)) ENGINE=ColumnStore;
|
INSERT INTO t1 VALUES (8999999999999999.0);
|
SELECT
|
CAST(a AS SIGNED),
|
HEX(CHAR(a USING latin1))
|
FROM t1;
|
+-------------------+---------------------------+
|
| CAST(a AS SIGNED) | HEX(CHAR(a USING latin1)) |
|
+-------------------+---------------------------+
|
| 8999999999999998 | CAFA7FFE |
|
+-------------------+---------------------------+
|
Looks wrong.
The expected result is:
DROP TABLE IF EXISTS t1;
|
CREATE TABLE t1 (a DECIMAL(17,1)) ENGINE=InnoDB;
|
INSERT INTO t1 VALUES (8999999999999999.0);
|
SELECT
|
CAST(a AS SIGNED),
|
HEX(CHAR(a USING latin1))
|
FROM t1;
|
+-------------------+---------------------------+
|
| CAST(a AS SIGNED) | HEX(CHAR(a USING latin1)) |
|
+-------------------+---------------------------+
|
| 8999999999999999 | CAFA7FFF |
|
+-------------------+---------------------------+
|
|