Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4631

CAST(double AS SIGNED) returns 0 or NULL

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.6.1, 6.1.1
    • Fix Version/s: 6.1.1
    • Component/s: PrimProc
    • Labels:
      None
    • Sprint:
      2021-5

      Description

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (d1 DOUBLE, d2 DOUBLE NOT NULL) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (9.2233720368547758e+18, 9.2233720368547758e+18);
      INSERT INTO t1 VALUES (18446744073709551614,18446744073709551614);
      SELECT d1, CAST(d1 AS SIGNED), CAST(d2 AS SIGNED) FROM t1;
      

      +-----------------------+---------------------+---------------------+
      | d1                    | CAST(d1 AS SIGNED)  | CAST(d2 AS SIGNED)  |
      +-----------------------+---------------------+---------------------+
      |  9.223372036854776e18 |                NULL |                   0 |
      | 1.8446744073709552e19 | 9223372036854775807 | 9223372036854775807 |
      +-----------------------+---------------------+---------------------+
      

      Notice:

      • the value 9.2233720368547758e+18 was erroneously converted to NULL and 0
      • a huger value 8446744073709552e19 was correctly converted to 9223372036854775807

      The expected result is:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (d1 DOUBLE, d2 DOUBLE NOT NULL) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (9.2233720368547758e+18, 9.2233720368547758e+18);
      INSERT INTO t1 VALUES (18446744073709551614,18446744073709551614);
      SELECT d1, CAST(d1 AS SIGNED), CAST(d2 AS SIGNED) FROM t1;
      

      +-----------------------+---------------------+---------------------+
      | d1                    | CAST(d1 AS SIGNED)  | CAST(d2 AS SIGNED)  |
      +-----------------------+---------------------+---------------------+
      |  9.223372036854776e18 | 9223372036854775807 | 9223372036854775807 |
      | 1.8446744073709552e19 | 9223372036854775807 | 9223372036854775807 |
      +-----------------------+---------------------+---------------------+
      

      The problem happens in this piece of the code in Func_cast_signed::getIntVal() in func_cast.cpp:

              case execplan::CalpontSystemCatalog::FLOAT:
              case execplan::CalpontSystemCatalog::UFLOAT:
              case execplan::CalpontSystemCatalog::DOUBLE:
              case execplan::CalpontSystemCatalog::UDOUBLE:
              {
                  double value = parm[0]->data()->getDoubleVal(row, isNull);
       
                  if (value > 0)
                      value += 0.5;
                  else if (value < 0)
                      value -= 0.5;
       
                  int64_t ret = (int64_t) value;
       
                  if (value > (double) numeric_limits<int64_t>::max())
                      ret = numeric_limits<int64_t>::max();
                  else if (value < (double) (numeric_limits<int64_t>::min() + 2))
                      ret = numeric_limits<int64_t>::min() + 2; // IDB min for bigint
       
                  return ret;
              }
      

      Let's print variable values after this statement:

      int64_t ret = (int64_t) value;
      

      (gdb) p	value
      $25 = 9.2233720368547758e+18
      (gdb) p	ret 
      $26 = -9223372036854775808
      

      Notice, ret is wrong. The code should check for the int64_t range before casting double to int64_t.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              bar Alexander Barkov
              Reporter:
              bar Alexander Barkov
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration