Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4649

CAST(double AS UNSIGNED) returns 0

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.6.1, 6.1.1
    • Fix Version/s: 6.1.1
    • Component/s: PrimProc
    • Labels:
      None
    • Sprint:
      2021-5

      Description

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (d1 DOUBLE, d2 DOUBLE NOT NULL) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (18446744073709551614,18446744073709551614);
      SELECT d1, CAST(d1 AS UNSIGNED), CAST(d2 AS UNSIGNED) FROM t1;
      

      +-----------------------+----------------------+----------------------+
      | d1                    | CAST(d1 AS UNSIGNED) | CAST(d2 AS UNSIGNED) |
      +-----------------------+----------------------+----------------------+
      | 1.8446744073709552e19 |                    0 |                    0 |
      +-----------------------+----------------------+----------------------+
      

      Looks wrong.

      The expected result is:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (d1 DOUBLE, d2 DOUBLE NOT NULL) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (18446744073709551614,18446744073709551614);
      SELECT d1, CAST(d1 AS UNSIGNED), CAST(d2 AS UNSIGNED) FROM t1;
      

      +-----------------------+----------------------+----------------------+
      | d1                    | CAST(d1 AS UNSIGNED) | CAST(d2 AS UNSIGNED) |
      +-----------------------+----------------------+----------------------+
      | 1.8446744073709552e19 | 18446744073709551615 | 18446744073709551615 |
      +-----------------------+----------------------+----------------------+
      

      The problem happens in this piece of the code in Func_cast_unsigned::getUintVal():

      289                 case execplan::CalpontSystemCatalog::FLOAT:
      290                 case execplan::CalpontSystemCatalog::UFLOAT:
      291                 case execplan::CalpontSystemCatalog::DOUBLE:
      292                 case execplan::CalpontSystemCatalog::UDOUBLE:
      293                 {
      294                     double value = parm[0]->data()->getDoubleVal(row, isNull);
      295
      296                     if (value > 0)
      297                         value += 0.5;
      298                     else if (value < 0)
      299                         value -= 0.5;
      300
      301                     uint64_t ret = (uint64_t) value;    ///   THIS LINE IS WRONG
      302
      303                     if (value > (double) numeric_limits<uint64_t>::max() - 2)
      304                         ret = numeric_limits<int64_t>::max();
      305                     else if (value < 0)
      306                         ret = 0;
      307
      308                     return ret;  
      

      GDB session screenshot:

      (gdb) p	value
      $4 = 1.8446744073709552e+19
      (gdb) p	ret
      $5 = 0
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              bar Alexander Barkov
              Reporter:
              bar Alexander Barkov
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration