Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4631

CAST(double AS SIGNED) returns 0 or NULL

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Fixed
    • 5.6.1, 6.1.1
    • 6.1.1
    • PrimProc
    • None
    • 2021-5

    Description

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (d1 DOUBLE, d2 DOUBLE NOT NULL) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (9.2233720368547758e+18, 9.2233720368547758e+18);
      INSERT INTO t1 VALUES (18446744073709551614,18446744073709551614);
      SELECT d1, CAST(d1 AS SIGNED), CAST(d2 AS SIGNED) FROM t1;
      

      +-----------------------+---------------------+---------------------+
      | d1                    | CAST(d1 AS SIGNED)  | CAST(d2 AS SIGNED)  |
      +-----------------------+---------------------+---------------------+
      |  9.223372036854776e18 |                NULL |                   0 |
      | 1.8446744073709552e19 | 9223372036854775807 | 9223372036854775807 |
      +-----------------------+---------------------+---------------------+
      

      Notice:

      • the value 9.2233720368547758e+18 was erroneously converted to NULL and 0
      • a huger value 8446744073709552e19 was correctly converted to 9223372036854775807

      The expected result is:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (d1 DOUBLE, d2 DOUBLE NOT NULL) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (9.2233720368547758e+18, 9.2233720368547758e+18);
      INSERT INTO t1 VALUES (18446744073709551614,18446744073709551614);
      SELECT d1, CAST(d1 AS SIGNED), CAST(d2 AS SIGNED) FROM t1;
      

      +-----------------------+---------------------+---------------------+
      | d1                    | CAST(d1 AS SIGNED)  | CAST(d2 AS SIGNED)  |
      +-----------------------+---------------------+---------------------+
      |  9.223372036854776e18 | 9223372036854775807 | 9223372036854775807 |
      | 1.8446744073709552e19 | 9223372036854775807 | 9223372036854775807 |
      +-----------------------+---------------------+---------------------+
      

      The problem happens in this piece of the code in Func_cast_signed::getIntVal() in func_cast.cpp:

              case execplan::CalpontSystemCatalog::FLOAT:
              case execplan::CalpontSystemCatalog::UFLOAT:
              case execplan::CalpontSystemCatalog::DOUBLE:
              case execplan::CalpontSystemCatalog::UDOUBLE:
              {
                  double value = parm[0]->data()->getDoubleVal(row, isNull);
       
                  if (value > 0)
                      value += 0.5;
                  else if (value < 0)
                      value -= 0.5;
       
                  int64_t ret = (int64_t) value;
       
                  if (value > (double) numeric_limits<int64_t>::max())
                      ret = numeric_limits<int64_t>::max();
                  else if (value < (double) (numeric_limits<int64_t>::min() + 2))
                      ret = numeric_limits<int64_t>::min() + 2; // IDB min for bigint
       
                  return ret;
              }
      

      Let's print variable values after this statement:

      int64_t ret = (int64_t) value;
      

      (gdb) p	value
      $25 = 9.2233720368547758e+18
      (gdb) p	ret 
      $26 = -9223372036854775808
      

      Notice, ret is wrong. The code should check for the int64_t range before casting double to int64_t.

      Attachments

        Issue Links

          Activity

            People

              bar Alexander Barkov
              bar Alexander Barkov
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.