Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4609

TreeNode::getIntVal() does not round: implicit DECIMAL->INT cast is not MariaDB compatible

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.6.1, 6.1.1, 5.5.1
    • Fix Version/s: 6.1.1
    • Component/s: PrimProc
    • Labels:
      None

      Description

      The problem described below is also with narrow and wide DECIMAL.

      The method TreeNode::getIntVal() performs truncation instead of rounding when converting from DECIMAL to signed integer:

              case CalpontSystemCatalog::DECIMAL:
              case CalpontSystemCatalog::UDECIMAL:
              {
                  if (fResultType.colWidth == datatypes::MAXDECIMALWIDTH)
                  {
                      return static_cast<int64_t>(fResult.decimalVal.getIntegralPart());
                  }
                  else
                  {
                      return (int64_t)(fResult.decimalVal.value / pow((double)10, fResult.decimalVal.scale));
                  }
              }
      

      This makes ColumnStore return MariaDB incompatible values:

      RAND()

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(10,1)) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (1.4),(1.5);
      SELECT a, RAND(a), RAND(CAST(a AS DECIMAL(10,1))) FROM t1;
      

      +------+---------------------+--------------------------------+
      | a    | RAND(a)             | RAND(CAST(a AS DECIMAL(10,1))) |
      +------+---------------------+--------------------------------+
      |  1.4 | 0.40540353712197724 |            0.40540353712197724 |
      |  1.5 | 0.40540353712197724 |            0.40540353712197724 |
      +------+---------------------+--------------------------------+
      

      Looks wrong. RAND() did not round the seed value from 1.5 to 2.

      The expected result is:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(10,1)) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (1.4),(1.5);
      SELECT a, RAND(a), RAND(CAST(a AS DECIMAL(10,1))) FROM t1;
      

      +------+---------------------+--------------------------------+
      | a    | RAND(a)             | RAND(CAST(a AS DECIMAL(10,1))) |
      +------+---------------------+--------------------------------+
      |  1.4 | 0.40540353712197724 |            0.40540353712197724 |
      |  1.5 |  0.6555866465490187 |             0.6555866465490187 |
      +------+---------------------+--------------------------------+
      

      TRUNCATE()

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(10,1)) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (1.4),(1.5);
      SELECT TRUNCATE(1.2345, a) FROM t1;
      

      +---------------------+
      | TRUNCATE(1.2345, a) |
      +---------------------+
      |              1.2000 |
      |              1.2000 |
      +---------------------+
      

      The expected result is:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(10,1)) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (1.4),(1.5);
      SELECT TRUNCATE(1.2345, a) FROM t1;
      

      +---------------------+
      | TRUNCATE(1.2345, a) |
      +---------------------+
      |              1.2000 |
      |              1.2300 |
      +---------------------+
      

      CONV()

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(10,1)) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (15.4),(15.5),(16.0);
      SELECT a, CONV(16,a,10) FROM t1;
      

      +------+---------------+
      | a    | CONV(16,a,10) |
      +------+---------------+
      | 15.4 | 21            |
      | 15.5 | 21            | <--- this is wrong
      | 16.0 | 22            |
      +------+---------------+
      

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(10,1)) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (15.4),(15.5),(16.0);
      SELECT a, CONV(16,a,10) FROM t1;
      

      +------+---------------+
      | a    | CONV(16,a,10) |
      +------+---------------+
      | 15.4 | 21            |
      | 15.5 | 22            |
      | 16.0 | 22            |
      +------+---------------+
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              tntnatbry Gagan Goel
              Reporter:
              bar Alexander Barkov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration