Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-4667

Precision loss on AVG(decimal)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 5.6.1, 6.1.1
    • Fix Version/s: 6.3.1
    • Component/s: PrimProc
    • Labels:
      None

      Description

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(18,17)) ENGINE=ColumnStore;
      INSERT INTO t1 VALUES (-8.99999999999999999);
      SELECT AVG(a) FROM t1;
      

      +--------------------------+
      | AVG(a)                   |
      +--------------------------+
      | -8.999999999999999989600 |
      +--------------------------+
      

      Looks wrong. The expected result is:

      DROP TABLE IF EXISTS t1;
      CREATE TABLE t1 (a DECIMAL(18,17)) ENGINE=InnoDB;
      INSERT INTO t1 VALUES (-8.99999999999999999);
      SELECT AVG(a) FROM t1;
      

      +--------------------------+
      | AVG(a)                   |
      +--------------------------+
      | -8.999999999999999990000 |
      +--------------------------+
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              bar Alexander Barkov
              Reporter:
              bar Alexander Barkov
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:

                  Git Integration