Uploaded image for project: 'MariaDB ColumnStore'
  1. MariaDB ColumnStore
  2. MCOL-641 Full DECIMAL support in ColumnStore
  3. MCOL-4478

avg() on wide DECIMAL doesn't round the last digit properly

    XMLWordPrintable

    Details

    • Type: Sub-Task
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.1.1
    • Component/s: ExeMgr, PrimProc
    • Labels:
      None
    • Sprint:
      2020-1, 2020-2, 2020-3, 2020-4, 2020-5, 2020-6, 2020-7, 2021-1, 2021-2, 2021-3, 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

      Description

      Consider the scenario:

      CREATE TABLE cs5(d DECIMAL(25,19)) ENGINE=columnstore;
      INSERT INTO cs5 VALUES (1),(0.1),(0.05),(0.05),(0.05),(0.05),(0.2),(0.154),(0.1),(0.05),(0.05);
      SELECT avg(d) FROM cs5;
      

      The expected result is:

      avg(d)
      0.16854545454545454545455
      

      however MCS doesn't round the last digit properly

      avg(d)
      0.16854545454545454545454
      

        Attachments

          Activity

            People

            Assignee:
            alexey.antipovsky Alexey Antipovsky
            Reporter:
            drrtuy Roman
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Git Integration