Full DECIMAL support in ColumnStore (MCOL-641)

[MCOL-4478] avg() on wide DECIMAL doesn't round the last digit properly Created: 2020-12-30  Updated: 2021-02-01  Resolved: 2020-12-30

Status: Closed
Project: MariaDB ColumnStore
Component/s: ExeMgr, PrimProc
Affects Version/s: None
Fix Version/s: 6.1.1

Type: Sub-Task Priority: Minor
Reporter: Roman Assignee: Alexey Antipovsky (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Sprint: 2020-1, 2020-2, 2020-3, 2020-4, 2020-5, 2020-6, 2020-7, 2021-1, 2021-2, 2021-3, 2021-4, 2021-5, 2021-6, 2021-7, 2021-8, 2021-9

 Description   

Consider the scenario:

CREATE TABLE cs5(d DECIMAL(25,19)) ENGINE=columnstore;
 
INSERT INTO cs5 VALUES (1),(0.1),(0.05),(0.05),(0.05),(0.05),(0.2),(0.154),(0.1),(0.05),(0.05);
 
SELECT avg(d) FROM cs5;

The expected result is:

avg(d)
 
0.16854545454545454545455

however MCS doesn't round the last digit properly

avg(d)
 
0.16854545454545454545454



 Comments   
Comment by Roman [ 2020-12-30 ]

Plz review.

Generated at Thu Feb 08 02:50:39 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.