[MCOL-3655] regr_sxy() returns different results across different runs of the same query Created: 2019-12-09  Updated: 2022-10-12  Resolved: 2022-10-12

Status: Closed
Project: MariaDB ColumnStore
Component/s: PrimProc
Affects Version/s: None
Fix Version/s: 22.08.2

Type: Bug Priority: Minor
Reporter: Gagan Goel (Inactive) Assignee: David Hall (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
PartOf
is part of MCOL-3631 regr_ tests returns doubles with high... Closed
is part of MCOL-5188 regr funtions have numeric error when... Closed

 Description   

Here is a sample query to demonstrate different values returned by regr_sxy across multiple runs:

select avg(l_extendedprice), l_discount, l_tax, regr_sxy(3, round(l_discount * l_extendedprice * (1.0+l_tax), 2)) discount from lineitem group by l_discount, l_tax order by l_discount, l_tax;



 Comments   
Comment by David Hall (Inactive) [ 2022-10-12 ]

With MCOL-5188, we changed the algorithm for the regr_nnn functions to one with less variance due to floating point inaccuracies.

Generated at Thu Feb 08 02:44:27 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.