Details
-
Bug
-
Status: Closed (View Workflow)
-
Minor
-
Resolution: Fixed
-
5.4.1
-
None
-
2021-2, 2021-3, 2021-4, 2021-5
Description
The current statistical functions code doesn't take into account a minuscule negative deviation value that is actually represents a float 0.
According with the basic properties of Variance [1] and the regr_r2 calculation algo [2] the covar_popy must be non-negative so that sqrt(covar_popy) in regr_r2::evaluate() doesn't return NaN.
4QA This query must return 0.0000000000 according with the reference
{ select round(regr_r2(l_tax, l_extendedprice),10) from lineitem where l_tax = 0.05 group by l_tax order by l_tax; }1. https://en.wikipedia.org/wiki/Variance
2. https://docs.oracle.com/cd/B28359_01/server.111/b28286/functions139.htm#SQLRF00696