[MCOL-148] sum() and avg() return incorrect result for float when calculation overflows float range Created: 2016-06-17 Updated: 2019-05-02 Resolved: 2019-05-02 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | ExeMgr |
| Affects Version/s: | 1.0.1 |
| Fix Version/s: | 1.2.3 |
| Type: | Bug | Priority: | Minor |
| Reporter: | Daniel Lee (Inactive) | Assignee: | David Hall (Inactive) |
| Resolution: | Duplicate | Votes: | 1 |
| Labels: | None | ||
| Issue Links: |
|
||||||||||||||||
| Epic Link: | ColumnStore Compatibility Improvements | ||||||||||||||||
| Description |
|
Build tested: getsoftwareinfo Fri Jun 17 17:03:28 2016 Name : mariadb-columnstore-platform Relocations: (not relocatable) MariaDB [mytest]> select cfloat from datatypetestm;
-------------
------------- MariaDB [mytest]> select SUM(CFLOAT) from datatypetestm;
--------------
-------------- MariaDB [mytest]> select AVG(CFLOAT) from datatypetestm;
-------------
------------- Both MariaDB and InfiniDB return 0. |
| Comments |
| Comment by David Hall (Inactive) [ 2016-06-21 ] |
|
This most likely is caused by undetected data overflow. |
| Comment by David Hall (Inactive) [ 2016-09-14 ] |
|
The accumulation occurs in the return value, which is the same type as the row value. The failed tests overflow the accumulator. Accumulation should take place in a DOUBLE, rather than FLOAT, but it isn't designed that way. This behaviour is seen in InfiniDB, so it's not new. This is not a trivial fix. Some learning curve on how the thing works is needed. |
| Comment by David Thompson (Inactive) [ 2016-10-10 ] |
|
The system behaves correctly when normal range float (and double) values are utilized, however should the accumulated values exceed the float (or double range) then this behavior will be observed. Since this is the existing behavior of infinidb and and an edge case, this is not a priority for beta. |