[MDEV-5008] Bug calculating/rounding Decimals Created: 2013-09-10 Updated: 2013-09-11 Resolved: 2013-09-10 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | None |
| Affects Version/s: | 10.0.4 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical |
| Reporter: | Adrian Stride | Assignee: | Sergei Golubchik |
| Resolution: | Not a Bug | Votes: | 0 |
| Labels: | None | ||
| Environment: |
Centos 6.3 |
||
| Description |
|
Something is not working correctly on handling decimals. I am still digging to the source, but this example returns NULL and should return 0.
However, changing the numbers to one that would not round up if rounded as below returns the proper 0
|
| Comments |
| Comment by Sergei Golubchik [ 2013-09-10 ] | ||
|
It's not related to decimals, but to floats. (DECIMAL is the exact numeric type, FLOAT is approximate). ACOS(), SIN(), COS() — they use floating-point and implicitly convert their arguments to FLOAT. You see NULL because MariaDB [test]> SELECT SIN(RADIANS(26.197506)) * SIN(RADIANS(26.197506)) + COS(RADIANS(26.197506)) * COS(RADIANS(26.197506)) as `sin^2+cos^2`;
--------------------
-------------------- Which is mathematically incorrect, but calculations with floating point numbers are approximate and that result above is not very surprising. As a workaround, you can wrap your sin^2+cos^2 part of the formula in LEAST(1, ...). | ||
| Comment by Sergei Golubchik [ 2013-09-10 ] | ||
|
alternatively, of course, you can simply replace it with 1 and avoid the problem altogether | ||
| Comment by Adrian Stride [ 2013-09-10 ] | ||
|
The numbers are dynamically inserted when the query is generated to measure distance between two points. This example was when the primary location matches the secondary location. I narrowed down the full query to this point to discover the problem. I have solved the problem in the database by changing the type on the column to FLOAT from DECIMAL, which is the opposite as I would have expected. However, this example shows that the floating point directly does not work when comparing two like calculations. I see many work arounds, but I think it should still calculate both sides equally since they are in fact equal. | ||
| Comment by roberto spadim [ 2013-09-11 ] | ||
|
i know it's not mariadb/mysql problem | ||
| Comment by roberto spadim [ 2013-09-11 ] | ||
|
i'm not sure if wolframalpha is a good source of test/result, i'm assuming that it is (i use it a lot at least) the result is 0 with 26.197506 in other words, it's a round problem... |