[MDEV-5008] Bug calculating/rounding Decimals Created: 2013-09-10  Updated: 2013-09-11  Resolved: 2013-09-10

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 10.0.4
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Adrian Stride Assignee: Sergei Golubchik
Resolution: Not a Bug Votes: 0
Labels: None
Environment:

Centos 6.3



 Description   

Something is not working correctly on handling decimals. I am still digging to the source, but this example returns NULL and should return 0.

SELECT
(ACOS(SIN(RADIANS(26.197506)) * SIN(RADIANS(26.197506)) + COS(RADIANS(26.197506)) * COS(RADIANS(26.197506)) * COS(RADIANS(-80.107953) - RADIANS(-80.107953)))*3959) AS `Distance` 

However, changing the numbers to one that would not round up if rounded as below returns the proper 0

SELECT
(ACOS(SIN(RADIANS(26.197504)) * SIN(RADIANS(26.197504)) + COS(RADIANS(26.197504)) * COS(RADIANS(26.197504)) * COS(RADIANS(-80.107953) - RADIANS(-80.107953)))*3959) AS `Distance` 



 Comments   
Comment by Sergei Golubchik [ 2013-09-10 ]

It's not related to decimals, but to floats. (DECIMAL is the exact numeric type, FLOAT is approximate).

ACOS(), SIN(), COS() — they use floating-point and implicitly convert their arguments to FLOAT.

You see NULL because

MariaDB [test]> SELECT SIN(RADIANS(26.197506)) * SIN(RADIANS(26.197506)) + COS(RADIANS(26.197506)) * COS(RADIANS(26.197506)) as `sin^2+cos^2`;
--------------------

sin^2+cos^2

--------------------

1.0000000000000002

--------------------
1 row in set (0.00 sec)

Which is mathematically incorrect, but calculations with floating point numbers are approximate and that result above is not very surprising.

As a workaround, you can wrap your sin^2+cos^2 part of the formula in LEAST(1, ...).

Comment by Sergei Golubchik [ 2013-09-10 ]

alternatively, of course, you can simply replace it with 1 and avoid the problem altogether

Comment by Adrian Stride [ 2013-09-10 ]

The numbers are dynamically inserted when the query is generated to measure distance between two points. This example was when the primary location matches the secondary location. I narrowed down the full query to this point to discover the problem. I have solved the problem in the database by changing the type on the column to FLOAT from DECIMAL, which is the opposite as I would have expected.

However, this example shows that the floating point directly does not work when comparing two like calculations. I see many work arounds, but I think it should still calculate both sides equally since they are in fact equal.

Comment by roberto spadim [ 2013-09-11 ]

i know it's not mariadb/mysql problem
maybe you should consider the round of 26.197506
if you use 26.197505, the result is 0
but i'm not sure if the problem is math, or round of sin/cos

Comment by roberto spadim [ 2013-09-11 ]

i'm not sure if wolframalpha is a good source of test/result, i'm assuming that it is (i use it a lot at least)

http://www.wolframalpha.com/input/?i=%28ACOS%28SIN%28RADIANS%2826.197506%29%29+*+SIN%28RADIANS%2826.197506%29%29+%2B+COS%28RADIANS%2826.197506%29%29+*+COS%28RADIANS%2826.197506%29%29+*+COS%28RADIANS%28-80.107953%29+-+RADIANS%28-80.107953%29%29%29*3959%29+&dataset=

the result is 0 with 26.197506

in other words, it's a round problem...
maybe some considerations should be done... or a more "precise" lib should be used...
don't know what is best (new functions / rewrite of current functions from float to decimal? or float to double?), since i'm not using mariadb for precise math with trigonometric functions

Generated at Thu Feb 08 07:00:56 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.