[MDEV-22995] float datatye behave strange even with defined precision of 3 Created: 2020-06-24  Updated: 2020-08-10

Status: Open
Project: MariaDB Server
Component/s: Data types, Storage Engine - InnoDB
Affects Version/s: 10.4.8
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Juerg Oehler Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Compatibility, innodb
Environment:

Linux 5.2.13 SMP PREEMPT Fri Sep 6 17:52:33 UTC 2019 x86_64 GNU/Linux
Server version: 10.4.8-MariaDB OIS-net on Arch Linux
Protocol version: 10
mysql Ver 15.1 Distrib 10.4.8-MariaDB, for Linux (x86_64) using readline 5.1


Attachments: File SimpleFloatTest.sql     File SimpleFloatTest2.sql    

 Description   

hi
i'm using floating point operations with the understanding of https://en.wikipedia.org/wiki/Floating-point_arithmetic and IEEE 754. i'm aware of precision problem, the 6 significant digits. i there for was testing the behavior of mariadb with less than five and last digit is a 0. hence all digits behind are defined.
please have a glimp @SimpleFloatTest.sql. on my implementation integers and anything with #.25 work fine. 0.025 or 0.23 or 0.24 fail to select.

the result is fancy. everything is stored correctly. if i extend precision the question rises if the last digit will be rounded or not (sprintf ("%e"). mariadb does rounding, perl does not, have to check compiler. but currently there is no precision question, if the result showed by SimpleFloatTest.sql appear on every installation.



 Comments   
Comment by Elena Stepanova [ 2020-07-08 ]

I don't understand the problem. You insert 1.24 – so why do you expect numflt = 0.24 to return anything?
Maybe instead of trying to describe it in words, you could just paste the actual result vs expected result, and explain why the latter is expected.

Comment by Juerg Oehler [ 2020-07-12 ]

hi i'm sorry not to be precise enough and was mistaken in one line.
i now did some improvements to the script now named SimpleFloatTest2.sql.
now i do expect all selects return exactly one result as every float number is less than 6 significant digits.

Generated at Thu Feb 08 09:19:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.