[CONPY-87] Error with numeric values returned from query. Created: 2020-07-16  Updated: 2020-07-17  Resolved: 2020-07-17

Status: Closed
Project: MariaDB Connector/Python
Component/s: Generic
Affects Version/s: 1.0.0
Fix Version/s: N/A

Type: Bug Priority: Major
Reporter: Michal Sarna Assignee: Georg Richter
Resolution: Not a Bug Votes: 0
Labels: None

Attachments: File termik.sql     File test.py    

 Description   

Mac OS X 10.15.5
Brew installed: python@3.8 and mariadb-connector-c
pip installed mariadb connector version 1.0.0

Python Version:
(Python 3.8.3 (default, Jul 8 2020, 14:27:55)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.)

In DB data looks like:
mysql> SELECT date, time, id, value FROM odczyty WHERE date = '2020-07-16' ORDER BY seq_no DESC LIMIT 10;
-----------------------------------------+

date time id value

-----------------------------------------+

2020-07-16 18:05:01 28-0116231917ee 17.875
2020-07-16 18:05:01 28-02161fc3e5ee 17.937
2020-07-16 18:05:01 28-01162336c8ee 17.875
2020-07-16 18:00:01 28-0116231917ee 17.875
2020-07-16 18:00:01 28-02161fc3e5ee 17.937
2020-07-16 18:00:01 28-01162336c8ee 17.937
2020-07-16 17:55:01 28-0116231917ee 17.875
2020-07-16 17:55:01 28-02161fc3e5ee 17.937
2020-07-16 17:55:01 28-01162336c8ee 17.875
2020-07-16 17:50:01 28-0116231917ee 17.875

-----------------------------------------+
10 rows in set (0.01 sec)

In script
cur.execute("SELECT date, time, id, value FROM odczyty WHERE date=? ORDER BY seq_no DESC LIMIT 10", (today,) )

returns values like:
Date: 2020-07-16 17:55:01 | id: 28-0116231917ee | value: 17.875
Date: 2020-07-16 17:55:01 | id: 28-02161fc3e5ee | value: 17.937000274658203
Date: 2020-07-16 17:55:01 | id: 28-01162336c8ee | value: 17.875
Date: 2020-07-16 17:50:01 | id: 28-0116231917ee | value: 17.875
Date: 2020-07-16 17:50:01 | id: 28-02161fc3e5ee | value: 17.937000274658203
Date: 2020-07-16 17:50:01 | id: 28-01162336c8ee | value: 17.875
Date: 2020-07-16 17:45:01 | id: 28-0116231917ee | value: 17.875
Date: 2020-07-16 17:45:01 | id: 28-02161fc3e5ee | value: 17.937000274658203
Date: 2020-07-16 17:45:01 | id: 28-01162336c8ee | value: 17.875
Date: 2020-07-16 17:40:01 | id: 28-0116231917ee | value: 17.875

but when run like this:
cur.execute("SELECT date, time, id, value FROM odczyty WHERE date='2020-07-16' ORDER BY seq_no DESC LIMIT 10")

returns proper values:
Date: 2020-07-16 17:55:01 | id: 28-0116231917ee | value: 17.875
Date: 2020-07-16 17:55:01 | id: 28-02161fc3e5ee | value: 17.937
Date: 2020-07-16 17:55:01 | id: 28-01162336c8ee | value: 17.875
Date: 2020-07-16 17:50:01 | id: 28-0116231917ee | value: 17.875
Date: 2020-07-16 17:50:01 |t id: 28-02161fc3e5ee | value: 17.937
Date: 2020-07-16 17:50:01 | id: 28-01162336c8ee | value: 17.875
Date: 2020-07-16 17:45:01 | id: 28-0116231917ee | value: 17.875
Date: 2020-07-16 17:45:01 | id: 28-02161fc3e5ee | value: 17.937
Date: 2020-07-16 17:45:01 | id: 28-01162336c8ee | value: 17.875
Date: 2020-07-16 17:40:01 | id: 28-0116231917ee | value: 17.875

Server is MariaDB 10 on Synology.
I've attached DB dump and python test script.



 Comments   
Comment by Georg Richter [ 2020-07-16 ]

Hi, thank you for your bug report.

Did you read https://mariadb.com/kb/en/floating-point-accuracy/ ?

Comment by Michal Sarna [ 2020-07-17 ]

Hi.
I have not. But what was weird for me is that returned values were not all the same. Hope You will be able to track down this. I've added _round _so now it's showing properly

Comment by Georg Richter [ 2020-07-17 ]

You will get only correct values, if it is converted to Decimal, so the corresponding datatype needs to be decimal instead of float or double:

mysql> select "1.1" * 3;
+--------------------+
| "1.1" * 3          |
+--------------------+
| 3.3000000000000003 |
+--------------------+
1 row in set (0,00 sec)

>>> # double/float in python is not accurate
>>> print(1.1 * 3)
3.3000000000000003
>>> import decimal
>>> print(Decimal("1.1") * 3)
3.3

or with a cursor:

>>> cursor.execute("drop table t1");
>>> cursor.execute("create temporary table t1 (a float, b double, c decimal(10,3))");
>>> cursor.execute("insert into t1 values (1.1, 1.1, 1.1)");
>>> cursor.execute("select a * 3, b * 3, c * 3 from t1");
>>> cursor.fetchone()
(3.3000000715255737, 3.3000000000000003, Decimal('3.300'))

Comment by Michal Sarna [ 2020-07-17 ]

Ok ... and where precisely in my script is any type of multiplication ?
And funny thing is that connector is working properly with static select but with any value passed as argument You get trash data on random lines.
So please check again.

Comment by Georg Richter [ 2020-07-17 ]

The multiplication should just show you that float is not accurate and the use of FLOAT should be avoided, as mentionen in MariaDB documentation: "Using FLOAT might give you some unexpected problems because all calculations in MariaDB are done with double precision"

The difference in output with and without placeholder is that one uses binary protocol (transferring float as 4-byte binary) and the other uses text protocol.

Comment by Michal Sarna [ 2020-07-17 ]

OK. I've get it. Changed in DB from float double.

Thx for Your time.

Generated at Thu Feb 08 03:30:09 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.