Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-23582

Unexpected result upon division of decimal values (comparing to other implementations)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Blocker
    • Resolution: Not a Bug
    • Affects Version/s: 10.1, 10.2, 10.3, 10.4, 10.5
    • Fix Version/s: N/A
    • Component/s: Data types
    • Labels:

      Description

      SELECT (0.996875 / 2791137695) / 0.000000000357;
      

      10.5 6708e67acc

      MariaDB [test]> SELECT (0.996875 / 2791137695) / 0.000000000357;
      +------------------------------------------+
      | (0.996875 / 2791137695) / 0.000000000357 |
      +------------------------------------------+
      |                         1.12044817927171 |
      +------------------------------------------+
      1 row in set (0.001 sec)
      

      The same result is returned on all MariaDB 10.1+.

      MariaDB prior to 10.1.46, MySQL 5.6, 8.0, PostgreSQL 9.6, Oracle 11g R2, MS SQL Server 2017 and the calculator in my tablet return this (within ~6-8 decimal digit precision):

      postgres=# SELECT (0.996875 / 2791137695) / 0.000000000357;
                  ?column?            
      --------------------------------
       1.0004404124457612918767507003
      (1 row)
      

      The result changed after this commit in 10.1:

      commit 62d73df6b270cc94ba577e96d3bf325170f306fe
      Author: Varun Gupta <varun.gupta@mariadb.com>
      Date:   Wed Jul 22 14:44:25 2020 +0530
       
          MDEV-19232: Floating point precision / value comparison problem
      

      I cannot figure out whether the effect was expected, but given that all other implementations agree on a different outcome, chances are it's a failure.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              varun Varun Gupta (Inactive)
              Reporter:
              elenst Elena Stepanova
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Git Integration