Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-41978

VEC_DISTANCE_COSINE fails cosine identities for an accepted tiny nonzero vector

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • Major
    • Resolution: Duplicate
    • 12.3.3
    • N/A
    • Vector search
    • None
    • Linux x86_64
    • Not for Release Notes

    Description

      I get failed cosine-distance checks for an accepted nonzero vector containing 1e-30 on MariaDB 12.3.3. The vector has a nonzero encoding and positive Euclidean distance from zero, but VEC_DISTANCE_COSINE fails both self-distance and same-direction comparisons. The same self-distance check also fails through a prepared statement, while a unit-vector control passes.

      This occurs when cosine distance is calculated for an accepted vector whose components are small enough for their products to underflow.

      The following single-session SQL reproduces the problem:

      SET @small = VEC_FROMTEXT('[1e-30]');
      SET @unit = VEC_FROMTEXT('[1]');
       
      SELECT 'inputs' AS check_name,
             @small IS NOT NULL AS valid_ok,
             HEX(@small) <> '00000000' AS nonzero_ok,
             VEC_DISTANCE_EUCLIDEAN(@small, VEC_FROMTEXT('[0]')) > 0 AS euclidean_ok;
       
      SELECT 'direct' AS check_name,
             COALESCE(ABS(VEC_DISTANCE_COSINE(@unit, @unit)) < 1e-6, 0) AS control_ok,
             COALESCE(ABS(VEC_DISTANCE_COSINE(@small, @small)) < 1e-6, 0) AS small_self_ok,
             COALESCE(ABS(VEC_DISTANCE_COSINE(@small, @unit)) < 1e-6, 0) AS same_direction_ok;
       
      SHOW WARNINGS;
       
      PREPARE cosine_stmt FROM
        'SELECT ''prepared'' AS check_name,
                COALESCE(ABS(VEC_DISTANCE_COSINE(?, ?)) < 1e-6, 0) AS cosine_ok';
      EXECUTE cosine_stmt USING @small, @small;
      DEALLOCATE PREPARE cosine_stmt;
       
      SET @small = NULL, @unit = NULL;

      The recorded semantic rows were:

      check_name  valid_ok  nonzero_ok  euclidean_ok
      inputs      1         1           1
       
      check_name  control_ok  small_self_ok  same_direction_ok
      direct      1           0              0
       
      check_name  cosine_ok
      prepared    0

      No warnings or errors were produced by this confirmation case.

      For a nonzero vector, cosine distance from itself must be zero. The vectors [1e-30] and [1] also have the same positive direction, so their cosine distance must be zero. All three tested comparisons should therefore evaluate to 1 within the supplied 1e-6 tolerance, and direct and prepared execution should agree. Instead, the unit-vector control evaluates to 1 while both tiny-vector checks and the prepared check evaluate to 0.

      The observed result is consistent with the component products underflowing before they are accumulated at wider precision, causing a valid nonzero vector to be treated as having a zero norm during cosine-distance calculation.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              faner fan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.