Details
-
Bug
-
Status: Closed (View Workflow)
-
Minor
-
Resolution: Won't Fix
-
11.8.1
Description
sql: SELECT VEC_DISTANCE_COSINE(VEC_FROMTEXT('[1,2,3]'), VEC_FROMTEXT('[0,0,0]'));
return value is 0.
static double calc_distance_cosine(float *v1, float *v2, size_t v_len) |
{
|
double dotp=0, abs1=0, abs2=0; |
for (size_t i= 0; i < v_len; i++, v1++, v2++) |
{
|
float f1= get_float(v1), f2= get_float(v2); |
abs1+= f1 * f1;
|
abs2+= f2 * f2;
|
dotp+= f1 * f2;
|
}
|
return 1 - dotp/sqrt(abs1*abs2); |
}
|
abs1 or abs2 maybe zero, so in this case，1 should be returned
Attachments
Issue Links
- relates to
-
-
- Open
-
Activity
I run ann bench mark with dataset nytimes-256-angular, this is zero data in dataset
| Field | Original Value | New Value |
|---|---|---|
| Assignee | Sergei Golubchik [ serg ] |
| Description |
sql: SELECT VEC_DISTANCE_COSINE(VEC_FROMTEXT('[1,2,3]'), VEC_FROMTEXT('[0,0,0]'));
return value is 0. ``` static double calc_distance_cosine(float *v1, float *v2, size_t v_len) { double dotp=0, abs1=0, abs2=0; for (size_t i= 0; i < v_len; i++, v1++, v2++) { float f1= get_float(v1), f2= get_float(v2); abs1+= f1 * f1; abs2+= f2 * f2; dotp+= f1 * f2; } return 1 - dotp/sqrt(abs1*abs2); } ``` abs1 or abs2 maybe zero, so in this case，1 should be returned |
sql: SELECT VEC_DISTANCE_COSINE(VEC_FROMTEXT('[1,2,3]'), VEC_FROMTEXT('[0,0,0]'));
return value is 0. {code:diff} static double calc_distance_cosine(float *v1, float *v2, size_t v_len) { double dotp=0, abs1=0, abs2=0; for (size_t i= 0; i < v_len; i++, v1++, v2++) { float f1= get_float(v1), f2= get_float(v2); abs1+= f1 * f1; abs2+= f2 * f2; dotp+= f1 * f2; } return 1 - dotp/sqrt(abs1*abs2); } {code} abs1 or abs2 maybe zero, so in this case，1 should be returned |
Right, the operation is undefined, 1 it no better than 0 here.
And, right, ann-benchmarks has 239 such vectors. But, again, because the distance for them isn't defined, one cannot say whether they're close to anything or not. They shouldn't cause bugs thpough — like data corruption, wrong results (for other vectors) or crashes.
| Fix Version/s | N/A [ 14700 ] | |
| Resolution | Won't Fix [ 2 ] | |
| Status | Open [ 1 ] | Closed [ 6 ] |
| Link | This issue relates to MDEV-36317 [ MDEV-36317 ] |
| Description |
sql: SELECT VEC_DISTANCE_COSINE(VEC_FROMTEXT('[1,2,3]'), VEC_FROMTEXT('[0,0,0]'));
return value is 0. {code:diff} static double calc_distance_cosine(float *v1, float *v2, size_t v_len) { double dotp=0, abs1=0, abs2=0; for (size_t i= 0; i < v_len; i++, v1++, v2++) { float f1= get_float(v1), f2= get_float(v2); abs1+= f1 * f1; abs2+= f2 * f2; dotp+= f1 * f2; } return 1 - dotp/sqrt(abs1*abs2); } {code} abs1 or abs2 maybe zero, so in this case，1 should be returned |
sql: SELECT VEC_DISTANCE_COSINE(VEC_FROMTEXT('[1,2,3]'), VEC_FROMTEXT('[0,0,0]'));
return value is 0. {code:c} static double calc_distance_cosine(float *v1, float *v2, size_t v_len) { double dotp=0, abs1=0, abs2=0; for (size_t i= 0; i < v_len; i++, v1++, v2++) { float f1= get_float(v1), f2= get_float(v2); abs1+= f1 * f1; abs2+= f2 * f2; dotp+= f1 * f2; } return 1 - dotp/sqrt(abs1*abs2); } {code} abs1 or abs2 maybe zero, so in this case，1 should be returned |
MDEV-36317 claims that returning a particular value significantly improves the recall on the nytimes-256-angular data set. This will be investigated in MDEV-36317
This operation is undefined, you can't calculate cosine distance when either vectors is the 0 vector.