Description

For a vector index created with a large M (specifically M ≥ 128), a ORDER BY VEC_DISTANCE(...) LIMIT k search returns wrong rows — in the extreme case the farthest rows instead of the nearest. The same data and query return correct results with the default M=6 (or any M ≤ 127).

How to repeat

CREATE TABLE t (
id INT PRIMARY KEY,
v VECTOR(1) NOT NULL,
VECTOR INDEX (v) DISTANCE=euclidean M=200
);

– 1-D vectors [1], [2], ... fill the graph densely
DELIMITER //
CREATE PROCEDURE fill_t()
BEGIN
DECLARE i INT DEFAULT 1;
WHILE i <= 10000 DO
INSERT INTO t VALUES (i, VEC_FromText(CONCAT('[', i, ']')));
SET i = i + 1;
END WHILE;
END //
DELIMITER ;
CALL fill_t();
DROP PROCEDURE fill_t;

– the 10 nearest to [1] should be ids 1..10
SELECT id FROM t FORCE INDEX(v)
ORDER BY VEC_DISTANCE_EUCLIDEAN(v, VEC_FromText('[1]'))
LIMIT 10;
– actual: a cluster of far-away ids (e.g. 770..778); expected: 1..10

Re-running with M=6 (or M=127) returns the correct 1..10.

Root cause

The HNSW graph serializes each node's per-layer neighbor list into the neighbors blob as <count><gref><gref>...<count>..., where count is written and read as a single byte:

• write — FVectorNode::save
  *ptr++= (uchar)(neighbors[i].num);
• read — FVectorNode::load_from_record
  size_t grefs= *ptr++;
• blob size
  total_size+= 1 + gref_len() * neighbors[i].num;

But layer 0 allows up to max_neighbors(0) = 2*M neighbors, and the per-index M option is bound to the default_m sysvar whose maximum is 200. So a node can legitimately accumulate up to 400 neighbors — well beyond the 255 a byte can hold.

When neighbors[i].num ≥ 256 the count byte wraps:

• 256 → 0 ⇒ the node is persisted with zero neighbors (isolated in the graph);
• 400 → 144 ⇒ the neighbor list is truncated, and for multi-layer nodes the following <count><gref>... segments are then parsed from the wrong offset.

This breaks graph connectivity, so the greedy descent can no longer leave the entry-point region and returns near-entry-point (i.e. far-from-target) candidates. The threshold is 2*M > 255, i.e. M ≥ 128.