vector search mhnsw: skip non-dirty nodes when invalidating the shared cache on commit

Description

While benchmarking MariaDB's vector index against pgvector with VectorDBBench (https://github.com/zilliztech/VectorDBBench), I observed that MariaDB's query latency under read/write concurrency is significantly higher than pgvector's, and grows with the volume of data already inserted.

Benchmark: an HNSW index is created at table-creation time, vectors are then inserted continuously, and read bursts are issued at 20% / 50% / 80% of the total insert progress so that reads and writes run concurrently.

Root cause

In MHNSW_Trx::do_commit() (sql/vector_mhnsw.cc), the partial-invalidation path walks every entry in the committing transaction's node cache and clears the shared-cache copy's vec pointer:

for (FVectorNode &from : trx->get_cache())

  if (FVectorNode *node= ctx->find_node(from.gref()))

    node->vec= nullptr;

ctx->start= nullptr;

The trx-local cache, however, contains every node the transaction touched, including the many nodes traversed by search_layer and evaluated by select_neighbors during the insert that were never actually modified. The shared-cache copies of those read-only nodes are still consistent with on-disk data, but they get invalidated anyway, forcing concurrent and subsequent readers to reload them through FVectorNode::load().

There is in fact a pre-existing TODO at the same location:

// also, consider flushing only changed nodes (a flag in the node)

Proposed fix

Add a dirty bit to FVectorNode and set it on the two paths that actually mutate the node:

• in FVectorNode::save() (covers both the newly-inserted target and every neighbor rewritten by update_second_degree_neighbors);
• in mhnsw_invalidate() alongside deleted = true.

The partial-invalidation loop in do_commit then skips entries whose dirty is false. The flag fits in the existing trailing byte alongside stored and deleted, so FVectorNode's footprint is unchanged. The flag is set before any storage-engine call inside save(), so that a write path failing mid-flight still forces the shared cache to reload.

Results

On the 1536D-50K VectorDBBench Streaming run (with an HNSW index created at table-creation time):

• Search latency p95 at 80% insert progress: ~100 ms → ~10 ms
• recall and QPS are unchanged

The fix is local to sql/vector_mhnsw.cc and small (≈10 lines).

I am happy to open a PR if the approach looks acceptable.

Attatchments

1. git format-patch of this change, on branch 11.8

2. Search latency p95 (under read/write concurrency) MariaDB vs pgvector

3. Search latency p95 (under read/write concurrency) after this fix