Benchmark indexed-vector disk search

[Yujie Zhang (yjiez)]

Summary

• add a benchmark-only search_api selector for legacy and indexed-vectors, defaulting to legacy for existing inputs
• add opt-in public API call latency and exact returned-vector payload metrics without changing the historical legacy hot loop by default
• keep new output fields backward-compatible with older benchmark artifacts and reject regression comparisons that mix API modes
• add paired Load-only, single-L examples that differ only in API selection
• add a manual GitHub Actions mode that runs legacy and indexed-vector APIs in separate processes on the self-hosted NVMe datasets
• document the Box-based collector, its enabled/disabled policy, uncapped capture count, metric semantics, and fresh-process ABBA comparison protocol

This is a test/measurement PR stacked on #1345. It does not change diskann-disk production Rust code beyond the commits already present in #1345.

Collector behavior under measurement

The indexed-vector API uses one enabled/disabled collector path rather than per-mode capture policies:

• legacy search() leaves collection disabled
• search_with_indexed_vectors() enables collection
• traversal capture has no L-based collector capacity
• indexed_vector_cache_entries logs the captured count before final-winner pruning
• final winner retention and missing-vector batch fallback remain unchanged

Online A/B result

Run: https://github.com/microsoft/DiskANN/actions/runs/32323112240

Configuration: Graph search, L=2000, K=100, beam width 4, four threads, and separate Load-only processes.

Dataset	Recall (legacy / indexed)	QPS delta	Mean API latency delta	P95 API latency delta	Peak-memory delta	Returned payload/query
Wikipedia-100K, 768d	99.8846 / 99.8846	-1.88%	+2.61%	+2.89%	+23.00 MiB	307,200 bytes
OpenAI-100K, 1536d	99.9124 / 99.9124	-3.37%	+3.93%	+4.10%	+47.50 MiB	614,400 bytes

The uncapped collector keeps recall and result payload unchanged. Its measured overhead remains in the low single digits. Compared with the earlier L-capped run, process-peak deltas remain effectively similar; cross-run latency differences should be treated as runner variance rather than a controlled direct comparison.

Validation

• cargo check -p diskann-benchmark --features disk-index
• cargo clippy -p diskann-benchmark --features disk-index -- -D warnings
• focused input, aggregation, backward-schema, and regression-selector tests
• both paired examples run in separate release processes with identical recall; indexed payload is exactly 10 * 128 * sizeof(f32) = 5120 bytes/query
• public indexed-vector API verification across Graph, InlineFilter, FlatScan, and DiverseGraph

The full local benchmark suite still encounters an existing Git LFS pointer fixture in flat_search_integration; affected and newly added disk-index tests pass.

🤖 Generated with Claude Code