Endee vs. TurboVec: A 1M-Vector Benchmark on Throughput, Latency, and Recall

Picking a vector search engine usually comes down to four questions: how many queries can it handle per second, how consistent is latency once real traffic hits it, how accurate are the results, and what hardware footprint does it need to get there. To see where Endee stands against TurboVec - the Rust-based, quantization-first vector index built on Google Research's TurboQuant algorithm - we ran both through an identical workload on a 1-million-vector dataset and recorded everything.

This post walks through the exact setup and the exact numbers, with nothing extrapolated beyond what the benchmark measured.

The short version

• Throughput: Endee served 8.30x to 19.75x more queries per second than TurboVec, depending on TurboVec's configuration and the concurrency level.
• Latency: Endee's p99 latency held flat at 3.3 ms from 1 to 16 concurrent clients. TurboVec's p99 latency ranged from 26.2 ms to 46.2 ms.
• Recall: Endee returned the most accurate results of the three configurations tested - 97.30% recall, versus 94.98% for TurboVec at bit-width=4 and 79.58% at bit-width=2.
• Memory: Endee started with a higher memory footprint at low concurrency, but scaled so much more efficiently that it ended up using less RAM than both TurboVec configurations by concurrency=8.
• Where TurboVec won: index build time. TurboVec was ready to serve queries in ~75–78 seconds; Endee's HNSW graph took ~478.7 seconds (about 8 minutes) to build. More on why, below.

Benchmark setup

Both engines were benchmarked with VectorDBBench on identical infrastructure, using the same 1-million-vector Cohere dataset (768 dimensions) and the same topK of 30.

Endee was configured as an HNSW index with m=16, ef_construction=128, ef_search=128, and int16 precision.

TurboVec was tested at two quantization levels: bit-width=4 and bit-width=2. Bit-width controls how aggressively TurboVec compresses each vector - lower bit-width means a smaller memory footprint but a coarser approximation, which shows up directly in the recall numbers later in this post.

Each engine was tested across a range of concurrent client counts. Endee was tested at concurrency 1, 2, 4, 8, and 16. TurboVec was tested at concurrency 1, 2, 4, 8, and 10. Every direct comparison below uses the four concurrency levels both engines share - 1, 2, 4, and 8 - and each engine's highest individually-tested point is called out separately so nothing gets compared apples-to-oranges.

Query throughput (QPS)

At every matched concurrency level, Endee outperformed TurboVec by a wide margin - averaging 18.15x the throughput of the bit-width=4 configuration and 9.71x the throughput of the bit-width=2 configuration. The advantage held remarkably steady across the whole range: it dips slightly around concurrency=4 before widening again at concurrency=8, but never falls below 8.30x even at its narrowest point.

For the concurrency levels tested only on one engine: Endee's own peak across every level it was tested at - including concurrency=16 - was still concurrency=8 (1,691.21 QPS, versus 1,655.33 at concurrency=16). TurboVec followed the same pattern on both configurations: bit-width=4 peaked at concurrency=8 (88.31 QPS, versus 82.60 at concurrency=10), and bit-width=2 also peaked at concurrency=8 (168.82 QPS, versus 166.05 at concurrency=10). All three configurations hit their ceiling at the same concurrency level on this hardware, then flattened or dipped slightly beyond it.

Query latency (p99)

Endee's p99 latency didn't move - 3.3 ms at every concurrency level tested, including concurrency=16. That flatness matters as much as the absolute number: a p99 that doesn't shift under load is what makes latency budgets predictable in production.

TurboVec's bit-width=4 configuration was also fairly stable (45.3–46.2 ms), just consistently higher - Endee's p99 was 13.73x to 14.00x lower across the matched concurrency range. TurboVec's bit-width=2 configuration started lower (26.2 ms) but climbed as concurrency increased, reaching 33.1 ms at concurrency=8 - a 26% increase from its own baseline. Endee's advantage over bit-width=2 ranged from 7.94x to 10.03x lower latency, and that advantage actually grew wider at each step, precisely as TurboVec's own latency crept up under load.

Recall

This is the number that gives the throughput and latency results their real meaning: Endee wasn't just faster, it was also more accurate. Its recall was 2.32 percentage points higher than TurboVec's bit-width=4 configuration and 17.72 percentage points higher than bit-width=2.

TurboVec's own two configurations show the classic quantization trade-off in action. Dropping from 4-bit to 2-bit compression cut TurboVec's own recall by over 15 percentage points (94.98% → 79.58%) - and the payoff for that trade was real: at concurrency=8, the bit-width=2 configuration used 35.9% less RAM than bit-width=4 (a gap that narrows to 20.0% at concurrency=1). Endee's int16 configuration didn't have to make that trade - it posted the highest recall of the three while also being the fastest.

Memory footprint during search

This is the metric with the most interesting shape. At concurrency=1, Endee actually used more RAM than either TurboVec configuration (2.81 GB vs. 1.65 GB and 1.32 GB) - a higher starting cost, likely reflecting the graph structure HNSW keeps resident in memory. But that starting cost barely grows: from concurrency=1 to concurrency=8, Endee's memory footprint grew only 1.27x. TurboVec's grew 4.83x (bit-width=4) and 3.87x (bit-width=2) over the same range.

The crossover happens fast. By concurrency=4, Endee (3.21 GB) is already using less memory than TurboVec's bit-width=4 configuration (4.36 GB). By concurrency=8, Endee is using less memory than both TurboVec configurations - 55.1% less than bit-width=4 and 29.9% less than bit-width=2. Higher-concurrency production workloads are exactly where this matters most, and exactly where Endee's memory profile pulls ahead.

Index load / build time

This is the one metric where TurboVec came out ahead, and by a wide margin - Endee took 6.12x to 6.38x longer to load the same 1M vectors. It's worth being straightforward about why: Endee's configuration builds a full HNSW graph (m=16, ef_construction=128), which means constructing and linking a navigable proximity graph across the entire dataset before the index is queryable. TurboVec's approach is comparatively lightweight to build - quantizing vectors into a compressed representation takes far less upfront computation than growing a graph, which is reflected in the near-identical load times between its two configurations (78.22s vs. 74.98s, a difference of under 5%, since the quantization step itself is cheap regardless of compression level).

That trade-off is real and worth weighing honestly. If an index needs to be rebuilt constantly, TurboVec's faster build time is a genuine advantage. But for the far more common production pattern - build (or update) the index once, then serve queries against it for hours, days, or weeks - an ~8-minute one-time cost is a small price against the throughput, latency, and recall gains shown above.

Why the gap: two different approaches to nearest-neighbor search

The size of the throughput and latency gap comes down to how much work each engine actually does per query.

TurboVec is an in-memory vector library built on Google Research's TurboQuant algorithm. Its indexing pipeline unit-normalizes each vector, applies a random rotation, calibrates per coordinate, and then quantizes per-dimension using Lloyd-Max quantization at a chosen bit-width - 2, 3, or 4 bits, mapping to 4, 8, or 16 quantization buckets respectively - before bit-packing the result. That's an effective way to shrink a vector's footprint. The cost shows up at query time: since the compressed vectors aren't organized into any navigable structure, every query performs an exhaustive linear scan across all N of them, and that work scales linearly with corpus size. For this benchmark's 1M vectors at 768 dimensions, that's roughly N × d ≈ 7.7×10⁸ computations per query, plus the DRAM traffic of streaming the entire compressed corpus through memory on every single request.

Endee's HNSW index (m=16, ef_construction=128, ef_search=128) takes a different approach: it's a navigable proximity graph, so a query only needs to visit roughly 2,000–4,000 vectors to find its nearest neighbors in the same 1M-vector corpus - a 250x to 500x reduction in compute and memory traffic versus an exhaustive scan. That difference in how much work each query actually has to do is the direct explanation for the throughput and latency numbers above.

One more note, for completeness: the exhaustive-scan cost has implications beyond cloud throughput. On passively cooled edge or mobile hardware - a deployment target for in-memory libraries like TurboVec - that same per-query compute and memory-bandwidth load can raise power draw enough to trigger thermal throttling. That's not a factor on the actively-cooled AWS instance used for this benchmark, but it's relevant if TurboVec is being evaluated for on-device or edge deployment rather than server-side search.

Takeaways

For any workload where an index is queried far more often than it's rebuilt - which describes most production vector search - this benchmark points in one direction: Endee delivered materially higher throughput, consistently lower and flatter latency, better recall, and a lower memory footprint under load, all on identical hardware and an identical dataset