Endee vs Vespa: A Comprehensive Vector Database Benchmark

Vector databases are increasingly at the heart of production AI systems - powering semantic search, recommendation engines, and retrieval-augmented generation (RAG) pipelines. Choosing the right one matters enormously: the wrong choice can mean 3x lower throughput, unpredictable latency spikes under load, or inflated infrastructure costs. This post presents a head-to-head benchmark of Endee and Vespa across dense semantic search workloads, covering throughput (QPS), tail latency (p99), retrieval quality (recall) across quantizations and concurrency levels.

---

1. Goal of the Analysis

This benchmark was designed to answer a specific set of engineering questions:

• Throughput (QPS): Which system handles more queries per second with equivalent hardware?
• Tail Latency (p99): Which system provides tighter, more predictable worst-case response times?
• Recall: Does quantization degrade retrieval quality, and by how much?
• Quantization: How do different precision levels perform across both systems?

---

2. Experimental Conditions

2.1 Benchmarking Tools

Dense vector search was benchmarked using VectorDBBench by Zilliz.

Note: Vespa's support in VectorDBBench originally covered only float32 and binary precision. We extended the VectorDBBench codebase to add support for additional Vespa quantization modes (int8, bfloat16). The extended branch is available at: github.com/endee-io/VectorDBBench/tree/vespa_precision_addition

---

2.2 Server & Client Configuration

Both server and client ran with the following server configuration:

• CPU: 4 Core
• RAM: 16 GB
• OS: Debian GNU/Linux 13 (trixie)
• Storage Type: NVMe SSD(100 GB)
• Network Region: server and client co-located

---

2.3 Software Versions

---

2.4 Datasets

We use the Cohere 1M dataset: 1,000,000(1 million) vectors of 768 dimensions with cosine distance.

---

2.5 HNSW Index Parameters

Both Endee and Vespa use HNSW-based indexing. The following parameters were used consistently:

Vespa uses different parameter nomenclature (target-hits, hnsw.exploreAdditionalHits) but these were tuned to produce equivalent search behaviour to Endee's ef_search=128.

---

2.6 Methodology

• All tests were repeated three times and the best result was recorded to minimize variance from transient system load and let the system warmup.
• Dense benchmarking used VectorDBBench with concurrency and TopK varied independently.

---

3. Comparative Analysis

3.1 Baseline: Comparing all Available Quantizations

We first show a set of comparisons between the different precisions(quantizations) available in Endee and Vespa. Endee's quantizations are designed to have low storage and compute overhead while capturing the most information leading to a high QPS and recall. To read more about it go . Here we keep TopK=30 and client concurrency to 5.

Recall with each precision

QPS with each precision

P99 Latency with each precision

At the baseline configuration (TopK=30, Concurrency=5, 1M Cohere 768D vectors), Endee int8 achieves the highest QPS of 1593 at a recall of 94.66%, while Endee int16 delivers 1368 QPS with 97.33% recall; while Vespa's best was bfloat16 at 860 QPS and 95.01% recall. Vespa's p99 latency ranges between 5.8ms to 6.8ms while Endee's ranges from 2.9ms to 4.0ms, a 50% reduction.

---

3.2 Varying TopK

Next, we will compare different metrics with varying TopK value. We pick the best precision for each vectorDB based on QPS, recall and latency. We base this choice on our empirical observations in the above set of experiments. We choose int16 for Endee and bfloat16 for Vespa.

The following tests vary TopK from 10 to 1000 at fixed client concurrency = 5.

Recall vs TopK

Here we record the observed recall when increasing the TopK value at search time. Endee maintains a 1.9-4.3% higher recall than Vespa across different TopK values.

QPS vs TopK

Here we record the QPS reported by VectorDBBench for different TopK values at search time. Endee maintains a 25-63% higher QPS than Vespa across different TopK values.

---

3.3 Varying concurrency

Next, we will compare different metrics with varying client concurrency value. Here again, we pick the same precisions as above to ease comparisons. Concurrency was varied from 2 to 24 to evaluate parallel query scaling behaviour.

QPS vs Concurrency

As the load from client end increases, Endee maintains smooth throughput saturation curve, saturating at 16 concurrency; while with Vespa we see an abrupt drop in QPS at concurrency=6. Moreover, Endee's QPS is upto 2.6x higher than Vespa's.

Note: To really understand the reason for Vespa's abrupt drop, we drilled deeper and found that this drop is caused by Vespa's TCP connections entering a TIME_WAIT state and exhausting the available connections.

---

Conclusion

Endee consistently outperforms Vespa across the metrics that matter most in production: throughput, tail latency, and retrieval quality.

At baseline, Endee int8 delivers nearly 2x Vespa's QPS while cutting p99 latency by ~50%. Across varying TopK and concurrency, Endee scales predictably, exhibiting a smooth throughput saturation curve.

For teams running RAG pipelines, semantic search, or recommendation systems at scale, the choice has real infrastructure implications: higher QPS means fewer nodes to serve the same load, and lower p99 latency means a better end-user experience without over-provisioning. Endee delivers more queries, at higher recall, with lower and more predictable latency - on identical hardware.

---