Retrieval Strategies for Natural Language Querying over CSV Data

1. Objective

Can an LLM reliably answer natural-language questions over CSV data without needing the entire dataset or schema pasted into every single prompt? Does narrowing candidates first with cheap retrieval, then handing only the relevant tables to an LLM for confirmation and SQL generation produce accurate and trustworthy results? If yes, where does that approach still break?

2. Dataset

a. Ingestion

A single dataset, TaskFlow Inc., is a synthetic company dataset used to test general-purpose retrieval strategies: 5 CSV files (774 rows total, covering customers, usage and support data) plus 10 Word/PDF documents, all describing a fictional SaaS company. All embeddings in this phase used the all-MiniLM-L6-v2 sentence-embedding model. The dataset supports schema-linking, deliberately seeded with edge cases (nulls, negative values and deceptive names) to stress-test retrieval. The same dataset was then represented through schema cards for schema-linking experiments.

Try it yourself: the dataset, along with its schema cards and benchmark queries are available in this public Google Drive folder.

b. Queries

The TaskFlow phase was evaluated against a 20-question benchmark spanning four query types, from simple lookups to multi-condition analytical questions. The 20-question TaskFlow benchmark was organized into four query groups, each testing a different kind of retrieval demand:

• Group A - Exact Lookup (a single, unambiguous fact, e.g. "What is the MRR of Meridian Logistics?")
• Group B - Analytical (a computed answer that doesn't exist anywhere as literal text, e.g. "Which region has the highest MRR?")
• Group C - Semantic (interpretive questions requiring an understanding of intent rather than a keyword match, e.g. "Which customers are struggling with reporting visibility?")
• Group D - Cross-Document (answers scattered across multiple documents or sections, e.g. "Which employees were involved in the outage?")

This breakdown mattered because a single overall pass rate can hide exactly where a method is strong or weak. Two methods can look nearly identical on paper while failing in completely different places.

3. Methods

Phase 1 — Row- and Document-Level Retrieval

Method 1 — Row-to-Text Embedding

Pipeline:

Every CSV row was converted into a natural-language sentence and embedded alongside document chunks, all in a single vector index. The setup used two collections, one for the 774 CSV-derived sentences, one for 111 document chunks pulled from the 10 Word/PDF files, searched together and merged by similarity, using the all-MiniLM-L6-v2 embedding model throughout.

Results:

Clean single-entity lookups worked well, but 0 of 4 analytical (aggregation) queries passed.

Group C was the clearest problem: a 648-row usage table consistently drowned out a 20-row customer table whenever both existed in the same index, which became the defining issue the rest of the project worked to solve.

Method 2 — SQL + Vector Hybrid

Pipeline:

A rule-based natural-language-to-SQL router was added in front of Method 1's unchanged vector layer, so analytical questions could be answered with SQL instead of embeddings. The router matched each incoming query against known entity and keyword patterns; anything that didn't match a pattern fell through unchanged to the same vector layer used in Method 1.

Results:

Analytical queries went from 0/4 to 4/4, the single largest improvement of this phase. Group C staying flat confirmed the volume-imbalance problem lived specifically in the vector layer, not in the absence of SQL.

Method 3 — Hybrid Chunking

Pipeline:

The 648 usage rows were compressed into 20 customer-level objects, each with two fused vector fields (profile and usage), combined using Reciprocal Rank Fusion. The existing SQL router from Method 2 was kept unchanged and a separate row-level collection was retained only for employee and ticket lookups not represented in the new customer profiles.

Results:

Semantic recall improved measurably. For example, a query with a 3-company ground truth went from a FAIL to a clean PASS without changing the embedding model at all. This confirmed the real bottleneck was data representation, not the embedding model's quality.

Method 4 — CSV → JSON → Embed

Pipeline:

Rows were serialized as JSON objects instead of natural-language sentences, using the same embedding model as Method 1. This was a deliberately narrow test: since JSON only changes how CSV rows are encoded, only the Semantic group's queries could show a difference, every other group routes through SQL or documents, both untouched by this change.

Results:

Performance was worse than Method 1 on every comparable query.

The cause was mechanistic: all-MiniLM-L6-v2 is a sentence-embedding model trained on natural grammar, and JSON's repeated structural boilerplate dilutes the very signal it relies on.

Method 5 — Query Decomposition

Pipeline:

Rule-based detection of compound conditions split a query into sub-queries, retrieved each separately and intersected the results. This was a rule-based simulation of a ReAct-style agent rather than a full LLM loop, built specifically to test whether multi-step retrieval helps on questions with more than one independent condition.

Results:

Correctly solved the one compound-condition query it targeted. The key finding was conceptual: this approach improves reasoning over evidence that has already been retrieved but it cannot recover evidence the retriever never surfaced in the first place.

Phase 2 — Schema-Level (Table) Retrieval

Across all five methods, the consistent lesson was that data representation and retrieval strategy mattered far more than embedding-model quality. That finding motivated a pivot away from row- and document-level retrieval toward schema-level (table) retrieval. Recent work such as RASL, CHESS, DBCopilot, CRUSH4SQL, LinkAlign and Rethinking Schema Linking explores different strategies for narrowing candidate schemas before SQL generation. While these approaches differ in how candidate tables are retrieved and ranked, they all aim to reduce the search space presented to the SQL generation model. The methods in this phase were designed to explore that same retrieval problem.

Method 6 — Whole-Table Embedding Schema Linking

Pipeline:

Schema cards were built for TaskFlow's five CSV tables (customers, employees, product_usage, sales_transactions, support_tickets), plus one additional card summarizing the 10-document corpus as a sixth source. Each card was embedded as a single vector, six in total. Incoming questions were embedded and compared via cosine similarity against all six source vectors, using the same all-MiniLM-L6-v2 model as every other method in this project. The top 3 highest-scoring sources were retrieved per query. Schema-level routing only has to find the right table, not compute the answer.

Results:

Scored with Recall@1 and Recall@3 against the same 20-question TaskFlow benchmark used in Methods 1--5, with ground truth pulled directly from each query's known source. Overall Recall@1 was 0.425 and Recall@3 was 0.850 across all 20 queries.

Method 7 — Column-Level Embedding Schema Linking

Pipeline:

Instead of one vector per table, each source was broken into individual entries and embedded separately. An additional schema was made for the documents source so it could be scored the same way as every real table rather than reverting to whole-table treatment. A table's score was based on its single best-matching entry, following the RASL-style approach and the top 3 highest-scoring tables were retrieved per query.

Results:

Scored against the same 20-question benchmark. Overall, a clear improvement over Method 6 in both metrics. The clearest gain was in Group A (Exact Lookup), where Recall@3 went from 0.667 to a clean 1.00: once a question's specific vocabulary could match a single column or document entry directly, it no longer got outvoted by unrelated content sharing the same table-level paragraph.

Group B (Analytical) was the only exception as Recall@3 actually dropped slightly, because column-level scoring is more sensitive to a single strong distractor column pulling a wrong table into the top 3, where whole-table scoring smoothed that out.

Method 8 — LLM-Based Table Prediction Schema Linking

Pipeline:

An LLM was given all six schema cards in a single prompt and asked to return its top 3 table predictions per question.

Results:

This was the strongest of the three retrieval approaches, achieving an overall Recall@1 of 0.825 and Recall@3 of 0.975, outperforming column-level embedding and whole-table embedding pipelines. At this dataset's small scale, giving the LLM the full schema directly is inexpensive; that advantage narrows sharply at real enterprise scale with thousands of tables, which is exactly why RASL treats embedding-based narrowing as a necessary first pass rather than optional.

To better illustrate the progression across the three schema-linking approaches, the following table summarizes their overall retrieval performance using Recall@1 and Recall@3.

4. Conclusion

Across both phases, the clearest finding was consistent: how data is represented and retrieved matters more than which embedding model is used. Raw row-level embedding suffered badly from volume imbalance between large and small tables; restructuring that representation fixed it without touching the model. The same pattern held in the schema-linking phase: narrowing candidate tables cheaply through embeddings, then handing the final, precise decision to an LLM, produced the strongest and most reliable results, achieving an overall Recall@1 of 0.825 and Recall@3 of 0.975.

This work evaluates only the schema-linking stage of the Text-to-SQL pipeline and does not re-evaluate end-to-end SQL generation. Earlier development included spot verification of SQL outputs, but once the project's focus shifted to comparing schema-linking strategies, end-to-end SQL execution was intentionally excluded to isolate retrieval performance as the primary variable. Evaluating complete Text-to-SQL accuracy would require a separate benchmark that measures both retrieval and SQL generation together.

This is a validated architecture on a small, hand-built dataset and not yet a production system. The next steps before a client-facing build are testing at the real scale RASL is designed for, moving to a production-grade vector database, adding SQL self-correction and calibrating entity-type weighting with labeled data. None of that changes the core result: a two-stage retrieval-then-confirmation pipeline, evaluated on a dataset containing representative edge cases, reliably retrieves the correct tables for a genuinely varied set of natural-language questions.

5. Further Reading

1. Eben, J., Ahmad, A., & Lau, S. (2025). RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL. arXiv:2507.23104
2. Talaei, S., Pourreza, M., Chang, Y.-C., Mirhoseini, A., & Saberi, A. (2024). CHESS: Contextual Harnessing for Efficient SQL Synthesis. arXiv:2405.16755
3. Nahid, M. M. H., Rafiei, D., Zhang, W., & Zhang, Y. (2025). Rethinking Schema Linking: A Context-Aware Bidirectional Retrieval Approach for Text-to-SQL. arXiv:2510.14296
4. Wang, Y., Liu, P., & Yang, X. (2025). LinkAlign: Scalable Schema Linking for Real-World Large-Scale Multi-Database Text-to-SQL. EMNLP 2025. arXiv:2503.18596
5. Kothyari, M., Dhingra, D., Sarawagi, S., & Chakrabarti, S. (2023). CRUSH4SQL: Collective Retrieval Using Schema Hallucination For Text2SQL. EMNLP 2023. arXiv:2311.01173
6. Wang, T., Chen, X., Lin, H., Han, X., Sun, L., Wang, H., & Zeng, Z. (2023). DBCopilot: Natural Language Querying over Massive Databases via Schema Routing. arXiv:2312.03463
7. Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of EMNLP-IJCNLP 2019. Model card: all-MiniLM-L6-v2