01 Retrieval Systems 3 min read 682 words

RAG Fundamentals (Dec 2025)

Retrieval-Augmented Generation (RAG) is the architectural pattern of providing an LLM with external, verifiable context to ground its responses. In late 2025, RAG has evolved from "simple vector search" to a complex, multi-stage reasoning pipeline.

ragretrievalfundamentalsintro

01vs. training

The Core Philosophy: Grounding vs. Training

Aspect	Fine-Tuning	RAG
Knowledge Type	Internalized (Weights)	externalized (Context)
Update Cycle	High Cost (Retraining)	Zero Cost (Update DB)
Attribution	None (Black box)	Explicit (Citations)
Privacy	Hard to "Unlearn"	Easy to filter/delete

2025 Insight: Fine-tuning is for Form (style, tone, syntax); RAG is for Fact (knowledge, data, grounding).

02taxonomy

The 2025 RAG Taxonomy

Production RAG systems are now categorized by their "Agentic Depth":

2.1

1. Naive RAG (Retrieve-then-Generate)

Flow: User Query -> Vector Search -> Top-K -> LLM.
2025 Status: Deprecated for production due to "Retrieval Gap" and low precision.

2.2

2. Advanced RAG (Multi-Stage)

Flow: Query Transformation -> Hybrid Search -> Reranking -> LLM.
Key Nuance: Uses RRF (Reciprocal Rank Fusion) to combine keyword and semantic results.

2.3

3. Agentic RAG (Loop-based)

Flow: Agent analyzes query -> Decides which tools/indices to search -> Evaluates results -> Re-retrieves if info is missing.
Techniques: Self-RAG, Corrective RAG (CRAG).

2.4

4. GraphRAG (Structured context)

Flow: Extract entities/relationships -> Build Knowledge Graph -> Traverse graph to find "connected knowledge."
Win: Solves "Aggregative Questions" (e.g., "Summarize all legal risks across 50 documents").

03hybrid era

RAG vs. 2M Context (The "Hybrid Era")

With context windows like Gemini 1.5 Pro (2M+) and Claude Sonnet 4.5 (1M+), RAG is changing.

In-Context RAG (ICR): For datasets < 50k tokens, we skip the vector DB and put EVERYTHING in the prompt.
Prompt Caching: Makes Long-Context RAG 90% cheaper by caching the "Background Knowledge" on the GPU.

Architectural Decision:

If your corpus is > 100k tokens and dynamic: Use Standard RAG.
If your corpus is < 100k tokens: Use In-Context RAG.

04quality gap

The Retrieval Quality Gap

The "Retrieval Gap" is the #1 cause of RAG failure.

Gap 1: Semantic Mismatch: Query says "fast cars," DB has "Porsche 911." Solved by Embedding Rerankers.
Gap 2: Missing Context: Relevant info is in the DB, but the Retriever missed it. Solved by Hybrid Search.
Gap 3: Lost-in-the-Middle: info is in the prompt, but LLM misses it. Solved by Context Compression.

05questions

Interview Questions

Q: In late 2025, why would you still use RAG if models have 2-million-token contexts?

Strong answer: Three tiers of reasons:

Cost and Latency: Even with prompt caching, re-reading 2M tokens for every new user query is significantly more expensive and has higher TTFT (Time to First Token) than retrieving 5 relevant chunks (approx. 2k tokens).
Freshness: RAG can access real-time APIs (Stock prices, News) which cannot be statically embedded in a context window.
Scale: Enterprise datasets (SharePoint, Terabyte logs) exceed even 2M tokens. RAG serves as the "Filter" to find the relevant 0.01% of data that should go into that high-value context window.

Q: What is "Agentic RAG" and how does it differ from "Advanced RAG"?

Strong answer: Advanced RAG is a deterministic pipeline (Linear: Rewrite -> Search -> Rerank). Agentic RAG is a stochastic loop. In Agentic RAG, the model is given tools to decide how to retrieve. For example, if the agent finds that the retrieved documents are irrelevant, it can decide to "Search Google" or "Query the SQL database" instead. It essentially adds a "Reasoning step" before and after retrieval to ensure the context is sufficient to answer the prompt.

06references

References

Gao et al. "Retrieval-Augmented Generation for LLMs: A Survey" (2024 update)
Microsoft. "From RAG to GraphRAG" (2024)
Google. "Long-context LLMs as Retrievers" (2025)

Next: Chunking Strategies

• summary · added by this rebuild

Key takeaways

Fine-tune for form, retrieve for fact

Weights carry style, tone and syntax; retrieval carries knowledge — and only retrieval gives citations, zero-cost updates and the ability to actually delete a document.

Four rungs of agentic depth

Naive retrieve-then-generate, advanced multi-stage with RRF fusion and reranking, agentic loops such as Self-RAG and CRAG, and GraphRAG for questions that span documents.

Long context does not retire RAG

Re-reading two million tokens per query costs more and raises time-to-first-token than fetching five chunks; retrieval also reaches live APIs and corpora exceeding any window.

Three named retrieval gaps

Semantic mismatch is answered by rerankers, missing context by hybrid search, and lost-in-the-middle by context compression — retrieval quality is the leading cause of RAG failure.