02 Retrieval Systems 3 min read 661 words
Chunking Strategies (Dec 2025)
Chunking is the process of splitting a document into discrete segments for retrieval. In late 2025, we have moved beyond "blind fixed-size splits" to Structure-Aware & Semantic Segments.
The Retrieval-Context Tension
| Aspect | Small Chunks (100t) | Large Chunks (1000t) |
|---|---|---|
| Precision | High (Exact match) | Low (Diluted) |
| Context | Poor (Broken sentences) | Rich (Surrounding info) |
| Storage | High (More vectors) | Low (Fewer vectors) |
| Latency | Low (Fast search) | High (Heavy retrieval) |
2025 Rule: Smaller is better for finding, but larger is better for thinking. Use Hierarchical Chunking to get both.
Recursive Structure Splitting
Instead of splitting at every 500 characters, we split at logical boundaries:
[Double Newline] > [Single Newline] > [Period] > [Space].
2025 Best Practice: Use Markdown-Aware Splitting. If a document has # headers, ensure the header is prepended to every child chunk to preserve context (Contextual Chunking).
Semantic Chunking (The 2025 Nuance)
Semantic chunking uses an embedding model to detect "topic shifts."
- Split text into individual sentences.
- Group sentences as long as their embedding similarity stays above a threshold (e.g., 0.82).
- If similarity drops, start a new chunk.
Nuance: In 2025, we use Cross-Encoder Segmenters. A tiny model scans the text and predicts a "Separator token" at every semantic break. This is 10x more accurate than cosine-similarity thresholding.
Hierarchical (Parent-Child) Chunking
This is the industry standard for production RAG in 2025.
Process:
- Create "Parent" chunks of 1,500 tokens.
- Sub-divide each parent into 5 "Child" chunks of 300 tokens.
- Index only the children.
- At retrieval, if a child matches, return the full parent context to the LLM.
- Why?: The child is small and easy for the vector DB to match. The parent provides enough context for the LLM to actually reason correctly without "Broken Context" hallucinations.
Content-Specific Strategies
1. Code Chunking
- Strategy: Use AST (Abstract Syntax Tree) parsing.
- Rule: Never split a function mid-body. Keep imports and class declarations with their methods.
2. Table Chunking
- Strategy: Use Markdown formatting for tables.
- 2025 Pattern: "Summarized Tables." Store a natural language summary of the table in the vector DB, but return the full Markdown table to the LLM.
3. PDF/Layout Chunking
- Strategy: Use Vision-Language Model (VLM) pre-processing (e.g., ColPali).
- Nuance: Instead of just text, store embeddings that represent the positional layout of the page, ensuring charts and sidebars don't get mixed into body text.
Interview Questions
Q: Why is fixed-size chunking with overlap problematic for production systems?
Strong answer: Fixed-size chunking is "content-blind." It frequently splits sentences mid-thought, breaks mathematical equations, and separates headers from their descriptive text. While "Overlap" (e.g., 10%) mitigates this by duplicating 10% of text across chunks, it doesn't solve the core issue: the model's attention is forced to reconstruct meaning from fragmented strings. In 2025, we prefer Semantic or Logical Chunking because it ensures each vector represents a "Complete Semantic Unit," leading to significantly higher retrieval precision.
Q: What is "Contextual Retrieval" (the Anthropic pattern)?
Strong answer:
Contextual Retrieval involves prepending a 1-sentence global context to every chunk before embedding it. For example, if a chunk is about "battery life," but it's from a manual for a "2025 Model X Drone," the text [Drone_Model_X_Manual]: is added to the chunk. This ensures that the vector for "battery life" is influenced by the "Drone" context, preventing it from being accidentally retrieved for "phone battery" queries.
References
- Anthropic. "Contextual Retrieval: Improving RAG Accuracy" (2024)
- LlamaIndex. "Advanced Chunking Strategies for RAG" (2025)
- LangChain. "RecursiveCharacterTextSplitter Benchmarks" (2024)
Next: Embedding Models
Key takeaways
01
Small chunks find, large chunks explain
The tension table pairs 100-token precision and cheap search against 1000-token context richness; hierarchical chunking is how the page refuses to make that choice.
02
Index the children, return the parents
1,500-token parents split into five 300-token children; only the children are embedded, and a child hit returns the whole parent so the model reasons on full context.
03
Split on structure before size
Recursive splitting descends double newline, single newline, period, space; markdown-aware splitting prepends the heading to every child chunk so context survives the cut.
04
Content type dictates the splitter
AST parsing keeps functions whole for code, table summaries are embedded while full markdown is returned, and VLM preprocessing preserves PDF layout position.