08 Frameworks & Tools 4 min read 757 words
Framework Selection Guide (March 2026)
The landscape of AI frameworks changed drastically in 2025. This guide provides the Decision Matrix for choosing your stack based on production requirements, team expertise, and system scale.
The 2026 Framework Landscape
| Framework | Tier | Primary Value | Key Weakness |
|---|---|---|---|
| LangGraph | L1 (Core) | Precise state control | Complexity |
| DSPy | L1 (Core) | Reliability & Optimization | Upfront cost (Training) |
| LlamaIndex | L2 (Data) | Advanced Retrieval (RAG) | Logic flexibility |
| CrewAI | L3 (App) | Business process speed | Hides failures |
| Semantic Kernel | L3 (Enterprise) | Microsoft Ecosystem | Python maturity |
| Claude Code | L1 (Coding) | Autonomous CLI coding agent | Requires Anthropic API |
| Cursor / Windsurf | L2 (IDE) | Tight IDE + agent integration | Closed-source infra |
| OpenHands | L2 (Coding) | Open-source autonomous agent | Requires self-hosting |
The Decision Matrix
Use this logic to select your stack:
- Is it a pure RAG app? → LlamaIndex.
- Does it require long-running state/Human-in-the-loop? → LangGraph.
- Is high reliability (99%+) and cross-model portability critical? → DSPy.
- Are you a C#/.NET shop? → Semantic Kernel.
- Are you building high-level automations for business users? → CrewAI + Flows.
- Are you doing autonomous file-system level coding tasks? → Claude Code (CLI) or Cline (VS Code).
- Need open-source coding agent that works with any LLM? → OpenHands (Docker).
- Want the best IDE experience with AI? → Cursor (closed) or Windsurf (Codeium).
Build vs. Buy vs. Framework
As a Staff Engineer, you must resist Framework Bloat.
- Use a Framework when it solves a Non-Trivial Computer Science Problem (e.g., State persistence, Bayesian prompt optimization, Vector-Graph linking).
- Build Custom (Thin Wrapper) when you are just making simple calls to an LLM. Frameworks add latency, update-churn, and debugging overhead that isn't worth it for a single-turn agent.
Anti-Patterns to Avoid
- Framework Tunnelling: Trying to force a complex logic flow into a framework that doesn't support it (e.g., using a pure RAG library for a coding agent).
- The Golden Hammer: Using LangChain just because it's popular, when a 50-line Python script would be faster and cheaper.
- Ignoring Observability: Deploying any framework without an LLOps layer (LangSmith/Phoenix).
Staff-Level Recommendation (March 2026)
For a modern, production-grade agentic system:
- Orchestration: LangGraph (for state and loops).
- Optimization: DSPy (to compile prompts for different model tiers).
- Retrieval: LlamaIndex (for multi-stage RAG).
- Observability: LangSmith (for tracing and evaluation).
- Autonomous coding: Claude Code (CLI) or Cline (VS Code) for file-level editing tasks.
- Open coding agent: OpenHands for self-hosted or CI pipeline integration.
The 2026 insight: Agentic coding tools (Claude Code, Cursor, OpenHands) are not replacements for orchestration frameworks — they are a new category that operates at the file-system level, above the LLM API but below the application logic.
Interview Questions
Q: Why do we see a trend towards "Programming" (DSPy) instead of "Prompting"?
Strong answer: Industrialization. Prompt engineering is "Alchemy": it is inconsistent and does not scale. Programming LLMs via Frameworks like DSPy allows us to treat AI as a Software Engineering discipline. We can apply CI/CD, unit testing (metrics), and automated optimization. This moves AI from "Nondeterministic Magic" to a Predictable Component of a larger distributed system, which is a requirement for any mission-critical production environment.
Q: If you had to build a system that works across OpenAI, Anthropic, and local Llama models, how would you architect it?
Strong answer: I would use DSPy for the prompt layer and LangChain/LangGraph for the orchestration layer. DSPy's Signatures allow me to decouple the task definition from the model's specific "Vibes." I would then use a Universal Model Gateway (like LiteLLM or an internal proxy) to handle the different API formats. This stack ensures that if I need to switch from GPT-4o to Claude Sonnet 4.5 for cost or latency reasons, I do not have to rewrite 50 prompts; I just re-compile or update the config.
References
- Google Cloud. "Enterprise Generative AI Reference Architecture" (2025)
- Gartner. "Magic Quadrant for AI Application Frameworks" (2025)
- Thoughtworks. "Technology Radar: The Rise of Agentic Frameworks" (Nov 2024/2025)
New Chapter: Section 10: Document Processing
Key takeaways
01
Adopt a framework for hard problems only
Reach for one when it solves genuine computer science — state persistence, Bayesian prompt optimisation, vector-graph linking — and write a thin wrapper for single-turn calls.
02
The stack is layered, not chosen once
The recommendation runs LangGraph for state and loops, DSPy for prompt compilation, LlamaIndex for multi-stage retrieval and LangSmith for tracing, rather than one framework for everything.
03
Coding agents are a separate layer
Claude Code, Cursor and OpenHands operate at the file-system level, above the LLM API and below application logic; they do not replace an orchestration framework.
04
Three named framework anti-patterns
Framework tunnelling forces a flow the tool cannot express, the golden hammer reaches for LangChain where 50 lines would do, and shipping without observability blinds debugging.
05
Portability needs signatures plus a gateway
Running across OpenAI, Anthropic and local Llama takes DSPy signatures to decouple task from model, and a gateway such as LiteLLM to normalise API shapes.