Case Study: Autonomous Coding Agent

The Problem

A developer tools company wants to build an AI coding assistant that can autonomously complete multi-file tasks: "Add authentication to this Express API" or "Refactor this module to use dependency injection."

Constraints given in the interview:

• Must work on codebases with 1,000+ files
• Cannot break existing functionality (tests must pass)
• Human must approve changes before commit
• Budget: under $0.50 per task completion

The Interview Question

Solution Architecture

flowchart LR
    subgraph Planning["Planning Phase"]
        TASK[User Task] --> PLANNER[Planner Agent<br/>Claude Sonnet 4.5]
        PLANNER --> PLAN[Task Plan]
    end

    subgraph Execution["Execution Loop"]
        PLAN --> CODER[Coder Agent<br/>Claude Sonnet 4.5]
        CODER --> SANDBOX[E2B Sandbox]
        SANDBOX --> TESTS{Tests Pass?}
        TESTS -->|No| DEBUGGER[Debugger Agent]
        DEBUGGER --> CODER
        TESTS -->|Yes| DIFF[Generate Diff]
    end

    subgraph Review["Human Review"]
        DIFF --> HITL[Human Approval]
        HITL -->|Approved| PR[Create PR]
        HITL -->|Rejected| FEEDBACK[Feedback]
        FEEDBACK --> CODER
    end

Key Design Decisions

The Codebase Understanding Problem

The agent cannot fit 1,000 files into context. We solve this with Tiered Retrieval:

flowchart TB
    QUERY[Agent Query:<br/>'Where is rate limiting configured?']
    
    subgraph L1["Tier 1: Semantic Search"]
        QUERY --> VECTOR[(Vector Index<br/>File Summaries)]
        VECTOR --> TOP10[Top 10 Files]
    end
    
    subgraph L2["Tier 2: Symbol Graph"]
        TOP10 --> SYMBOLS[(Symbol Graph<br/>Functions/Classes)]
        SYMBOLS --> RELEVANT[Relevant Symbols]
    end
    
    subgraph L3["Tier 3: Full Content"]
        RELEVANT --> CONTEXT[Load Full File Content]
        CONTEXT --> AGENT[Agent Context Window]
    end

Implementation:

1. Index file summaries (generated by a smaller model during onboarding)
2. Build a symbol graph using tree-sitter for AST parsing
3. Retrieve in stages: summaries → symbols → full content

The Self-Correction Loop

Agents fail. The key to reliability is structured self-correction:

async def execute_with_retry(task: str, max_attempts: int = 3):
    for attempt in range(max_attempts):
        # Generate code
        code_changes = await coder_agent.generate(task)
        
        # Apply to sandbox
        sandbox.apply_changes(code_changes)
        
        # Run tests
        test_result = await sandbox.run_tests()
        
        if test_result.passed:
            return code_changes
        
        # Feed failure back to agent
        task = f"""
        Previous attempt failed. Error:
        {test_result.error}
        
        Original task: {task}
        
        Fix the issue.
        """
    
    raise MaxRetriesExceeded()

Cost Breakdown

Under budget at $0.21 per task.

Interview Follow-Up Questions

Q: How do you handle tasks that require changes across 20+ files?

A: We break them into sub-tasks during planning. The planner outputs a DAG of changes with dependencies. The executor processes them in topological order, running tests incrementally. If step 5 breaks, we only re-run steps 5+ not the whole task.

Q: What if the agent gets stuck in an infinite retry loop?

A: Three safeguards: (1) Max attempt limit (3). (2) If the same test fails with the same error twice, escalate to human. (3) Total token budget per task ($0.50) triggers termination.

Q: How do you prevent the agent from introducing security vulnerabilities?

A: We run a static analysis tool (Semgrep) in the sandbox as part of the test suite. Security rule violations are treated as test failures and fed back to the agent for correction.

Key Takeaways for Interviews

1. Separate planning from execution for checkpointing and cost control
2. Sandbox all generated code for security (E2B, Docker, etc.)
3. Tiered retrieval solves large codebase scale: summaries → symbols → content
4. Self-correction loops need hard limits: attempts, tokens, time

Related chapters: Tool Use and MCP, Error Handling