Case Study: Multi-Tenant AI SaaS Platform

The Problem

A B2B startup is building an AI-powered document analysis platform where each customer uploads their own contracts, and the AI answers questions about them. Customers include competitors who must never see each other's data.

Constraints given in the interview:

• 500 enterprise customers, each with 10,000-100,000 documents
• Absolute data isolation: Customer A's data cannot leak to Customer B
• Shared infrastructure for cost efficiency
• Compliance: SOC 2 Type II, GDPR
• Query latency under 2 seconds

The Interview Question

Solution Architecture

flowchart TB
    subgraph Gateway["API Gateway Layer"]
        REQ[User Request] --> AUTH[Auth Service]
        AUTH --> TENANT_ID[Extract Tenant ID]
        TENANT_ID --> VALIDATE[Validate Tenant Access]
    end

    subgraph Isolation["Tenant Isolation Layer"]
        VALIDATE --> ROUTER{Isolation Strategy}
        ROUTER -->|Small Tenant| SHARED[(Shared Vector DB<br/>Namespace Isolation)]
        ROUTER -->|Large Tenant| DEDICATED[(Dedicated Pod<br/>Physical Isolation)]
    end

    subgraph Query["Query Execution"]
        SHARED --> FILTER[Tenant Filter<br/>WHERE tenant_id = X]
        DEDICATED --> DIRECT[Direct Query]
        FILTER --> LLM[LLM with Tenant Context]
        DIRECT --> LLM
    end

    subgraph Audit["Audit Layer"]
        LLM --> RESPONSE[Response]
        LLM --> LOG[(Audit Log<br/>Immutable)]
    end

Key Design Decisions

The Data Ingestion Pipeline

flowchart LR
    subgraph Upload["Document Upload"]
        DOC[Customer Document] --> VALIDATE[Validate Format]
        VALIDATE --> TAG[Tag with Tenant ID]
    end

    subgraph Process["Processing Pipeline"]
        TAG --> PARSE[Parse Document]
        PARSE --> CHUNK[Chunk + Embed]
        CHUNK --> ENCRYPT[Encrypt Metadata]
    end

    subgraph Store["Storage"]
        ENCRYPT --> VECTOR[(Vector DB)]
        ENCRYPT --> BLOB[(Blob Storage)]
        ENCRYPT --> AUDIT[(Audit Log)]
    end

Critical: The tenant_id is attached at the earliest possible point (upload validation) and travels with the document through every stage. It is not derived or looked up later.

Handling the Compliance Requirements

async def delete_tenant_data(tenant_id: str):
    # 1. Delete from vector DB
    await vector_db.delete(filter={"tenant_id": tenant_id})
    
    # 2. Delete from blob storage
    await blob_storage.delete_prefix(f"tenants/{tenant_id}/")
    
    # 3. Anonymize audit logs (cannot delete for compliance)
    await audit_log.anonymize(tenant_id=tenant_id)
    
    # 4. Generate deletion certificate
    return generate_deletion_certificate(tenant_id)

Cost Analysis (500 Tenants)

Interview Follow-Up Questions

Q: What if a bug in your ORM bypasses the tenant filter?

A: Defense in depth. Even if the ORM fails, the database enforces RLS (Row-Level Security). The query SELECT * FROM documents internally becomes SELECT * FROM documents WHERE tenant_id = current_tenant(). This is enforced at the Postgres level, not the application level.

Q: How do you handle a tenant who wants to export all their data?

A: We provide a data portability API that streams all documents with their embeddings and metadata. The export is triggered by an admin, logged in the audit trail, and delivered to a customer-controlled S3 bucket (not our infrastructure).

Q: What if the LLM hallucinates information from its training data that matches a competitor's confidential info?

A: This is a real risk. We mitigate by: (1) Using only retrieval-grounded generation (the LLM cannot answer without retrieved docs). (2) Filtering outputs for any content that does not trace back to the tenant's uploaded documents. (3) Offering a "private model" tier where we fine-tune a tenant-specific model on their data only.

Key Takeaways for Interviews

1. Multi-tenancy is about layers: never rely on a single isolation mechanism
2. Tiered isolation balances cost and security: not all tenants need dedicated infrastructure
3. Tenant ID must be immutable and early: tag at upload, not at query time
4. Compliance is an architecture concern: design for audit, deletion, and portability from day one

Related chapters: Multi-Tenant RAG Isolation, RBAC for AI Systems