6 minute read

“Don’t build a generalist. Build a specialist.”

TL;DR

Generic agents fail in high-stakes domains: they cannot reliably audit legal documents or code medical diagnoses. Domain-specific agents use a four-layer vertical stack: RAG for proprietary data, specialized tools for domain APIs, fine-tuning for format and tone, and guardrails for safety. Use RAG for facts (dynamic, groundable) and fine-tuning for style (static, behavioral). The hardest part is evaluation: building a golden dataset with human-expert answers is as difficult as building the agent itself. Human-in-the-loop review is non-negotiable for regulated domains. For the RAG layer, see Retrieval-Augmented Generation and Vector Search for Agents.

A precision Abbe refractometer on a laboratory bench

1. Introduction

Generic agents (ChatGPT) are “Jack of all trades, master of none”.

  • Ask them to write a poem? Great.
  • Ask them to audit a Series A Term Sheet for “Liquidation Preference”? Dangerously vague.

Domain-Specific Agents are the future of SaaS. These agents are constrained to a “Vertical” (Law, Medicine, Finance, Coding). They trade Breadth for Depth and Reliability.


2. Core Concepts: The Vertical Stack

To build a “Harvey.ai” (Legal Agent) or “Devin” (Coding Agent), you need 4 layers of specialization:

  1. Data Layer (RAG): Access to proprietary, non-public documents (e.g., 10k Court Cases).
  2. Tool Layer: Access to specialized APIs (e.g., Westlaw Search, Terminal Access).
  3. Model Layer (Fine-Tuning): Knowing the syntax and “vibe” of the domain.
  4. Guardrail Layer: Strict rules (“Never give financial advice”).

3. Architecture Patterns: RAG vs Fine-Tuning

The biggest debate is: “Do I fine-tune an open model or just use RAG?”

Feature RAG (Retrieval) Fine-Tuning (Training) Hybrid (Best)
Knowledge Source Dynamic (DB lookup) Static (Weights) Both
Hallucination Low (Groundable) Medium Lowest
Adaptability Instant (Update DB) Slow (Retrain) Mixed
Style/Tone Weak Strong Strong

Design Choice:

  • Use RAG for Facts (“What is the termination clause?”).
  • Use Fine-Tuning for Format/Behavior (“Write a memo in the style of a Senior Partner”).

4. Implementation Approaches

4.1 The Medical Scribe Pattern

  1. Input: Doctor-Patient Audio.
  2. Specialized ASR: (Speech).
  3. Extraction Agent: Extracts “Symptoms”, “Meds”.
  4. Coding Agent: Maps Symptoms -> ICD-10 Codes.
    • Tool: lookup_icd10(query).
    • Validation: “Is ‘Chest Pain’ valid for a 5yo?”
  5. Output: EMR Entry.

5. Code Examples: The Hybrid Agent

A simplified implementation using LangChain with a specialized tool and system prompt.

from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain.tools import tool

# 1. Specialized Tool (The "Hands")
@tool
def search_case_law(query: str) -> str:
    """Search the case-law database for precedent. Returns case IDs with summaries."""
    # Mock return; in production this calls your licensed legal research API
    return "Case 12-404: Smith v. Jones established that..."

# 2. Specialized Persona (The "Brain")
# This is where "Prompt Engineering" meets "Domain Expertise"
legal_prompt = """
You remain in character as a Senior Litigation Associate.
- Never invent cases.
- Always cite the Case ID.
- Use 'IRAC' format (Issue, Rule, Application, Conclusion).
- If uncertain, state "Further research required".
"""

# 3. The Agent (LangChain 1.x API)
agent = create_agent(
    model=init_chat_model("openai:gpt-4.1", temperature=0),  # low temperature for rigor
    tools=[search_case_law],
    system_prompt=legal_prompt,
)

# 4. Execution
result = agent.invoke({"messages": [
    {"role": "user", "content": "My landlord evicted me without notice. What are my rights in NY?"}
]})
print(result["messages"][-1].content)

6. Production Considerations

6.1 Evaluation (The “Bar Exam”)

You cannot eval a Legal Agent with “Vibe Check”. You need a Golden Dataset:

  • 100 real legal questions.
  • 100 answers written by human lawyers.
  • Metric: “Fact Recall”, “Citation Accuracy”.
  • Auto-Eval: Use a strong LLM judge to grade the Agent’s answer against the Human answer, and check the judge against a sample of lawyer grades first.

6.2 Liability Firewall

The Agent output should never go directly to the client. It goes to a Human in the Loop. UI Pattern: “Draft generated. [Approve] / [Edit]”.


7. Common Pitfalls

  1. Over-retrieval: Feeding the agent 50 pages of irrelevant case law confuses it.
    • Fix: Ranking/Reranking in RAG pipeline.
  2. Tone mismatch: A medical agent sounding “excited” about a diagnosis.
    • Fix: Fine-tuning on medical datasets reduces the “Customer Service Voice” of base models.

8. Best Practices

  1. Ontology First: Define the domain vocabulary. What exactly does “Churn” mean for this company?
  2. Few-Shot Prompting: Include 5 examples of “Perfect Answers” in the prompt. This guides the style better than instructions.

9. Connections to Other Topics


10. Real-World Examples

  • GitHub Copilot: Launched on OpenAI Codex, a GPT-3 model fine-tuned on public code, and pairs the model with editor context (open files, cursor position) so suggestions fit the codebase.
  • Harvey.ai: OpenAI has described working with Harvey on a custom model trained on case law, built for legal work at large firms.

11. Future Directions

  • Self-Improving Verticals: The agent drafts a contract. The lawyer edits it. The edit is saved. The model is fine-tuned on the edit. The agent gets smarter every day (Data Flywheel).

12. Key Takeaways

  1. Depth over Breadth: A tool that does one thing perfectly is valuable. A tool that does everything okay is a toy.
  2. Hybrid Architecture: RAG + Fine-Tuning + Tools. You usually need all three.
  3. Eval is hard: Building the “Bar Exam” for your agent is as hard as building the agent itself.

Next: Knowledge Graphs for Agents covers how structured facts and multi-hop reasoning make agents more reliable.


FAQ

Should I use RAG or fine-tuning for a domain-specific agent?

Use both. RAG is best for dynamic factual knowledge like “What is the termination clause?” because you can update the database instantly. Fine-tuning is best for format and behavioral style like “Write a memo in the style of a Senior Partner.” The hybrid approach gives you low hallucination from RAG grounding plus domain-appropriate tone from fine-tuning.

How do you evaluate a domain-specific AI agent?

Build a golden dataset of 100+ real domain questions with human-expert answers. Measure Fact Recall and Citation Accuracy. Use a strong LLM as an auto-evaluator to grade the agent’s answers against human baselines, after checking it against a sample of expert grades. Vibe checks are not sufficient for high-stakes domains like law or medicine.

What is the vertical stack for domain-specific agents?

The vertical stack has four layers: Data (RAG with proprietary documents), Tools (specialized APIs like Westlaw or ICD-10 lookup), Model (fine-tuning for domain syntax and tone), and Guardrails (strict rules like “never give financial advice”). Each layer adds specialization that generic agents lack.

How do you handle liability in domain-specific AI agents?

Never let agent output go directly to end users in high-stakes domains. Use a Human-in-the-Loop pattern where the agent generates a draft and a domain expert reviews it. The UI should present “Draft generated: Approve / Edit” rather than treating the output as authoritative.