Error Handling and Recovery

18 minute read

“Agents that don’t quit: Building resilient AI that can fix itself.”

If your agent just “crashes” every time this happens, it’s useless for production. Reliability in agentic engineering isn’t about preventing errors; it’s about Autonomous Recovery.

In this post, we will explore the different classes of errors, the systematic patterns for self-healing, and the engineering guardrails (like circuit breakers and validation layers) that allow you to deploy agents with confidence. We will also dive into the “Human Escalation” protocol, which is the most critical part of an error handling strategy—knowing when to admit the AI has reached its limit.

2. The Four Classes of Agent Errors: A Taxonomy of Failure

To fix a problem, you must first categorize it. In traditional software, we have stack traces. In AI agents, we have “Cognitive Failures.”

2.1 Syntactic Errors (The “Broken JSON” Problem)

This occurs when the LLM generates output that violates the required structure.

Example: Forgetting a closing quote in a JSON block or adding conversational filler (“Sure, here is your JSON:”) before the actual data.
Engineering Challenge: Your parser throws a JSONDecodeError.
Recovery Strategy: Use Structured Output Mode or an automated Repair Agent that reads the broken JSON and outputs a fixed version.

2.2 Semantic Errors (The “Wrong Tool” Problem)

The output is valid JSON, but the content makes no sense in the context of your application.

Example: Passing a negative number to an “Age” field or calling a tool with a parameter that is technically a string but conceptually a hallucination.
Engineering Challenge: The tool execution fails with a logical error (e.g., “User not found”).
Recovery Strategy: Pass the literal error back to the LLM (Section 7) and ask it to “Reflect and Correct.”

2.3 Environmental Errors (The “Downtime” Problem)

The agent did everything right, but the external world failed.

Example: A 3rd party API is down, a rate limit is reached, or the internet connection is unstable.
Engineering Challenge: You receive a 500 or 429 status code.
Recovery Strategy: Exponential Backoff (Section 3). You must pause the agent and retry later, using your State Management to ensure no progress is lost.

2.4 Intentional Errors (The “Hallucination” Problem)

This is the most dangerous error. The agent thinks it succeeded, but it actually fabricated the answer.

Example: An agent searching for a specific law and “inventing” one that sounds realistic but is 100% fake.
Engineering Challenge: There is no error message. The system thinks everything is fine.
Recovery Strategy: The Validator Agent Pattern. You spin up a second agent whose only job is to “Fact Check” the first agent’s work using a separate source of truth.

3. Systematic Retries: Exponential Backoff for Agents

When an agent hits a rate limit or a connection timeout, don’t just retry in a infinite loop. You will burn tokens and potentially get your IP banned from the API provider.

2.1 The Math of Backoff

In agentic systems, we use the formula: $Wait = Base \times 2^{Attempt} + Jitter$.

Base: Usually 1 second.
Attempt: The current retry count.
Jitter: A random value between 0 and 1000ms.
Why Jitter? If you have 100 agents all hitting a rate limit at the same time, they shouldn’t all retry at the exact same millisecond. Jitter desynchronizes the “Retry Storm.”

2.2 Stateful Retries

Don’t keep the agent process alive while waiting for a 10-minute backoff.

Serialize: Save the agent’s current progress to the DB.
Schedule: Create a “Wake Up” task in a task queue (like Celery or RabbitMQ).
Kill: Terminate the current runner.
Resume: The task queue spawns a new runner when the timer expires.

4. Circuit Breakers for Agents: Preventing the Token Burn

An agent with tools is effectively an agent with a credit card. If an agent gets into an infinite “Search -> Fail -> Retry” loop, it can spend hundreds of dollars in a single hour. Circuit Breakers are hardcoded limits that “Trip” and stop execution when a threshold is exceeded.

4.1 The Step Circuit Breaker

Never allow an agent to run forever. Most production tasks can be completed in under 5-10 turns.

Implementation: Set a max_iterations=10 variable. If the agent reaches step 11, the orchestrator forcibly terminates it and alerts a human.

4.2 The Cost Circuit Breaker

LLM providers charge per token. If you are using a high-end model (like GPT-4o) and the conversation history grows too large, each retry becomes exponentially more expensive.

Implementation: Track the cumulative token cost in your Trace. If total_cost > $2.00, stop the agent.

4.3 The Repetition Detector

If an agent calls the exact same tool with the exact same parameters three times in a row, it is stuck in a logic loop.

Implementation: Keep a hash of the last 3 tool calls. If they match, trip the breaker. This is the AI equivalent of a “StackOverflow” error.

5. Validation Layers: The Pydantic Shield

Catch structured output errors before they ever reach your expensive tools or database.

4.1 The Guardrail Workflow

LLM Generation: The model outputs a JSON string.
Schema Check: Use a library like Pydantic or Zod to validate the types.
Constraint Check: Go beyond types. Check if email is a valid format, or if stock_quantity is greater than zero.
The Error Message: If it fails, don’t just say “Invalid.” Generate a Specific Hint: “Error: ‘price’ must be a positive number, but you provided -50. Please correct this.”

4.2 Handling Partial Validity

Sometimes an agent outputs 10 fields correctly but 1 field fails. Advanced agents use Partial Parsers to save the 10 good fields and only re-ask the agent for the 1 bad one.

6. Fallback Models: Escalating Intelligence

If a cheap model (e.g., Llama-3-8B) fails 3 times, switch the agent’s “Brain” to a more powerful model like Claude 3.5 Sonnet to “Rescue” the task.

7. The “Try-Rewrite-Retry” Pattern (The Feedback Loop)

This is the core of “Self-Correction.” Instead of failing silently, you use the LLM’s reasoning capability to debug its own mistakes.

The Workflow:

Execution: The LLM outputs a tool call: search_db(querry="laptop"). (Error: “querry” is misspelled).
Detection: Your Python code fails to find the function.
Feedback Injection: You inject a new message into the history: “Error: The tool ‘search_db’ does not recognize the parameter ‘querry’. Valid parameters are: [query, limit]. Please fix your call and retry.”
Reflection: The LLM reads the error, realizes its typo, and outputs a corrected call: search_db(query="laptop").

Why this works: Deep learning models are excellent at following specific error logs. By providing the literal error message (and a list of valid options), you narrow the “Search Space” for the model, making the retry almost 100% successful.

8. Idempotency: The Secret to Safe Retries

Avoid double-charging or double-executing.

Generate a unique requestId for every tool call.
The API must check this ID to prevent duplicate actions during a retry.

Junior engineer rule of thumb: if a tool call “writes” (charge a card, create a ticket, send an email), it must be idempotent. If you can’t make the third-party API idempotent, wrap it with your own service that stores requestId -> result and blocks duplicates.

9. Recovering from State Corruption

If an agent has a “False Memory” at step 5, delete the last $N$ messages from its history and re-prompt it with the correct facts. This is “Selective Amnesia” to break a logic deadlock.

10. Case Study: The “DevOps Auto-Repair” Agent

A DevOps agent sees a PodFailed error. It tries to restart it, fails, reads the logs, finds a missing ConfigMap, creates it, and verifies the fix. If it breaks something else, it uses a Rollback Tool.

11. Advanced Pattern: The LLM-as-a-Debugger

Sometimes, you need a more advanced “Surgeon” to fix a problem that a general “Doctor” can’t. If your primary agent (e.g., Llama-3-70B) is stuck in a loop despite multiple retries, you can spin up a Debugger Agent (a more powerful model, like Claude 3.5 Sonnet).

The Workflow:

Primary Failure: The agent has failed 3 times to solve a coding bug.
Context Transfer: The system packs the entire Trace—history, tool results, and error logs—into a “Package.”
Audit: The Debugger Agent reads the package and identifies the hidden logic error (e.g., “The primary agent is ignoring a secondary constraint in the API docs”).
Injection: The Debugger’s advice is appended to the primary agent’s history as a “System Hint.”
Recovery: The primary agent reads the hint and successfully completes the task.

12. High-Availability Agents: Handling Model Timeouts and Latency

In production, LLM APIs are not 100% reliable. They have high latency spikes and occasional 503 errors.

12.1 The “Strict Timeout” Strategy

Don’t let an agent hang for 60 seconds waiting for a response.

The Metric: Monitor TTFT (Time to First Token).
The Threshold: If TTFT > 5 seconds, “Abort” the request.
The Action: Switch to a Backup Model (e.g., from OpenAI to Anthropic) immediately.

12.2 The “Dual-Homing” Pattern

Run two different model providers in parallel for critical tasks. If one fails, the other is already half-finished. This costs 2x more tokens but ensures 99.9% reliability for mission-critical operations.

13. Recovering from Tool Hallucinations

Sometimes an agent gets “Creative” and invents a tool that doesn’t exist, like search_the_dark_web(query="...").

13.1 Fuzzy Matching for Recovery

When your orchestrator receives a call for an unknown tool:

Search: Look for tools with a similar name using Levenshtein distance.
Correct: If search_web is found, send a response: “Error: Tool ‘search_the_dark_web’ does not exist. Did you mean ‘search_web’? Proceeding with correction.”
Auto-Fix: For “High Confidence” matches (>90%), just run the correct tool and tell the agent in the next step.

14. The “Adversarial” Test: Injection of Chaos

Red-team your agent by intentionally returning Internal Server Error from your tools. A resilient agent should handle the failure gracefully.

15. Multi-Agent Error Handling: Cascade Failures

In a squad, an error in Agent A can crash Agent B. Use Input/Output Contracts (Validation Proxies) between agents to prevent error propagation.

16. Post-Mortem Agents: Automated Self-Improvement

After an agent fails, a Post-Mortem Agent identifies the root cause and generates a Pull Request to update the system prompts.

17. Economic Engineering: The Cost of Failure

By using Checkpointing, you only retry the Single Failed Step, reducing recovery costs by 90% compared to restarting the whole task.

18. Deterministic Retries with “Frozen History”

When an agent fails, its history is polluted. Deleting the “Bad” history before retrying prevents the LLM from repeating the same mistake due to self-consistency bias.

19. High-Stakes Recovery: The “Safe Mode” Agent

If a critical error is detected (PII leak), the system disables all tools except notify_human() and pauses for a manual audit.

20. Strategic Defeat: The “Abort” Protocol

The most mature agent is the one that says “I’m stuck.”

If retries > 5, escalate to a human.
Provide the human with a “Debug Summary” of why it failed.

21. Field-Specific Recovery: SQL vs. Search

SQL Error: Return the raw database engine error (e.g., “Table ‘users’ has no column ‘age’“).
Search Error: Return a summary of the 404 or “No results found.”

22. Detailed Debugging Workflow for Junior Engineers

When your agent breaks (and it will), follow this “Resilience Loop”:

Reproduce: Use the seed from the checkpoint to run the exact same prompt again.
Inspect the Trace: Open your observability tool. Look at the Raw Prompt.
Prompt vs. Code: Is the failure because the prompt was vague, or because your Python tool had a bug?
Add a Guard: If it was an edge case, add a Validation Layer (Section 4) to catch it next time.
Audit the Cost: Did the error cost $5 in retries? If so, lower your Circuit Breaker (Section 4) limits.

When agents work with images, errors become visual.

The Failure: The model says “I don’t see any button,” but the button is clearly there.
The Fix: Resolution Scaling. Re-take the screenshot at a higher resolution and retry.
The Fix: Vision Pruning. Use a sub-agent to “Crop” the image to the area where the button should be and ask the Vision LLM to look again.

24. The “Panic Button”: Manual Intervention API

Sometimes, an agent shouldn’t try to fix itself.

The Scenario: An agent is trying to fix a production database error and discovers a “Data Corruption” bug it doesn’t understand. The Recovery:

Safe Mode: Disable all write tools.
UI Alert: Send a Slack notification with a link to the “Time Travel Trace”.
Handoff: The human can manually edit the agent’s state (e.g., delete a wrong fact) and click “Resume.”

25. The Cost of Resilience: Token Overhead

Resilience isn’t free.

Retry Tokens: Every retry sends the entire conversation history again. If the history is 32k tokens, a single retry can cost $0.50.
Optimization: When retrying, use Context Pruning. Don’t send the failed attempt’s messy internal thoughts. Only send the “Action” and the “Error Message.” This can save 80% on retry costs.

26. Future Trends: Self-Healing Neural Architectures

As we move toward “Agentic Native” LLMs, error handling is shifting from hand-coded logic to inherent model capabilities.

26.1 Training on Error Trajectories

We are seeing a rise in datasets specifically designed to teach agents how to recover. Instead of just training on “Perfect Paths,” we train on “Failure Paths” where the first answer is wrong but the agent eventually corrects itself. This allows small models (7B) to achieve recovery rates previously only seen in GPT-4.

26.2 Logic-Constrained Decoding

Emerging libraries like Outlines and Guidance use a technique called “Grammar-Based Sampling.” By modifying the model’s logits during inference, they prevent the model from even generating a character that would violate the JSON schema. This eliminates the “Syntactic Error” category entirely, allowing developers to focus on the much harder “Semantic” and “Intentional” errors.

26.3 The “Memory of Mistakes”

Future state management systems will include a “Negative Memory” buffer—a list of things the agent tried that didn’t work. When the agent is deciding its next move, it will check this buffer to ensure it doesn’t repeat the same failing strategy.

27. Error Handling for Voice Agents (Latency and VAD Errors)

Voice agents introduce a new dimension of failure: Timing.

VAD False Positives: The agent thinks the user started talking (due to background noise) and interrupts them.
The Fix: Implement a “Noise Gate” and a second, fast LLM to classify if the sound was speech or a door slamming.
Latency Spikes: If the ASR (Speech-to-Text) takes $>2$ seconds, the user thinks the agent is dead.
The Fix: “Filler Words.” The system plays a pre-recorded “Umm…” or “Let me check that…” while the main LLM is thinking. This is “Psychological Error Handling.”

28. Security Deep Dive: Sanitizing Error Messages

A common vulnerability in agentic systems is Instruction Leakage via Error Logs.

The Threat: An attacker sends a malicious query that causes a Python error. If you pass the raw Python traceback back to the LLM, the traceback might contain sensitive info like API keys, database paths, or system prompts. The Fix: The “Error Masking” Middleware. Before sending an error back to the agent:

Redact: Remove any strings that look like keys or internal paths.
Summarize: Instead of the full 50-line traceback, only send the “Error Class” and the specific “Field Name.”

29. Building a “Recovery Dashboard”: UI Patterns

As a junior engineer, you shouldn’t just log errors to a text file. You should build a Recovery Dashboard.

Must-Have Features:

Heat Map: Which tools are failing most often? (e.g., Stripe API is failing 5% of the time).
The “Retest” Button: Allow developers to click a button to replay a failed trace with a different model.
Token Burn Alert: A “Panic” meter that glows red if an agent has spent $>$1.00$ in retries in the last minute.

30. Implementation: The “Validator Agent” Prompt

To implement the pattern from Section 2.4, you need a highly adversarial prompt for your validator.

The Validator System Prompt:

“You are a Cold, Cynical Fact-Checker. Your only goal is to find a flaw in the provided Agent Answer.

Check every number against the provided source text.

Flag any sentence that sounds like a guess.

If the answer is 100% correct, output ‘VALID’.

If there is even a tiny mistake, output ‘INVALID’ followed by a detailed list of corrections.”

31. Summary & Junior Engineer Roadmap

Resilience is what turns an AI “Toy” into an AI “Tool.”

Your Roadmap to Mastery:

Project: The Self-Healing API Client. Build a Python wrapper that uses an LLM to automatically fix and retry 400-level API errors.
Project: The Circuit Breaker Dashboard. Create a React UI that shows real-time token spend per agent and allows you to “Kill” a runaway process.
Project: The Pydantic Proxy. Build a middleware that validates JSON outputs between two Llama-3 agents.
Practice: Intentionally break your code. Pass wrong data to your agents and see if they can recover. If they can’t, figure out which of the 30 patterns in this post would have saved them.

Congratulations! You’ve built an agent that doesn’t just work—it Survives. Further reading (optional): If you want to debug failures and token burn at scale, see Observability and Tracing.

32. Double Logic Link: Cycles and Redundancy

In the DSA track, we solve Clone Graph. Handling Cycles is the core challenge. An agent in an error-retry loop is a “Cycle” in a state graph. Your Circuit Breaker (Section 4) is the “Visited Set” that prevents the algorithm from running forever.

In the ML track, we discuss Model Replication. In a distributed system, you don’t trust a single node. In an agentic system, you use Model Diversity (switching models during errors) to ensure that a cognitive bias in one model doesn’t crash the entire mission.

Arun Baby