Arun Baby

Becoming Unlabelable

14 minute read

“Agents become reliable when they carry an internal model of reality: state, uncertainty, and predictions—not just chat history.”

1. What is a “world model” in agent systems?

A world model is an internal representation an agent uses to answer questions like:

  • “What is true right now?”
  • “What will happen if I take action X?”
  • “What do I still not know?”
  • “Which action is likely to succeed under constraints?”

In other words, it’s the agent’s internal “map” of the environment.

This environment can be:

  • physical (robotics, sensors, navigation)
  • digital (websites, APIs, operating systems)
  • organizational (tickets, workflows, policies)
  • conversational (user intent, preferences, ambiguity)

If you don’t build a world model, your agent tends to behave like a “stateless autocomplete”:

  • it guesses what’s true
  • it repeats actions that already failed
  • it doesn’t track uncertainty or contradictions

World models are the difference between:

  • “I think this will work” and
  • “I predict this will work because state says Y, constraints say Z, and evidence supports it.”

2. The core problem: partial observability

Most real systems are partially observable:

  • Your agent can’t see the entire database; it only sees queried rows.
  • A UI agent can’t see the whole OS; it sees screenshots and window state.
  • A browsing agent can’t know “truth”; it sees pages that may be stale or wrong.

So the agent must maintain a belief state: what it thinks is true, with confidence.

That belief state becomes your world model.

A useful mental model

Think of the agent loop as:

  1. Observe: gather observations (tool outputs, logs, page text, screenshots)
  2. Update belief: incorporate observations into state
  3. Plan: choose an action given belief + constraints
  4. Act: take the action

If you skip step (2), planning becomes unreliable because the agent keeps “planning on vibes.”

3. What goes into a practical world model?

A production world model is rarely “one thing.” It’s a bundle of state components.

3.1 Task state

  • objective
  • progress (what’s done, what’s blocked)
  • budgets (steps, tool calls, cost)
  • stop conditions

3.2 Environment state

Depending on the domain:

  • UI: current screen, clickable elements, cursor position
  • API: last responses, rate limit headers, authentication status
  • Filesystem: what files exist, what changed, where outputs are stored

3.3 Knowledge state (“facts with provenance”)

  • facts extracted from sources
  • source URLs / tool outputs
  • timestamps (“last verified at”)

3.4 Uncertainty state

  • unknown variables
  • conflicting observations
  • confidence per fact

This is what enables good behavior like:

  • asking clarifying questions
  • re-verifying uncertain claims
  • avoiding high-risk actions without sufficient evidence

4. World model representations (three common types)

4.1 Symbolic / structured state (recommended baseline)

This is a JSON-like representation:

{
  "user_intent": "refund_status",
  "order_id": "…",
  "facts": [{"k": "status", "v": "shipped", "source": "api:get_order"}],
  "unknowns": ["delivery_date"],
  "budget": {"steps_left": 5}
}

Pros: debuggable, easy to validate, easy to persist
Cons: manual schema design

If you’re building an agent product, start here.

4.2 Latent / embedding world model

Instead of explicit fields, you maintain a latent vector representation of the environment and retrieve relevant memories via similarity search.

Pros: flexible, handles messy unstructured info
Cons: harder to debug, can hide contradictions

This is useful for long-term memory retrieval, but it’s rarely sufficient alone for high-stakes action selection.

4.3 Learned predictive world model (simulation-capable)

This is a model that can predict “next state” given current state and action.

Pros: supports planning via rollouts
Cons: expensive, hard to train well, can be wrong in subtle ways

This is common in robotics and reinforcement learning, and increasingly appears in digital environments (e.g., “simulate UI transitions”).

4.5 Hybrid world models: what most production agents actually use

In real systems, you rarely pick exactly one representation. A common hybrid setup is:

  • Structured state for critical fields and decisions:
    • budgets, permissions, tool status, user identifiers
  • Retrieval memory (embeddings) for large text corpora:
    • policies, documentation, prior notes
  • Lightweight predictors for “next-step expectations”:
    • rate limit backoff, UI transition guesses, tool success probability

Why hybrid works:

  • structured state keeps you debuggable and safe
  • retrieval memory keeps you scalable
  • predictors keep you efficient (fewer blind retries)

If you try to use only embeddings as your world model, you’ll often struggle with:

  • “did we already do this?”
  • “is this action allowed?”
  • “what is the budget left?”

Those questions want explicit fields, not similarity search.

5. Prediction: why agents need “what happens next?”

Planning requires prediction:

  • If I click this button, will it navigate or open a modal?
  • If I call this API, will it return 403 (auth) or 429 (rate limit)?
  • If I run this migration, will it break compatibility?

The simplest “predictor” can be rule-based:

  • If last call returned 429, next call should backoff.
  • If UI shows “Sign in,” next step must authenticate.

More advanced predictors:

  • a small model that predicts whether an action will succeed
  • a simulator that “dry-runs” changes
  • a verifier that checks preconditions

The key is that prediction is not necessarily “ML.” It’s anything that turns observations into expectations.

5.5 Preconditions and invariants: the simplest predictors you can build

Before doing any action, check preconditions (“is it even legal/possible?”). This is a form of prediction: you’re predicting failure early.

Examples:

  • If auth_state != authenticated, don’t call a privileged API.
  • If rate_limit_state.next_allowed_time > now, don’t call the API yet.
  • If the UI is showing a modal, don’t click elements behind it.

Invariants are similar:

  • budgets never go negative
  • tool calls require schema validation
  • write actions require approval gates

Treat these as hard-coded rules around the world model. They dramatically improve reliability because they prevent the model from improvising unsafe or doomed actions.

6. Uncertainty: the feature that prevents reckless autonomy

Most agent failures are not due to ignorance; they’re due to overconfidence.

World models become safer when uncertainty is explicit.

6.1 Track unknowns

Maintain an unknowns list:

  • missing IDs
  • ambiguous requirements
  • missing evidence for claims

6.2 Track conflicts

Maintain a conflicts list:

  • Source A says X, Source B says Y
  • API returned “success” but UI shows error

6.3 Use confidence gates

Before risky actions, require:

  • confidence above a threshold, or
  • additional verification, or
  • human approval

This is how you make the agent cautious in the right places without being paralyzed everywhere else.

Further reading (optional): for critique/checklist patterns that enforce evidence, see Self-Reflection and Critique.

6.5 Uncertainty in practice: “confidence is a state field, not a feeling”

If you want uncertainty to matter, you must wire it into the system.

Practical technique:

  • every key fact has a confidence and a source
  • decisions have thresholds:
    • “If confidence < 0.7, ask the user”
    • “If confidence < 0.9 and action is write-risky, require approval”

Example:

{
  "facts": {
    "account_balance": {"value": 120.50, "source": "api:get_balance", "confidence": 0.95},
    "currency": {"value": "USD", "source": "ocr:screenshot", "confidence": 0.60}
  }
}

Here, the agent can proceed with balance but should re-verify currency before charging or displaying a final statement.

This is what turns “I’m not sure” into predictable behavior.

7. State estimation: turning noisy observations into stable beliefs

In many environments, observations are noisy:

  • OCR errors in screenshots
  • partial logs
  • flaky APIs
  • inconsistent web pages

State estimation is the discipline of updating beliefs carefully:

7.1 Prefer primary observations over derived guesses

  • “API returned status=SHIPPED” is stronger than “the website looks like shipped.”

7.2 Time matters

Store timestamps:

  • if a fact is old, re-check it

7.3 Don’t overwrite truth with noise

If a fact is high-confidence and a new observation is low-confidence, don’t immediately replace it. Record a conflict and re-verify.

This approach prevents agents from “flip-flopping” between states and wasting budgets.

7.5 Conflict resolution: how to handle two “truths”

Conflicts are inevitable. The worst thing you can do is to silently pick one and continue as if nothing happened.

A better approach:

  1. Record the conflict: what sources disagree, and what they claim.
  2. Rank sources: primary API > official docs > reputable sites > random blog.
  3. Re-verify: fetch a primary source or run an additional tool call.
  4. If still unresolved: ask the user or escalate with a clear explanation.

This is “world model hygiene.” It keeps your system honest and prevents subtle errors from compounding.

8. How world models connect to planning architectures

World models become most powerful when combined with a planning architecture:

The key connection is:

Planning consumes the world model, and execution produces new observations that update the world model.

If the world model isn’t updated, planning doesn’t learn.

8.5 World models for hierarchical planning: what each layer needs

Hierarchical planning works best when each layer reads a different slice of the world model:

  • High-level planner needs:
    • objective, constraints, budgets, and major unknowns
  • Step planner needs:
    • current environment state (tool status, auth, rate limit)
    • evidence gaps for the current chunk
  • Executor needs:
    • the next action and strict tool constraints
  • Verifier needs:
    • evidence, conflicts, and checks to run

This is a performance and reliability win:

  • smaller context → fewer distractions
  • role-specific state → less prompt dilution

Further reading (optional): planning architectures: Autonomous Agent Architectures and Hierarchical Planning.

9. Implementation sketch: a minimal world model in code (pseudocode)

def update_world_model(world: dict, observation: dict) -> dict:
    # observation could come from a tool call: {type, source, payload, timestamp}
    world["observations"].append(observation)

    # Example: update facts if observation is strong
    if observation["type"] == "api_result" and observation["payload"].get("status"):
        world["facts"]["status"] = {
            "value": observation["payload"]["status"],
            "source": observation["source"],
            "ts": observation["timestamp"],
            "confidence": 0.9
        }

    # Example: track unknowns / conflicts
    world = recompute_unknowns(world)
    world = detect_conflicts(world)
    return world

def decide_next_action(world: dict, constraints: list[str]) -> dict:
    if world["unknowns"]:
        return {"type": "ASK_USER", "question": f"I need: {world['unknowns'][0]}"}

    if world["facts"].get("auth_state", {}).get("value") == "unauthenticated":
        return {"type": "TOOL_CALL", "tool": "login", "args": {}}

    return {"type": "STOP", "status": "SUCCESS"}

The point is not this exact code—it’s the separation:

  • observations are raw
  • facts are derived and versioned
  • unknowns/conflicts are explicit
  • decisions are policy-driven

9.5 A minimal “world model schema” you can start with

If you want to implement a world model quickly, start with these top-level fields:

  • observations: append-only log (tool outputs summarized)
  • facts: a dictionary of verified fields
  • unknowns: what you still need
  • conflicts: disagreements you must resolve
  • budget: remaining resources
  • permissions: what actions are allowed

Conceptual schema:

{
  "observations": [],
  "facts": {},
  "unknowns": [],
  "conflicts": [],
  "budget": {"max_steps": 10, "steps_used": 0},
  "permissions": {"allow_writes": false, "allowed_domains": []}
}

This is enough to build useful behavior. You can always extend later.

10. Case study: a UI agent navigating a complex workflow

Environment: a multi-step web UI with popups and redirects.

World model essentials:

  • screen_id or page_state
  • visible_elements (buttons/fields)
  • form_state (values entered)
  • errors (validation messages)
  • auth_state

Why it matters:

  • Without a world model, the agent might re-click “Submit” repeatedly.
  • With a world model, the agent can detect: “Submit was clicked, now waiting for confirmation modal.”

Good behavior comes from predicting transitions:

  • If modal appears, next action is “Confirm”
  • If validation error appears, next action is “Fix field”

This turns UI automation from “random clicking” into “stateful navigation.”

Further reading (optional): if you use screenshots and UI control, see Screenshot Understanding Agents and Computer Use Agents.

10.5 Case study: a browsing agent avoiding stale sources

Environment: the open web (untrusted, sometimes stale).

World model essentials:

  • sources: list of URLs with retrieved timestamps
  • claims: extracted claims with quotes + sources
  • freshness_policy: acceptable update window
  • conflicts: conflicting claims across sources

Good behavior:

  • Prefer sources with recent updates (when the question is time-sensitive).
  • Downgrade or reject claims without quotes.
  • If conflicts exist, either:
    • fetch a primary source, or
    • surface disagreement explicitly instead of guessing.

This turns browsing from “summarize search results” into “evidence-driven reasoning.”

Further reading (optional): Web Browsing Agents.

11. Case study: a tool-using agent managing rate limits

Environment: external API that returns 429 or 503 under load.

World model essentials:

  • rate_limit_state: last headers, retry-after, error rates
  • backoff_policy: next retry time
  • tool_failures: recent failure hashes

Good behavior:

  • After a 429, set next_allowed_time = now + retry_after.
  • Don’t keep the process alive; checkpoint and resume later.

This is a world model because it models the external world’s constraints and predicts what will happen if the agent acts too early.

Further reading (optional): for retry and circuit breaker patterns, see Error Handling and Recovery.

12. Evaluation: how you know your world model helps

World models should improve outcomes, not just add complexity.

Measure:

  • repetition rate: does the agent repeat the same tool call less?
  • success rate under noise: does it succeed more often when tools are flaky?
  • time-to-success: does it converge faster?
  • escalation quality: when it asks the user, are the questions crisp and minimal?

Also measure failure types:

  • fewer “confident but wrong” outputs
  • fewer “stuck in loops”
  • fewer “acted without sufficient evidence”

Further reading (optional): for tracing fields and span-level debugging, see Observability and Tracing.

12.5 Common rookie mistakes (avoid these)

  1. Treating chat history as the world model: chat is noisy and unstructured; you need explicit facts and unknowns.
  2. No provenance: if you don’t store sources for facts, you can’t debug or re-verify.
  3. Overwriting instead of tracking conflicts: silent flips create unstable behavior and wasted tool calls.
  4. No budgets in state: without budgets, agents don’t stop predictably.
  5. Storing huge raw payloads: store references plus summaries, not entire web pages or logs.

13. Summary & Junior Engineer Roadmap

World models make agents reliable because they turn tool outputs into stable beliefs and predictions.

Key takeaways:

  1. Most environments are partially observable: track beliefs, not just outputs.
  2. Represent state explicitly: facts, unknowns, conflicts, budgets.
  3. Track uncertainty: it prevents reckless autonomy.
  4. Update state carefully: don’t overwrite high-confidence truth with noisy observations.
  5. Connect state to planning: planning consumes state; execution produces observations.

Build a small world model for a tool-using agent:

  • Define a JSON schema for facts, unknowns, conflicts, and budget.
  • Run 20 tasks and log:
    • repeated tool calls
    • number of clarifying questions
    • time-to-success

Then add one improvement (conflict detection or a backoff state) and measure if metrics improve.

Further reading (optional)

Related across topics

Jump Game II

24 minute read

“Finding the optimal path through a sequence of choices.”