8 minute read

“Your agent remembers nothing. It re-reads the entire conversation every time it speaks.”

TL;DR

Agent memory is a data structuring problem, not a storage problem. Memori converts dialogue into semantic triples. Its authors at Memori Labs report 81.95% accuracy on LoCoMo at 1,294 context tokens per query, about 5% of the 26,031 tokens a full-context prompt uses, and 67% fewer than Zep. That ratio is where the “20x” comes from: at the same per-token price, the context part of the bill shrinks about twentyfold. The comparison numbers are compiled from other papers, so treat them as indicative. For the foundational memory architecture patterns, see memory architectures.

A library card catalog with thousands of individual index cards compressed into a single compact drawer, warm archival reading room light casting s...

Why is agent memory not just RAG?

Because RAG retrieves. Memory learns.

RAG indexes a static corpus, embeds documents into vector space, and retrieves the most similar chunks when queried. It is stateless: the system does not change based on interactions. It is read-only: new information requires explicit re-indexing. It answers one question: “What does our knowledge base say about X?”

Agent memory answers a different question: “What did I learn about this user across our past 47 conversations?” That requires a write-manage-read loop: not just retrieval but active extraction, structuring, updating, and forgetting. (Forgetting is its own design decision; see what to forget.) When a user mentions they moved from London to Tokyo, memory should update, not append. When a preference expressed in session 12 contradicts one from session 3, memory should resolve the conflict.

The survey on autonomous agent memory (arXiv 2603.07670) formalizes five mechanism families:

  1. Context-resident compression: sliding windows, rolling summaries. Simple but drifts over time.
  2. Retrieval-augmented stores: vector embeddings with ANN search. Better recall, no structure.
  3. Reflective self-improvement: store critiques and observations. Reflexion achieved 91% on HumanEval this way.
  4. Hierarchical virtual context: OS-inspired tiering (MemGPT exemplifies this).
  5. Policy-learned management: memory ops as RL actions. AgeMem learns when to consolidate vs forget.

Most production agents use option 1 (stuff the conversation history into the prompt) or option 2 (embed and retrieve). Both scale badly. Option 1 costs linearly with conversation length. Option 2 loses structure: a vector embedding of “user moved to Tokyo” has no relationship to “user’s address” unless you engineer that connection manually.

What does Memori do differently?

Memori (arXiv 2603.19935, March 2026, written by the team at Memori Labs that sells the product) treats memory as a data structuring problem. Instead of storing conversation text and retrieving similar chunks, it converts unstructured dialogue into two coordinated outputs.

Semantic triples. Subject-predicate-object facts extracted from each conversation turn. “User:4521 → kyc:level → 2” or “User:4521 → preferred_language → Spanish.” Each triple is atomic and queryable, so a changed fact such as a new KYC level is one record to correct rather than a sentence buried in a transcript.

Conversation summaries. High-level narrative capturing user intent and dialogue progression. Each triple is linked to its source summary, preventing “granular facts divorced from context”. You can always trace a fact back to the conversation that produced it.

graph LR
    A[Raw dialogue] --> B[Memori Pipeline]
    B --> C[Semantic Triples<br/>one fact each]
    B --> D[Conversation Summary]
    C --> E[(Database<br/>Queryable KG)]
    D --> E
    E --> F[Query: 1,294 tokens<br/>5% of full-context]

The efficiency gain comes from compression. For illustration, a customer service exchange might run 200 tokens while the useful fact from it, “user wants to upgrade to premium plan”, fits in about 8 tokens as a triple. That is roughly 25:1 in this example, and the saving comes from discarding the pleasantries, confirmations, and filler that make up most conversational text. At query time, Memori retrieves triples with a hybrid of embedding similarity and BM25 keyword matching.

Memori Labs markets the product as a SQL-native memory layer, and the open-source repository (Apache 2.0) supports bringing your own database. The enterprise argument is sound whatever the backend: SQL databases are auditable, durable, transactional, and familiar to every backend engineer. You can query a user’s complete memory profile with a SELECT statement, audit what the agent knows with standard database tooling, and enforce retention policies with DELETE queries.

How does Memori compare to alternatives?

The LoCoMo benchmark (ACL 2024) provides a common evaluation. The 10-conversation set used in these comparisons averages about 600 dialogue turns and 26,000 tokens per conversation, and scoring covers single-hop, multi-hop, temporal, and open-domain questions.

System LoCoMo accuracy Context tokens per query Approach Best for
Memori 81.95% 1,294 Semantic triples + summaries Accuracy + cost efficiency
Zep (Graphiti) 79.09% 3,911 Temporal knowledge graph Temporal reasoning, fact evolution
LangMem 78.05% not reported LangGraph function calls LangGraph-native teams
Mem0 62.47% 1,764 Compression engine Low search latency
Full context 87.52% 26,031 Entire history in prompt Maximum accuracy, maximum cost

Source: the Memori paper. Memori’s scores are the average of three runs with GPT-4.1-mini answering and judging. The baseline accuracies were taken from another paper (Du et al.) and the token counts from Mem0’s paper, not rerun side by side. Under its own judge, Mem0’s paper (arXiv 2504.19413) reports Mem0 at 66.88%, Zep at 65.99% and LangMem at 58.10%, so the same systems move by 10 to 20 points between papers.

On the paper’s numbers, Memori leads on the accuracy-to-cost ratio: 81.95% at 1,294 tokens beats Zep’s 79.09% at 3,911 tokens, and comes within about 5.6 points of full context’s 87.52% while using about 5% of its 26,031 tokens. That 26,031 / 1,294 ratio, about 20x, is the headline number.

Each system has a distinct failure mode. Memori is weakest on open-domain questions (63.54%), and on temporal questions (80.37%) it trails LangMem (86.92%) and Zep (83.33%). Zep: Mem0’s authors reported that retrieval right after ingestion often failed and improved after a delay of several hours, which they attribute to asynchronous background graph construction. Mem0 trades accuracy for speed: 62.47% trails the field in the Memori comparison, while Mem0’s own paper reports the lowest search latency of the systems it tested (0.148 s p50). LangMem is a Python library built around LangGraph, and in Mem0’s benchmark it used very few tokens (127) but had a 17.99 s p50 search latency.

When should you use structured memory versus vector memory?

Structured memory (semantic triples, knowledge graphs) when:

  • Facts need to be updatable (user preferences, account state, relationship status)
  • Audit trails are required (compliance, enterprise governance)
  • Temporal reasoning matters (what was the user’s plan before they changed it?)
  • Token efficiency is a cost constraint (production at scale)

Vector memory (embeddings, ANN retrieval) when:

  • Content is unstructured and diverse (meeting notes, research documents)
  • Similarity search is the primary access pattern (“find conversations like this one”)
  • Integration speed matters more than precision (prototype, MVP)

Both when:

  • The agent needs factual recall (structured) and contextual similarity (vector)
  • Different memory types serve different reasoning steps in the same workflow

The hybrid approach is increasingly common. Memori provides the structured layer. A vector store (Pinecone, Qdrant, Chroma) provides the similarity layer. The LLM queries both and synthesizes.

Key takeaways

  • Memory is not RAG. RAG retrieves from static documents. Memory writes, manages, reads, and updates from conversations. Different problem, different architecture.
  • Structuring is where the savings come from. Memori’s paper reports 1,294 context tokens per query against 26,031 for full context, about 20x fewer.
  • Memori leads on accuracy per token, on its own paper’s numbers. 81.95% at 1,294 tokens. Zep gets 79.09% at 3,911 tokens. Full context gets 87.52% at 26,031 tokens. Baselines were compiled from other papers, and scores shift a lot between evaluations.
  • Database-backed memory is an enterprise feature. Queryable, auditable, durable. Standard database tooling works on the agent’s memory.
  • LoCoMo is the most widely reported benchmark. About 600-turn conversations, four scored question types. Use it as a first filter, then test on your own conversations before production deployment.
  • Temporal and open-domain questions remain hard. Memori scores 87.87% on single-hop facts but 80.37% on temporal and 63.54% on open-domain questions. Zep and LangMem score higher on temporal questions in the same table.

Further reading