Executive Summary

A language model is stateless by default. It answers based only on the prompt carried in a single request, and the moment it emits an answer, that conversation vanishes from the model's head. Everything outside the context window is a world that does not exist. So when you follow up with a support chatbot — "So why is that?" — it loses the earlier context and answers off the mark. We often call this "poor memory," but more precisely, the earlier conversation was simply never sent.

This diagnosis matters because it changes the difficulty of the fix entirely. If the problem were "the absence of long-term memory," you would need a heavy system with a vector database, a retrieval pipeline, and extraction logic. But if the problem is "the previous conversation wasn't sent," then simply loading the prior turns into every request solves most of it. The gap Pebblous's support product (internal codename No. 4) faces right now is exactly the latter. A large share of the memory problem is not that long-term memory is missing, but that the words just exchanged were never handed back.

Sending the whole history makes cost grow quadratically

But implement "send the previous conversation along" naively and you soon hit a wall. If you re-send the entire conversation so far on every turn, input tokens balloon with the square of the conversation length — O(n²). At turn 10 you send the prior 9 turns; at turn 50 the prior 49 — so cumulative cost runs away. The alternative is simple: fold older turns into a fixed-size summary and keep only the most recent few turns verbatim. Then cumulative tokens converge to linear, O(n).

The table below estimates that difference with an illustrative model. Taking one support turn (question plus response) as an average of 150 tokens, Method A re-sends the full history every turn, while Method B keeps a fixed 600-token summary plus only the last 10 turns verbatim. This is not a figure from any specific paper — it is a calculation to compare the growth curves of the two approaches.

Illustrative estimate (our own calculation, not a figure from any specific paper). Converted at the Claude Sonnet 5 introductory input price of $2/M, the cumulative cost of a 200-turn conversation is about $6.03 for Method A and about $0.81 for Method B.

The widening gap is clearer as a picture. Below, cumulative input tokens are plotted against conversation length. Method A curves upward; Method B lies close to a gentle straight line.

That summary folding is really a design choice about what to keep and what to discard is something we return to later. For now the point to note is that this method is the lightest solution available — buildable with the standard library alone, no framework required.

The single word "remember" actually lumps together five or six distinct functions. Splitting out what you actually need is what makes the tool choice clear. Conversational AI memory divides broadly along two axes. One is the working memory (context window) held inside the prompt at this very moment; the other is the long-term memory left outside once the conversation ends. Long-term memory splits again into episodic memory, which records when and what happened, and semantic memory, which compresses facts and preferences.

Overlaid on this are three operations for handling memory: what to store, what to retrieve at query time, and what to update or expire over time. The five frameworks stand at different spots on this map. Letta edits the working memory itself; Mem0 attaches a thin layer of semantic memory; Zep accumulates episodic and semantic facts into a graph with a time axis. LangGraph saves and restores snapshots of working memory, and LlamaIndex binds retrieval (RAG) with a conversation buffer.

A big context window does not replace summarization

As of 2026, the context windows of leading models reach 1M tokens, and most of the surcharge for exceeding a threshold has disappeared. So the objection — "can't we just drop the whole conversation in now?" — comes naturally. Below are the window sizes and input prices of representative models.

Sources: Anthropic official announcement (2026-06-30), Morph LLM context window comparison (2026). As of March 2026, Anthropic eliminated the long-context surcharge above 200K.

The catch is that the advertised size and the size you can actually use are different. NVIDIA's RULER reported that across 22 leading models, the usable context comes to only 50–65% of the advertised figure; Adobe's NoLiMa observed that at the 32K-token mark, 11 of 12 models dropped below half of their short-prompt performance. Chroma's context-rot research pointed out that, independent of window size, degradation accelerates once you pass roughly 30K tokens.

The originator of this field is Letta — formerly MemGPT. Its premise is the most elegant. Just as an operating system feigns unlimited memory by paging between limited RAM and ample disk, MemGPT treats the limited context window like RAM and external storage like disk. The model edits the core memory blocks inside its own prompt, and pages overflow information out to an archival store to recall when needed. However long the conversation runs, it maintains its core memory by rewriting it by hand.

The original paper (Packer et al., 2023, arXiv:2310.08560) reported that this approach scored 93.4% on the DMR (Deep Memory Retrieval) benchmark, far ahead of a recursive-summarization baseline at 35.3%. But that comparison must be read carefully. The "recursive summarization" used as the baseline here is precisely the setup the Zep team later criticized as "too easy," and on the same DMR, Zep reproduced 94.4% using full context as the baseline. The evaluation conditions differ, so the two numbers cannot be compared directly. As we will see again later, benchmarks in this field mostly cross wires this way.

As elegant as the premise is, the body is heavy. Letta is not a library but effectively a full-stack server. Its direct dependencies on PyPI reach 118, and it drags along Postgres, migration tooling, and more. Its GitHub stars stand at 23,817 — buzz enough — but its monthly PyPI downloads (about 149K) are 20–450× smaller than the other four. The gap between the buzz and the actual install base captures the tool's character well. The core license is Apache-2.0, so there is no restriction on self-hosting.

In short: the most elegant premise, the heaviest body. The picture of "the model manages its own memory" is appealing, but for a product like No. 4 that prioritizes controllability and lightness, it is an overkill tool at this stage.

If Letta puts memory inside the model, Mem0 and Zep build memory outside it. Both extract facts from conversations, store them, and retrieve them to attach when needed — but the shape of what they store differs. Mem0 accumulates per-user facts as a thin layer; Zep accumulates them as a knowledge graph with a flow of time.

Mem0 — a thin per-user memory layer

Mem0's picture is simple. Each time a conversation goes back and forth, it extracts facts like "what this user asked before and preferred" and stores them; on the next conversation it retrieves only the relevant ones and lays them thinly onto the prompt. Instead of stuffing in the whole history, it attaches only the fragments this user needs. Its community is also the largest — 60,967 GitHub stars, first among the five, with about 3.19M monthly PyPI downloads.

The numeric claims are flashy. The original paper (Chhikara et al., 2025, arXiv:2504.19413) reported that on LOCOMO it cut tokens per query by about 90% versus full context (26K → about 1,764 tokens) and sharply reduced p95 retrieval latency. A 2026 follow-up blog claims a new algorithm reaches 91.6 on LOCOMO and 93.4 on LongMemEval. But it is worth remembering that these follow-up numbers exist only in a vendor blog, without separate peer review.

Zep — a knowledge graph of time

Zep goes a step further, accumulating the facts pulled from conversations into a knowledge graph with a bi-temporal axis. It records "when this fact was true, and when we came to know it" together. At query time it searches that graph to strengthen the answer — which is, in effect, one branch of graph-augmented retrieval (graph RAG). The original paper (Rasmussen et al., 2025, arXiv:2501.13956) reported 71.2% on LongMemEval (gpt-4o), ahead of full context at 60.2%. The Zep team itself admitted in its own paper that the older DMR benchmark had become too easy for modern models and lost its discriminating power — a rare case of a vendor conceding the saturation of its own benchmark.

It is not uniformly beneficial, either. On the same LongMemEval, most question types improved, but the "single-session assistant" type actually got worse (a drop of about 17.7 percentage points on gpt-4o) — the only category to decline. That means splitting facts into a graph is not advantageous for every question. Third-party reviews also flagged that the paper's latency measurement was asymmetric: the round-trip network latency to the remote Zep server was loaded onto Zep's measurement only, and absent from the full-context baseline. This exception item, if anything, honestly reveals that a vendor's own numbers are easily framed in their favor.

The two tools' performance claims rebut each other. The Mem0 paper put Zep's token usage at 600K+ per conversation, but Zep contested Mem0's benchmark setup (speaker labeling, timestamps inserted into the message text, sequential retrieval) and reported 75.14% on LOCOMO on its own reproduction — a higher number than the Zep score Mem0 published (65.99%). Neither side can be taken as settled. This mutual rebuttal is itself the lesson of this section.

There is also a case that shows the danger of over-optimizing to a benchmark. A maintainer of one memory system corrected a headline "100%" figure to "98.4% of 450 questions," admitting that the last 0.6% was matched by inspecting each wrong answer one by one. Study for the test after peeking at it and the score goes up, but that score does not amount to skill. The moment a vendor tunes a system to its own benchmark, that number drifts further and further from independent verification.

You want graph memory — but Zep is no longer self-hostable

Here we must flag one fact unique to this report. In April 2025, Zep announced on its official blog that it was discontinuing maintenance and updates for Zep Community Edition, its self-hosted open-source offering. The code was moved to the repository's legacy/ folder and is no longer supported. The stated reason: "maintaining a fully open-source community edition and a feature-rich commercial service at the same time forced mutual compromise." Since then, Zep's open-source effort has concentrated on Graphiti (Apache-2.0), its temporal-graph engine.

This change reshapes the terrain. Needing graph-augmented retrieval no longer means you can "install Zep." Instead you provision a graph database like Neo4j, FalkorDB, or Kuzu against the Graphiti engine yourself, and you have to build the multi-tenancy, ingestion pipeline, and observability that Zep used to provide as managed features. There are signs, too, that the self-hosted path is a low maintenance priority. According to community reports, the official Docker image sat on an old version (v0.10) for more than six months while the actual latest release passed v0.22. Symbolically as well, the Graphiti repository (28,799 stars) is more than 6× larger than the Zep repository (4,757 stars) — a signal that people who want self-hosting have already flocked to the engine side.

Zep — one branch of the graph-augmented retrieval angle worth watching — can no longer be bought as a self-hosted option. When the day comes that graph memory is genuinely needed, the candidates are no longer "adopt Zep" but "assemble Graphiti" or "attach Mem0."

Where the previous three specialize in "memory," LangGraph and LlamaIndex are broader, all-in-one frameworks; memory is just one function inside them. LangGraph is strong at saving conversation state as checkpoints and restoring it. It keeps per-thread snapshots of a conversation so it can pick up again from where it stopped. LlamaIndex is strong at binding retrieval-augmented generation (RAG) with a chat memory buffer. It weaves two flows into one — pulling relevant content from a document index and keeping recent conversation in a buffer.

These two are often called "heavy baggage." But that is not a licensing problem. Both cores are MIT-licensed, so there is no restriction on commercial use or self-hosting. The problem is the weight of the dependencies and the way of thinking they impose.

On dependencies, the surface numbers actually mislead. The metapackage's own direct dependencies are 6 for LangGraph and 4 for LlamaIndex — seemingly the lightest. But the real weight hides in the core subpackages beneath: langchain-core and llama-index-core, and the embedding, vector-DB, and LLM-provider adapters they pull in turn. Judge "light" from a single surface dependency count and the actual install tree will betray you.

The adoption metrics illustrate this trap well. LangGraph's monthly PyPI downloads sit at about 66.69M — overwhelmingly first among the five — but that figure heavily includes automatic installs pulled in as transitive dependencies by other packages in the langchain ecosystem, not a count of active users. The proof is that its GitHub-star rank (37,419, second place) diverges from its download rank (first, 20–450× the others). Stars are not usage, and downloads are not usage either.

An all-in-one framework is valuable when you want many functions at once. But if all you need is "continue the conversation state," bringing in an entire worldview for that one thing is an expensive choice.

Place all five tools on one screen and the axis of choice emerges. The vertical axis is weight, the horizontal axis is capability. Heavier means more gets done, but for a stack that treats self-hosting, zero dependencies, and read-only as constraints — as Pebblous does — the candidates you can "just adopt" are fewer than you might think. The column to watch most closely in the table below is self-host. Only Zep is blocked there.

Snapshot 2026-07-16. Dependency counts queried directly from PyPI metadata. Benchmark figures are all vendor self-reported and measured under differing conditions. For LangGraph and LlamaIndex, "N surface" is the metapackage's direct dependencies; the actual install tree is far larger.

Pebblous's No. 4 stack stands on a self-hosted model, the standard library, zero dependencies, and a read-only grounding principle that answers only within the evidence. The constraint looks like a limitation, but it is really a design principle that decides first what not to use. The points where heavy all-in-one frameworks clash with it split three ways. Lump the three together and judgment blurs, so they must be seen separately.

First is the compulsion of dependencies and architecture. The weight of LangGraph and LlamaIndex comes not from licensing but from the actual install tree and the imposed way of thinking. Second is the blockade of self-hosting itself. Zep has had no self-hosted workaround since April 2025. This is a different kind of problem from the first: it is not solved by stripping things away — it is impossible to install in the first place. Third is the cost jump of the managed service.

Seen as a table, the third reveals a pattern: the more managed you go, the harder cost becomes to predict. The jump from free to the first paid tier is especially large.

Snapshot 2026-07, per official pricing pages. For LangGraph and LlamaIndex the core framework is free and the paid tiers are only observability and managed deployment. Zep, by contrast, has no self-host workaround, so there is effectively no way to avoid the managed charge.

So No. 4's next sprint keeps its ambitions small. Start with what breaks today, fix exactly that much, and no more. The direction is already clear.

Priority 1 — history plus summary folding, no framework

What is needed right now is not a long-term memory system but a thin layer that loads conversation history into every request. The implementation can be sketched not as code but as three policies. First, set a token budget. Cap what you spend on context, and set it well below the 30K tokens where context rot begins. Second, keep the most recent few turns verbatim with a sliding window. Third, fold the older turns that exceed the budget into a single fixed-size summary block. All three can be built with the standard library alone, honoring the zero-dependency principle.

This choice is also consistent with practitioner observation. In a study comparing a fixed token budget and a sliding window against various update policies, re-filling the entire context at every step had the highest accuracy but also the highest latency. The sliding window, by contrast, cut latency sharply with little quality loss, sitting closest to the quality-vs-cost Pareto frontier. The "pass the history and fold older turns into a summary" that No. 4 is heading toward lands exactly on that point.

Summary folding is not free, though. As summaries accumulate, the nuance of the original context gets subtly distorted — context drift. That is why the design of what to keep in the summary and what to discard is itself a data-quality design. A mechanism that periodically checks whether the summary distorts the facts must ride along with it.

Priority 2 — attach Graphiti or Mem0 alongside when it's needed

The day may come when per-customer long-term memory or graphing of support records is genuinely needed. When it does, the substantive move is not a wholesale replacement but attaching a proven tool alongside. If it is thinly accumulating per-user facts, use Mem0; if it needs a graph with a flow of time, assemble a graph database against Graphiti. The Zep brand, blocked from self-hosting, drops from the candidates. But this stage brings new risks.

Why Pebblous Cares

This report is not an abstract survey but a blueprint for the next sprint of a real product. Here are four angles on why Pebblous measures the memory-framework landscape this honestly.

Business and technical connection

Pebblous's support product No. 4 is a real product built on a self-hosted model, the standard library, and a zero-dependency stack, and this report's conclusion (pass the history, fold into summaries) points directly at that product's very next improvement. It is an extension of the "only as much as you need, and verifiably" product philosophy shared by the DataClinic and AI-Ready Data lines. Instead of reaching for the strongest tool in the catalog first, we fill what this product actually lacks today.

The data-quality view

Long-term memory is essentially a data-quality problem — because it is a question of what to store as fact and when to update or expire it. A wrongly stored memory pollutes grounding and hardens hallucination from a one-off into something long-lived. In summary folding, the design of "what to keep and what to discard" is itself a data-quality design; and in graphing, so is "which facts to promote to nodes and when to expire them." What DataClinic does for training data must also be done for the memory layer.

Practical implications for customers and partners

Support records mix in personal data. Long-term memory and graphing tie directly to privacy, retention policy, and the right to erasure (the right to be forgotten). The moment you stream support data into a managed memory service, questions of data sovereignty and regulatory compliance arise. Zep in particular now has no self-host workaround, so "to use graph memory you must send data to the cloud" becomes the default. When an enterprise customer evaluates adopting long-term memory, laying out these conditions and risks clearly is itself of direct value to their operational decision.

Pebblous's positioning

The very posture of this report is the positioning. Not a wholesale framework adoption, but minimal intervention that respects the constraints. It reveals a distinctly Pebblous balance point — keeping a read-only principle that answers only within the evidence, while still making the conversation continue. Unlike outlets that list vendor catalogs, Pebblous shares "the result of measuring honestly against our own constraints." That restrained engineering posture is itself the basis of trust.

References

Academic (arXiv)

• 1.Packer, C. et al. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560.
• 2.Chhikara, P. et al. (2025). Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413 (ECAI 2025).
• 3.Rasmussen, P. et al. (2025). Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956.
• 4.Wu, D. et al. (2025). LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. arXiv:2410.10813 (ICLR 2025).

Vendor announcements & blogs

• 5.Mem0 (2026). State of AI Agent Memory 2026.
• 6.Mem0 (2026). Introducing the Token-Efficient Memory Algorithm. (LOCOMO 91.6 · LongMemEval 93.4 — vendor self-reported)
• 7.Zep/getzep (2025-04). Announcing a New Direction for Zep's Open Source Strategy. (Official discontinuation of Community Edition)
• 8.Zep/getzep (2025). Zep Is the New State of the Art in Agent Memory.
• 9.Anthropic (2026-06-30). Introducing Claude Sonnet 5.

Policy, statistics, pricing & independent signals

• 10.GitHub REST API — repository metadata queried directly (stars/forks/license/release), 2026-07-16.
• 11.PyPI (pypistats.org) / npm (registry.npmjs.org) download statistics, queried directly 2026-07-16.
• 12.Letta official pricing docs. letta.com/pricing.
• 13.Mem0 official pricing page. mem0.ai/pricing.
• 14.Zep official pricing page. getzep.com/pricing.
• 15.LangChain official pricing page (LangSmith/LangGraph Platform). langchain.com/pricing.
• 16.LlamaIndex official pricing page (LlamaCloud). llamaindex.ai/pricing.
• 17.Morph. LLM Context Window Comparison 2026.
• 18.NVIDIA RULER / Adobe NoLiMa / Chroma "context rot" research — independent studies on effective context and performance degradation.

Related Pebblous articles

• 19.An Agent's Endurance Is Memory, Not Intelligence — the need for memory.
• 20.Whose Memory Is an AI Agent's? — memory governance and data sovereignty.
• 21.The Agent Context Layer — an overview of context infrastructure.