Executive Summary

Four numbers fix the size of this problem: how many times more innocent data dataset-level deletion swept away in training data; how far information exposure jumps in multi-user systems the moment you attach memory; how much more the internal channels agents use to talk to each other leak than the final output; and how fast the agent adoption widening this attack surface is moving.

For a long time, the default nature of an AI chatbot was forgetting. Close the window and the conversation you'd just had was gone; open it again and the other side greeted you as a stranger. In 2026 that default quietly flipped. As tools that persist information beyond the session — Anthropic's Claude Memory Tool, OpenAI's ChatGPT Memory, Google Gemini's personalization — arrived one after another in real workflows, agents now keep their observations about you in a file even after the conversation ends. Next time you meet, they read that file first. Not-remembering, once the default, has been replaced by remembering.

Technically, this memory is usually a human-readable text file. During a conversation, the agent writes whatever it judges "likely to help next time" into a memory file with a name like session-context.md. That can include your name, your affiliation, the projects you're working on, your preferences and habits — sometimes even sensitive context like health or family. No one dictates line by line what gets kept. The agent decides, and the agent writes. That autonomy is what makes the memory feature convenient, and it is also where the problem this report examines begins.

This is roughly how the memory piles up. Below is a typical example of what an agent leaves in an out-of-session file. Even though the user never explicitly said "save this," personal details mentioned in passing during the conversation have been written down as observations.

- User leads a fintech team in Seoul. Mentioned working late often.
- Recently weighing a job change (avoids naming current employer).
- Spouse managing diabetes — often asks diet-related questions.
- Prefers: conclusion first, prose over tables. Focuses in the morning.

If this were a handful of experimental cases, it wouldn't be worth a report. But the pace of adoption turns it into a real-world problem. Gartner projects that the share of enterprise applications embedding task-specific AI agents will reach 40% by the end of 2026, up from under 5% in 2025 — roughly an eightfold jump in a single year. More agents means the volume of personal data piling up out of session with no name tag grows at the same rate. As the attack surface expands, no one still knows what is being written inside it, or under whose name.

Writing about the privacy of agent memory, the legal and compliance outlet Astraea compressed the problem into a single sentence: "You cannot delete what you do not know the agent stored." Deletion is an operation that begins by identifying its target, and with untagged memory it fails at that very first step. The heart of the problem is not the absence of a delete button, but that no one knows what to press.

Pick the structure apart and it looks like this. A company that deploys an agent is, legally, the data controller. The store is theirs; the file-system access is theirs. In other words, they control the container. But they do not control what the agent wrote inside it, when, or about whom — because the author of the record is not a person but an agent making autonomous judgments. That mismatch — the controller owning the store yet not knowing its contents — is exactly the point at which deletion becomes physically impossible to execute.

Make it concrete. A user requests, "Delete everything you've stored about me." The deployer opens the memory store. Inside, memory files derived from conversations with thousands of users are jumbled together. Which of those sentences belong to this requester? The file only holds an observation like "this user works late often" — no identifier linking it to who the user is. The instant they try to pick out precisely what belongs to the requester, they run into the fact that the information they hold cannot draw that boundary.

The options narrow to two: delete every memory that looks related — damaging other users' records in the process — or, citing the impossibility of identification, effectively fail to carry the deletion out at all. Neither satisfies the original meaning of deletion: erasing exactly that person's share and nothing else. These two options are exactly the shape of the dilemma we already saw in training-data deletion: delete wholesale and you over-delete, don't delete and you fail to comply. The next section takes that parallel up.

This mismatch is separate from whether a product ships a deletion feature. Tools like Claude Memory Tool do offer a memory-delete operation, but the responsibility for implementing that delete handler and deciding what to erase falls on the developers and companies that deploy the tool. So the question is not whether a delete feature exists, but whether the target of deletion can be identified. And that identifiability depends on whether a name tag was left at the moment of recording. If it wasn't, even a well-built delete button has nowhere to aim.

Yesterday's companion piece dealt with deletion in training data. When someone invokes GDPR's right to be forgotten or a copyright withdrawal to demand "remove what I wrote from your training corpus," you have to carve out exactly that person's share from data with hundreds of millions of intermingled lines. Yet the data-versioning tools used in practice usually record provenance at the file or dataset level. So to remove a single contributor with a 1% share, dataset-level processing sweeps away 101× the amount that actually needed to go — the other 99% being innocent data unrelated to the deletion. Provenance tracking that follows author identity all the way down to the token cut that over-deletion to 1.3×.

The runtime memory this piece addresses is a different layer where the same principle breaks down. Training data learned without tags, so deletion over-deletes; memory is written without tags, so it's blocked at the very first step of identifying what to delete. The training side is a "too much gets erased" problem; the memory side is a "can't find what to erase" problem. The symptoms differ, but the root is one and the same — no name tag was left at write time. The diagram below sets the two layers side by side to show how one cause splits into two different failures.

The two layers differ in character, too. Training data is static. The corpus is fixed before training begins, and the deletion problem operates on that fixed mass. Runtime memory, by contrast, is dynamic. New observations keep getting added the whole time the agent runs; they accumulate across many sessions; and sometimes several users' contexts get mixed into one store. So the memory-side deletion problem is not "clean it up once and you're done" — it's the kind of problem you can only handle later if you attach a name tag at every moment a write happens.

This parallel connects causally to another case Pebblous has covered. In the case of AI agent memory ownership in China, we examined the outcome when regulation forced the wholesale deletion of a specific agent's memory. If that was "what regulation erased," this piece takes up the prior question — the technical reason why it can't be erased precisely and safely. The very reason blunt, wholesale deletion becomes the only option is the absence of a name tag.

Memory you can't erase doesn't just sit where it is. In agent systems used by many people at once, memory piled up without a name tag can also drift to the wrong person. A deletion that can't start and a leak look like separate problems, but they share the same root: memory that isn't marked as anyone's can neither be erased nor kept in place. The two benchmarks in this section quantify how large that leakage actually is. One caveat to be clear about, though: these figures are evidence of the size of the leak, not direct proof of the earlier argument that "deletion can't happen." The deletion-can't-start claim is a matter of design and logic; leakage is the risk growing alongside it.

Measure a different place, get a different result

AgentLeak is a benchmark that measures privacy leakage in multi-agent LLM systems channel by channel (1,000 scenarios, 7 channels, 5 production models). Its central finding is that the picture changes completely depending on where the leak happens. Audit only the final output the user sees and you catch just 27.2% of violations; look into the internal message channels agents use to talk to one another and the leakage rate jumps to 68.8%. An output-only audit misses 41.7% of the actual violations — meaning memory that drifted through internal channels without a name tag sits outside the audit net.

The moment you attach memory, the leak surface widens

PiSAs is a benchmark that measures whether information reaches only the appropriate party in multi-user settings — contextual integrity. Its most directly relevant finding for this report is what happens when you attach memory. Add memory to the system and violations don't disappear; they migrate into memory, and the overall exposure-violation rate jumps from 36–47% to 63–90%. The very feature that stores memory widened the leak surface. Even in a single-agent configuration, even the latest models exceeded 77% on inappropriate-information-exposure violations. The moment you switch memory on for convenience, you also open the channel through which that memory can leak out.

The conclusion the two benchmarks point to together is simple. In multi-user systems, memory leaks, and that leakage grows from the same root as the deletion that can't start. Memory not marked as anyone's leaks in audits and slips out of deletion requests alike. The absence of a mark creates both problems at once.

Having diagnosed the problem, one question is natural: couldn't you just instruct the model via the prompt to "not store sensitive information"? The measurements meet that hope only halfway. In PiSAs, prompt-level defenses cut violations by up to 50% in the information-collection stage and the agent-to-agent communication channel, but only 28–29% in the memory channel. Memory is the surface hardest to block with a prompt. An instruction added after the fact, in words, can't erase memory that has already been written. Control holds only through design at write time.

The solution Astraea proposes isn't to make the delete button more sophisticated, but to change the structure of the moment of recording. It has three layers. First, a write wrapper — every time the agent stores memory, it attaches a user identifier saying whose record this is. That is the name tag, i.e. write-time provenance. Second, an external index — a mapping of users to the memory files that belong to them, kept separately outside the store. Third, a programmatic deletion API — when a deletion request arrives, it follows this index straight to that user's original records and erases them. Only with all three layers in place does "delete me" finally become an executable command.

That design can reduce the risk is not conjecture but a measured fact. A prototype sanitizing interceptor built by the AgentLeak team lowered information leaking through internal channels from 31.5% to 2.4%. It isn't free — task success rate dropped 4.7 percentage points as the cost — but the direction is clear. Leakage and deletion alike are controlled when you intervene by design at the junctures of writing and passing information. Not after-the-fact audits or prompts, but structure, is the answer.

Engineering design is compliance

This design isn't an optional virtue but the instrument for meeting incoming regulation. Regulations uniformly impose obligations that hold only if you can trace "where personal data came from and how it was used." California's Automated Decision-Making Technology (ADMT) regulation requires businesses that use automated systems for significant decisions — finance, housing, employment, healthcare — to conduct risk assessments and provide pre-use notice, an opt-out, and consumer access rights, with the key compliance deadline on January 1, 2027. GDPR's right to be forgotten carries top-tier fines for violations (up to 4% of global revenue or €20 million, whichever is greater). South Korea's Personal Information Protection Act has, since March 15, 2024, guaranteed the right to demand an explanation of, and to refuse, fully automated decisions. The EU AI Act's high-risk-system obligations were originally set to apply on August 2, 2026, but the timing is still in flux — the Digital Omnibus discussions are pushing for a delay to December 2, 2027.

Once you get this far, you can see one principle recurring in the same shape across three layers. Pebblous's DataClinic is a tool that diagnoses the lineage and quality of training data. Until now, data quality has mostly been a question about "what you put in" — is it accurate, is it unbiased, is its provenance clear? What this report shows, together with its companion piece, is that quality is also a question about "what you can take out." Lineage and erasability are two sides of the same coin. You can erase only that person's share if you know which data came from whom, and you can respond to the demands of regulation and trust only if you can erase.

Set that principle side by side across layers and it reads like this. In training data, token-level provenance cut over-deletion from 101× to 1.3×. In a model's internal representations, model editing that rewrites facts within the weights (the ROME and MEMIT lineage) is attempted, but it only changes a specific fact; it doesn't solve the provenance question of "who did this knowledge come from." And in runtime memory, write-time tagging is what finally makes deletion possible at all. The techniques differ by layer, but the condition is the same: data without provenance cannot be erased precisely at any layer — in training, inside the model, or at runtime.

For an organization looking to put a memory feature into its product, what this report leaves in practical terms is a single checklist. To make deletion requests executable, do you have all three layers — write wrapper, external index, deletion API? This is the point where the concept of "AI-Ready Data" extends from pre-training data to the agent operations layer, and where the data-quality conversation broadens into "AI-Ready Compliance." Legal outlets treat this problem in the language of advisory counsel, academic benchmarks in the language of measurement. Translating it into the language of data governance — connecting training-data deletion and runtime-memory deletion as one continuum — is where Pebblous stands.

In an age when AI has started remembering me, for "delete me" to hold technically, a name tag saying whose it is must be attached at every moment memory accrues. The name tag can't be added afterward. So whatever memory feature you switch on next, there is one question to ask — does this memory have a name tag? If it doesn't, then the day a deletion request arrives, we simply never had a way to answer it in the first place.

This report is a perspective piece prepared by the Pebblous Data Communication Team on the basis of public materials and academic literature. The figures in the text are cited from benchmarks and regulatory documents with stated sources, as well as from prior Pebblous articles; some items, such as regulatory effective dates, are current as of July 2026 and may change thereafter.

Academic · Benchmarks

• 1.ServiceNow AI Research, Mila, et al. (2026). "PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems." arXiv:2607.05318. Cited only as quantitative evidence of leakage risk.
• 2.El Yagoubi, Badu-Marfo, & Al Mallah. (2026). "AgentLeak: A Full-Stack Benchmark for Privacy Leakage in Multi-Agent LLM Systems." arXiv:2602.11510.

Policy · Statistics

• 3.Astraea Law. "AI Agent Memory Tools: Privacy & Compliance." Central source for the non-deletability thesis and the three-layer solution.
• 4.Gartner. (2025). "40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026, Up From Less Than 5% in 2025." Press release (2025-08-26).
• 5.California Privacy Protection Agency (CPPA). (2025). "Final regulations on ADMT, risk assessments, and cybersecurity audits." Key ADMT compliance deadline 2027-01-01.
• 6.GDPR Art. 17 (right to erasure / right to be forgotten) & Art. 83 (fine tiers). / Republic of Korea, Personal Information Protection Act Art. 37-2 (right to refuse and demand an explanation of automated decisions, effective 2024-03-15).

Prior Pebblous Articles

• 7.Pebblous. (2026). "Token-Level Provenance: Erasing One Person's Share From Training Data" (2026-07-28). Companion piece — the training-weights counterpart, 101× → 1.3×.
• 8.Pebblous. (2026). "AI Agent Memory Ownership in China" (2026-07-07). What regulation erased ↔ this piece covers the technical cause.
• 9.Pebblous. (2026). "Privacy Leakage in Agents' Internal Channels (AgentLeak)" (2026-07-05). The channel-by-channel leakage frame.