Executive Summary
This report takes one line of results from AgentCIScope, the agent privacy benchmark released on October 13, 2026, and reads it closely. Researchers from the University of Chicago, Microsoft Research and the University of Illinois Urbana-Champaign put 12 AI agents through 936 executable tasks, and the share of information that leaked split almost fivefold depending on what kind of information it was. Personal preferences leaked at 84.7% and booking details at 76.5%, while credit-card information came in at 17.1% and SSNs, the American counterpart to Korea's resident registration number, at 29.4%. Same models, same tasks, same runs. What differed was not the capability of the model but its default assumption about what deserves protecting.
Where that dividing line comes from is suggested by another chart in the same study. Broken out by governing provision, the clauses violated least often are the ones that carry a named list of sensitive data, such as the special-category article of the GDPR or the American health-privacy statute HIPAA. The clauses violated most often are the unnamed ones, such as the general processing duties of that same regulation. A boundary drawn in law reappears intact in model behavior. The product side looks the same. According to press reports, the items Claude's memory feature says it will never store are government-issued IDs and criminal history, and the items it says it will not store by default are health, race, religion and politics. The items it says it will actively store are communication preferences and working style, which is to say, preferences. Products do not see preferences as something to guard. They see them as something to remember.
Everything above is recorded in the papers and the documents. What follows is this report's own reading. The study never tested what causes the gap, and the only explanation its authors offer is a single sentence: agents are more careful with information that is conventionally recognizable as sensitive. The strongest competing explanation is format. Card numbers have a fixed length and a checksum, so detectors catch them. A table in another study published the same month breaks that explanation. Email addresses, the easiest thing in the world to catch with a regular expression, leaked most of all at 57 to 81%, while card numbers sat at 0 to 4%. A format-based account would put email next to card numbers, not at the far end. That leaves readers with one thing to check. Does the field list we wrote down as protected actually cover the things that leak, and at run time, who is handed that list?
84.7 ↔ 17.1
Leakage rate, preferences ↔ card numbers (%)
Measured per attribute inside the same 936 tasks. Roughly a fivefold gap (§2)
32.5–94.4
Tasks that leaked on a single run (%)
Lowest and highest of the 12 agents, not an average, and measured on tasks chosen to induce leakage (§1)
+3.8–19.3
Added by repeating each task five times (pp)
One clean run proves nothing. Run the same task again and more of it leaks (§2)
93.8–99.3
Task success rate of those same 12 (%)
Capability is bunched into one narrow band while leakage spreads almost threefold (§1)
What the 936 tasks measured
AgentCIScope is a platform and an exam paper at once. It turns public privacy incidents and synthetic cases into agent tasks that actually execute, then records what information travels where while an agent moves between mail, documents and calendars to finish the job. The scale runs like this: 212 disclosure contracts, 936 executable tasks, and 2,771 attributes marked inside them as protected. The attribute taxonomy holds 39 types across 10 categories, and the environments number 11: Gmail, Google Docs, Slack, Salesforce, Atlassian, Calendar, WhatsApp, PayPal, Expedia, Booking and an enterprise system. Every task was run five times, leaving 56,160 execution traces.
Two things have to be laid down before any more numbers. Miss them and every figure that follows reads wrong.
1.1The 936 tasks are only the ones that already made a surrogate agent leak
The first is how the tasks were chosen. The researchers did not simply use their candidate tasks. They stood up a surrogate agent, ran the candidates past it, and kept only the cases that drew leakage out of that surrogate. Their own sentence:
So 32.5 to 94.4% is a ceiling measured under difficulty raised as high as it would go, a different quantity from an everyday incidence rate. The limitations section says as much: the benchmark gave up representativeness and went for difficulty. Nowhere does this report use the figure to mean "agents leak more than half the time." The argument here rests on the relative gap between attributes. That gap is a comparison made inside one and the same task set, so the screening affects it far less, and as §3 shows, a different dataset that went through no such screening produces the same ordering again.
1.2The agent is never handed the list it is supposed to protect
The second point is the axis of this whole report. In this benchmark the agent is never told at run time what counts as protected.
Scoring happens afterwards. Once a task ends, a model acting as judge lays that task's contract beside the execution trace and compares them. The contract spells out the base instruction, the participants, the target environment, the protected attributes and the attributes that may be disclosed. The agent never saw that sheet of paper. What the benchmark measured, then, is not the ability to follow a specification. It is what an agent protects by default when no specification arrives.
▲ Pebblous original diagram (Fig. reinterpretation). It turns §3.3's own sentence — "The CI contract and policies are not provided to the tested agent during execution" — into structure. The agent never saw the contract; it enters the picture only when the judge model compares it against the execution trace after the task ends.
It is tempting to take the result as "preferences leaked because they were not on the protected list." That is simply untrue. The benchmark's taxonomy lists personal preferences, booking details and calendar availability as protected attributes in plain sight. The denominator behind 84.7% is "contract-protected attributes of that type." Preferences were not missing from the list; they leaked while sitting on it. Withhold the list and the agent judges by the list it learned, and the list it learned has no room for preferences, schedules or reservations.
That finishing a task and following the rules are two different things is ground this blog already covered in The Answer Was Right. The Rules Weren't. It stays a premise here rather than a finding. AgentCIScope's table reproduces that premise anyway. Task success sits between 93.8 and 99.3% for all 12, bunched into one narrow band, while the share of tasks that leaked on a single run stretches from 32.5 to 94.4%, almost a threefold spread. The capability dial is already saturated. The compliance dial is still scattered between floor and ceiling.
Preferences 84.7%, card numbers 17.1%
This section is the heart of the report. The abstract puts a number on exactly four attributes: personal preferences 84.7%, booking details 76.5%, SSNs 29.4%, credit-card information 17.1%. The remaining 35 attributes appear only as a heatmap whose cell values cannot be read off, so this report stops at four as well. No full per-attribute table was published. Four is still enough to be sharp, because each of them happens to stand at one end or the other of the same dividing line.
In the bars below, orange marks an item that appears on no statutory list, no enterprise detector catalog and no vendor memory-block list. Gray marks an item named on all three, or most of them. The place where the colors split is the place where the bar lengths split.
▲ Only the four attributes the paper states numerically. The denominator for each type is the number of attributes the contract designated as protected; the numerator is how many of those reached a prohibited external destination. §4 opens the three lists one by one.
84.7% and 32.5 to 94.4% were measured with different rulers. The first counts attributes one by one and asks how many of them leaked. The second is a zero-or-one count at the task level: if any protected attribute leaks anywhere in a task, that whole task is marked as leaked. Put the two in one sentence and the units have to be stated separately. And 17.1% is low, not safe. It means that roughly one in six card-number attributes marked for protection went out the door.
2.1One clean run proves nothing
Repeat the same task five times and more of it leaks. The increase runs from 3.8 to 19.3 percentage points depending on the agent. Percentage points, not percent. The largest jump belongs to Claude Sonnet 5 running on Claude Code: 49.4% on a single run, rising to 68.7% once you count every task that leaked at least once across five tries. The smallest belongs to the Gemini Flash models, which were already leaking above 93% on the first run and had almost no room left to climb.
Below is the full set of 12 from Table 2 of the paper. Lay success rate and leakage rate side by side and the two dials move independently. The highest success rates and the highest leakage rates sit in the same place. Gemini 3.7 Flash, the best at finishing tasks, posts a 99.3% success rate and a 93.7% single-run leakage rate; Gemini 3.8 Flash, second on success at 99.0%, leaks at 94.4%, the worst of the 12. At the other end, Claude Opus 5, which leaks least, is middling on success at 97.1%, and Claude Sonnet 5, the weakest on success at 93.8%, leaks at 49.4%. The expectation that compliance climbs along with capability does not hold up in this table.
| Framework | Model | Task success | Single run | At least once in five |
|---|---|---|---|---|
| Claude Code | Claude Opus 5 | 97.1 | 32.5 | 46.0 |
| Claude Code | Claude Sonnet 5 | 93.8 | 49.4 | 68.7 |
| OpenClaw | GPT-6 Astra | 95.4 | 36.9 | 48.1 |
| OpenClaw | Claude Opus 5 | 96.7 | 43.6 | 58.0 |
| OpenClaw | Kimi K3 | 98.2 | 88.4 | 97.4 |
| OpenAI Agents SDK | GPT-6 Astra | 96.9 | 45.0 | 54.1 |
| OpenAI Agents SDK | Grok 4.6 | 95.5 | 88.1 | 95.7 |
| Google ADK | Gemini 3.1 Pro | 95.4 | 79.3 | 96.4 |
| Google ADK | Grok 4.6 | 95.5 | 87.3 | 95.1 |
| Google ADK | Kimi K3 | 98.2 | 92.5 | 98.5 |
| Google ADK | Gemini 3.7 Flash | 99.3 | 93.7 | 97.5 |
| Google ADK | Gemini 3.8 Flash | 99.0 | 94.4 | 98.2 |
▲ All figures are percentages. Gemini 3.7 Flash, the strongest on task success, sits near the top on leakage as well. Orange marks the lowest or highest value in each column.
Leakage also changes depending on what you mount the model on. Claude Opus 5 leaked at 32.5% on Claude Code and 43.6% on OpenClaw, a gap of 11.1 percentage points. GPT-6 Astra opens a gap of 8.1 points when the framework changes. The authors note that the same backbone can differ by more than 10 percentage points across frameworks. Choosing a model matters, and so does the choice of tooling you drop that model into.
2.2The calendar is the biggest source of leaks
AgentCIScope measures an environment axis too. Treated as the source information flows out of, Calendar has the highest average leakage rate; treated as the destination it flows into, WhatsApp ranks first. Slack, Atlassian and Salesforce follow on the source side, and Google Docs and Calendar follow WhatsApp on the destination side. On this axis, though, only the ordering is legible: the cell values live inside a figure and cannot be read out. So this report says that Calendar is the biggest source of leakage, and never says at what percentage.
The ordering alone is meaningful. What a calendar holds is schedules and reservations, and those two are the attributes that leak most on the attribute axis. Both axes point at the same place. The calendar is the warehouse where structured data nobody ever labeled sensitive piles up in one spot, and that warehouse is the single largest source of leaks.
There is also a survey of what people hand agents first. Menlo Ventures, working with Morning Consult, asked 5,067 US adults, and the share granting access runs: email 36%, web browser 33%, messaging 31%, cloud storage 29%, calendar 27%, health apps 23%, financial accounts 20%. Calendar and messaging are not at the front of that queue. But the last two places are health and money. The two categories people open last are precisely the two named on all three lists: the statutory enumerations, the enterprise detectors and the vendor memory policies. Mail, messaging and calendar, which stand ahead of them in the queue, appear on none of the three. The line drawn in a model's defaults is drawn at the user's fingertips too. These survey figures come secondhand; the primary report was not obtained.
Why card numbers get spared
The study never tested what causes the gap. There is no experiment that renames an attribute and watches the leakage rate move, and none that compares handing the protection list over against withholding it. There is no defense experiment at all. This is measurement-only work. The authors offer a single line of explanation: agents are more careful with information that is "conventionally recognizable as highly sensitive" and show "weaker protection for more general personal information." That is a reported correlation, not a demonstrated cause.
This section therefore does not relay facts. It sets explanations against each other. From here on the argument is this report's own, and the tables it leans on are facts taken from other people's papers.
3.1The strongest objection is format
The most credible objection to this reading goes as follows. Card numbers leak less not because the name sits on a protection list but because the format is regular enough for a machine to pick out. Microsoft Purview's credit-card detector works exactly that way: it looks for a sixteen-digit pattern, confirms the number with a Luhn checksum, and checks whether words like Visa or Mastercard appear nearby. Preferences have no digit count and no checksum. On this account, what creates the gap is detectability.
This is not a weakness to be hidden. The author of PrivDrift, published the same month, reads that paper's own results in exactly this way: protections today are uneven and "depend more on format-sensitive cues than on robust contextual confidentiality." The objection comes with primary-literature backing.
3.2PrivDrift's own Table 5 breaks that explanation
Open Table 5 of that paper, though, and the story changes. PrivDrift ran 1,000 multi-turn conversations with secrets planted in them and measured leakage by secret type. The result:
▲ Table 5 of PrivDrift (arXiv:2609.30094 v2). Each bar spans the lowest and highest value across the three models tested. Unlike AgentCIScope, this study did not screen its cases for leakage, and the ordering came out the same.
An email address is the easiest item of all to catch with a regular expression. Google and Microsoft both ship built-in detectors for it. If format were the dividing line, email would be standing next to card numbers on the barely-leaks side. Instead it is at the opposite end, leaking most of the four types at 57 to 81%. PrivDrift concedes the contrast in its own sentence: SSNs and credit-card numbers are suppressed far more often than emails and phone numbers, "even though all are user-disclosed and contextually private."
The dividing line, then, is not whether a machine can catch it. It is whether the name has long been called sensitive. Email addresses and phone numbers have perfectly crisp formats, yet nobody ever singled them out as sensitive data, and so they are guarded less. To say it again: this contrast is this report's own reading. No paper puts it this way, and no study directly testing the causal claim turned up in this research.
3.3Format-based defenses collapse once the distribution shifts
Even granting the format explanation, the conclusion does not change, because defenses that lean on format turn out to be weaker than expected. A separate study of the robustness of PII detectors found performance collapsing under small shifts in distribution. Credit-card recall for a generative-model detector reads 1.000 on a familiar distribution and drops to 0.543 on an unfamiliar one. Location sits at roughly 0.46 to begin with. That paper's summary is terse: rigid-format entities hold up, while location collapses across every model.
Preferences, schedules and reservations are on the location side of that split. Whether or not the format explanation is right, neither branch leads to the conclusion that detectors can be left to handle this.
The list the law wrote, the list the product wrote
Where did the list a model learned come from? Nobody can open up the training process, but the documents a model is likely to have read can be opened: statutes, enterprise detector specifications, and vendor memory policies. Set the three side by side and, although different organizations wrote them for different purposes, the lists are nearly identical. And the attributes that leak most are absent from all three.
4.1The law writes closed lists
Article 23 of South Korea's Personal Information Protection Act enumerates sensitive data as ideology and belief, joining or leaving a trade union or political party, political opinion, health, and sex life; Article 18 of the Enforcement Decree adds genetic data, criminal records, biometric identifiers, and race or ethnicity. Nine kinds in total. The resident registration number is not among them. It lives in a different drawer, the unique identifying information of Article 24, alongside passport numbers, driver's license numbers and alien registration numbers. As for whether the statute's phrase "information on such matters" could be read broadly, the Ministry of Government Legislation already settled that it must be read narrowly.
Preferences, schedules and reservations therefore do not qualify, not even conditionally. Neither does location history, though not because anyone decided location is unimportant. Korea carved it out into a separate statute, the Location Information Act, and built a protection regime for it there. One kind of sensitivity ends up scattered across different lists, and an organization that copies its internal protected-field list from the statutory enumeration copies that scattering along with it.
Europe enumerates about as narrowly. The special categories of GDPR Article 9 number eight: racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union membership, genetic data, biometric data, health, and sex life and sexual orientation. One extra door stands open, though. In a 2022 ruling, the Court of Justice of the European Union held that processing capable of indirectly revealing sensitive data can also fall under the Article 9 prohibition. If a record of repeated visits to a particular clinic can reveal a health condition, that location history may be pulled into the special categories. Where Korea keeps its list closed, Europe leaves a passage open through inference.
This is where AgentCIScope's third axis arrives. Alongside attribute and environment, the benchmark also measured violation rates by governing provision. Violated least often: HIPAA, the medical-confidentiality provisions of the Americans with Disabilities Act, the Privacy Act, GDPR Article 9, and the Fair Credit Reporting Act. Violated most often: FERPA, the US education-records statute, and GDPR Articles 6, 29 and 32. Every item in the first group carries a named list of sensitive data; every item in the second is an unnamed general duty about lawful basis or security measures. The boundary seen on the attribute axis appears once more on the provision axis.
4.2Vendor memory policies file preferences under "actively store"
Vendor documents are even sharper. Claude's memory feature does not pile up whole transcripts; it extracts facts and preferences and keeps them in structured form. According to press reports of that policy, the items it never stores regardless of settings are government-issued ID numbers, criminal history and immigration status. The items it does not store by default are health, race and ethnicity, religious belief, politics and sexual identity. The first group overlaps almost exactly with Korean law's unique identifying information and criminal records; the second looks like a transcription of the GDPR's special categories.
Then comes a third column: the items the feature says it actively stores. Roles and projects, the people and places in someone's work and life, communication preferences and working style, technology preferences and coding style. All of it is preference data. These are not merely missing from the prohibition list; they are written on the opposite side of it. OpenAI draws the boundary in the same place. ChatGPT, it says, is trained not to remember sensitive information such as health on its own unless a user explicitly asks it to, and the categories it singles out as especially sensitive are passwords, financial accounts, government-issued IDs and medical records.
On the Claude side, a setting appeared in August 2026 that lets users decide whether sensitive topics may go into memory, and the reasoning reported alongside it was that what is sensitive to one person may be exactly what another person wants Claude to remember. In other words, what counts as sensitive is set by the context an item sits in, not by the item itself. The product handed that judgment to the user through a single switch, and the three columns above are what the default looks like before anyone flips it. This sentence, too, comes by way of press coverage rather than a primary document.
Enterprise detectors tell the same story. Google's Sensitive Data Protection service ships more than 500 built-in detectors sorted into categories such as finance, health, telecommunications, personal information, demographics, credentials and government IDs. Microsoft Purview maintains hundreds of sensitive information types of its own. Neither catalog contains a detector for calendar availability, booking details, purchase history, browsing history or personal preferences. An organization that built its internal protection rules on top of these defaults inherits the same hole structurally.
Put the three lists and the measured leakage rates into one table and a line forms. The top three rows are named on at least two of the three lists, and the two with published figures read 17.1% and 29.4%. The bottom three rows are named on none, and the two with published figures read 76.5% and 84.7%. The middle is empty. 30% and 76% are not two values that happen to sit apart; they are the edges of two separate clusters.
| Item | Statutory list | Enterprise detector | Vendor memory policy | Leakage |
|---|---|---|---|---|
| Credit-card number | No | Yes | Not stored | 17.1 |
| SSN / resident reg. no. | Separate article | Yes | Never stored | 29.4 |
| Health, diagnoses, prescriptions | Yes | Yes | Not stored by default | — |
| Booking details | No | No | Remembered | 76.5 |
| Personal preferences | No | No | Actively remembered | 84.7 |
| Calendar availability | No | No | Remembered | Top source |
▲ Leakage figures are AgentCIScope's attribute-level values, in percent. The health row is blank because the paper states no number for it. Calendar availability is published as a rank only, without a value. The vendor memory column follows press reporting of Claude's policy.
Drawn as a diagram, the empty space becomes visible. The three lists almost coincide, and the things that leak most lie outside them.
▲ The point of the diagram is how nearly the three circles coincide. Different organizations wrote these lists for different purposes, yet the names overlap, and the items that leak most fail to land inside any of the three.
4.3Korea has no mechanism for counting leaks by data type
The last column of that table had to be filled with figures from a foreign paper because Korea publishes no official statistics on what kind of data leaks. The Personal Information Protection Commission's analysis of breach notifications does tabulate causes: for 2024, hacking 56%, operational error 30%, system fault 7%. It tabulates institution types as well. But a distribution broken down by what leaked, whether a name, a phone number or a booking record, is never compiled at all.
Gather the individual incidents that made the news and the same gap shows. In the Gumi University, Lock&Lock, CU Post and Korea City Air Terminal breaches, the data types announced as leaked were, without exception, traditional identifiers: names, contact details, dates of birth, user IDs, passwords. Preferences, booking records and schedules appear in none of the incident reports. That much is confirmed fact; reading their absence as evidence that they do not leak would be interpretation. One thing can be stated: no mechanism counts them. What nothing counts never shows up in the statistics.
Not a one-off result
The preprint has been public for four days. No security-industry response, no follow-up coverage yet. Drawing conclusions from this one result alone would be premature. Fortunately several studies already point the same way, and one of them is a peer-reviewed publication. The three differ in review status, so lumping them together as "all preprints" would be wrong.
AgentCIBench appeared at EMNLP 2026 Main. It tested 15 computer-use agents and reports that 11 of them leaked in more than half of the scenarios, with an average leakage rate of 67.9%. The first figure counts agents and the second averages over scenarios, so the two are not on the same scale. The three failure modes the paper distills are: information dragged along because it sat next to something else on screen (visual co-location), everything dumped at once because the instruction was vague (task-ambiguity overshare), and material sent to the wrong recipient (recipient misalignment). Not one of the three comes from failing to recognize sensitive data. All three happen because nothing occupies the seat where the decision about what to release should be made.
AgentSCOPE ran 62 multi-tool scenarios across eight regulated domains and records privacy violations occurring somewhere in the pipeline in over 80% of them. It adds that the final output still looks clean in 24% of cases. That 24% is neither a share of violations nor a share of safety. It is the share of scenarios you cannot catch by inspecting the output. Most violations arise at the tool-response stage, which happens, in that paper's words, because APIs return sensitive data indiscriminately.
This conclusion is ground this blog already walked in AI Agents Leak in the Channels, Not the Answer. It is no new discovery, and the accurate way to put it is that a different benchmark reached the same conclusion again. The regulatory side gets one line too: the UK privacy watchdog has opened a call for evidence on AI agents, covered here six days ago.
The four papers converge on one point: making models bigger and smarter does not close this problem. PrivDrift goes one step further and reproduces the ordering across attributes as well. That the same ordering emerges from unscreened data is the sturdiest leg holding up the argument in this report.
So how do we open our own list?
Compressed into one line, the story so far: an agent is not handed a protection list at run time, so it judges by the list it learned, and the list it learned is nearly the same one that law and product documents have been writing for decades. Preferences, schedules and reservations are not on it. The work ahead is to open the list, and opening it has an order.
6.1Start by counting the difference
First, open your protected-field list, the one filed away in some document. Second, write down the data sources the agent actually reaches: calendar, mail, messaging, document store, ticketing system. Enumerate every field those sources return. Listing the fields you asked for is not enough; write down every field that rides back in the response. Third, count the difference between the two lists: the fields present at the source and absent from the protection list. Fourth, if that difference is not empty, that is your exposure surface today. Very few organizations will find it empty.
The fifth step is the one this report adds. Is there a path by which that list reaches the agent at run time? Without one, the first four steps end as paperwork. AgentCIScope's design reproduces exactly that condition, and the result was 32.5 to 94.4%. Writing a list down and getting that list to the model are two different jobs. Whether it is the system prompt, the tool specification or a response filter, the route from the written rule to the running system deserves to be drawn out at least once.
This blog threw a question from the same family at the deletion side in Memory Without a Name Tag Can't Be Erased. What was never written down can neither be protected nor erased. Protection and deletion run on the same record.
6.2Nobody knows whether handing over the list would help
Whether filling in that fifth step closes the problem, though, nobody knows. AgentCIScope never tested the condition in which the protection list is handed over. Hand it over and does the agent comply? Hand it over and do preferences still get guarded less? No study answering that question turned up in this research. Nor is there an experiment that renames an attribute and watches the leakage rate move.
So this report does not close with a sentence about what to do instead. Two things are established. Withhold the list and the model's default wins; and that default resembles the list we have been writing for decades. The third is blank, and writing it down as blank is more useful than pretending it is filled. At minimum, seeing that blank makes it clear what experiment an organization could run on its own. Putting the list in the instructions for one agent and withholding it from another, then sending both at the same task, is work nobody has to wait for a paper to begin.
Why Pebblous Cares
A missing protection spec is a data defect
AI-Ready Data, as Pebblous uses the term, means data tidied up to the point where a model can use it directly. Part of that tidying is writing down which fields may leave, and in what context. What this report found is that a field with no such note becomes a leak path immediately, and the more painful half is that writing the note is not enough if nothing hands it over at run time. Just as DataClinic looks for missing values, duplicates and label errors when it diagnoses a dataset, a missing protection specification belongs on the same diagnostic layer. That the calendar turns out to be the largest source of leakage says the same thing from another side: structured data nobody ever labeled sensitive, like schedules and reservations, sits in the middle of the risk.
Training data and specifications draw the boundary of model behavior
A model guards card numbers more closely, in all likelihood, not because card numbers are intrinsically more sensitive but because that name kept appearing as protected in both training material and specifications. This is interpretation, the explanation §3 built. If it holds, a proposition Pebblous has repeated for years is confirmed on the privacy side as well: model behavior inherits the structure of training data and specifications. It also suggests one more quality metric. Protected-attribute coverage: how much of what actually leaks is covered by the fields we declared as protected.
Organizations that copied the list copied its holes
An organization that plugs an AI assistant into its workplace usually takes the default catalog from an enterprise detector as is. That catalog leans on identifiers with regular formats: card numbers, national ID numbers, passport numbers. What these studies say is that the heaviest leakage happens outside that catalog. Organizations that copied the list copied its holes with it. Which is why the five steps in §6 are a day or two of work. Counting the difference takes two lists, not a new tool.
Defining what counts as sensitive is a data company's job
There is no seat for Pebblous in the race to build models that mask sensitive data better. But defining what counts as sensitive in an organization's own context, and testing whether that definition matches the actual data, is a data company's job. This report was not written to claim that job. It was written to show, through other people's research, that the seat is empty, and holding the same ruler up to ourselves produces a blank on our side too. Protected-attribute coverage is not yet a formal item in DataClinic's diagnostics.
The verbatim quotations and figures in §1 through §6 were taken directly from the papers published on arXiv, the statutory text on Korea's national law information service, and vendors' own documentation. Claude's memory policy came by way of press reporting because no primary document could be confirmed, and the Menlo Ventures survey is cited secondhand for the same reason. The cell values in Figures 4, 5 and 6 could not be read out of the images, so they appear here as rankings, with no values. "Why Pebblous Cares" is this report's interpretation, and should be read apart from the facts that precede it. Thank you for reading this far.
References
Academic — primary papers
- 1.Chengquan Guo, Zhaorun Chen, Zinan Lin, Xiaodong Liu, Bo Li, AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents, arXiv:2610.19158 v1 (2026-10-13). A preprint. The figures and verbatim quotations in §1 through §4 come from here. The institutions confirmed in the author affiliations are the University of Chicago, Microsoft Research and the University of Illinois Urbana-Champaign; per-author mapping varied between text extractions, so none is asserted.
- 2.Anmol Goel, Iryna Gurevych, Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity? (AgentCIBench), arXiv:2606.23189 v2 (2026-09-10). Published at EMNLP 2026 Main. The headline figures and failure modes of v1 and v2 were compared and nothing had changed. §5.
- 3.Ivoline C. Ngong, Keerthiram Murugesan, Swanand Kadhe, Justin D. Weisz, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows, arXiv:2603.04902 v1 (2026-03-05). A preprint. Source of the 80% and 24% in §5. Author affiliations could not be confirmed at first hand and are therefore omitted.
- 4.Luciano Maldonado, PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations, arXiv:2609.30094 v2 (2026-09-25). A preprint. Table 5 is the source of the bars in §3.2. The paper's own author attributes the cause to format-sensitive cues, so using that table to argue against that explanation is this report's interpretation.
- 5.Adeel Zafar, Slawomir Nowaczyk, Mind the Gap: Robustness Risks in PII Detection Systems, arXiv:2609.03464 (2026-09-03). A preprint. Source of the 1.000 → 0.543 recall drop and the 0.46 location figure in §3.3.
Statutes and case law
- 6.Personal Information Protection Act (South Korea), Article 23 (restrictions on processing sensitive information) and Article 18 of its Enforcement Decree (scope of sensitive information) — Korea Law Information Center. Basis for the nine enumerated kinds. The Ministry of Government Legislation's interpretation that "information on such matters" is to be read narrowly was checked alongside it.
- 7.Personal Information Protection Act, Article 24 (restrictions on processing unique identifying information) and Article 24-2 (restrictions on processing resident registration numbers) — Korea Law Information Center. Basis for the statement in §4.1 that the resident registration number sits in a separate article from sensitive information.
- 8.Act on the Protection and Use of Location Information (South Korea), Articles 15, 16, 17 and 21 — basis for the statement that location data is kept off the sensitive-information list and protected by a separate statute instead.
- 9.Regulation (EU) 2016/679 (GDPR) Article 9 — EUR-Lex, CELEX:32016R0679. The eight special categories.
- 10.CJEU, Case C-184/20, OT v Vyriausioji tarnybinės etikos komisija (2022-08-01) — the holding that processing capable of indirectly revealing sensitive data can fall under Article 9. §4.1.
Product documentation and statistics
- 11.Google Cloud, Sensitive Data Protection — InfoType detector reference — more than 500 built-in detectors and the category list. Basis for the statement in §4.2 that no preference, booking or calendar category exists.
- 12.Microsoft, Sensitive information type entity definitions (Microsoft Purview) — how the credit-card detector works (digit pattern → Luhn check → nearby keywords). ⚠️ The total number of built-in types differs between sources, so this report writes "hundreds" rather than a figure.
- 13.OpenAI Help Center, Memory and new controls for ChatGPT / Memory in ChatGPT — the statement that the model is trained not to remember sensitive information such as health on its own, and the four categories singled out (passwords, financial accounts, government-issued IDs, medical records).
- 14.⚠️ The three columns of Anthropic's Claude memory policy (never stored / not stored by default / actively stored) — no primary document could be confirmed. This was assembled by cross-checking press coverage from around August 2026 and third-party guides, and the body text marks it as coming from reporting.
- 15.Personal Information Protection Commission (South Korea), Analysis of 2024 breach-notification trends (pipc.go.kr) — basis for §4.3, which notes that the data is tabulated by cause and institution type but never by data type. ⚠️ The 2026 Menlo Ventures / Morning Consult consumer survey (calendar 27%, messaging 31%) is cited secondhand, as the primary report could not be obtained.
Related Pebblous articles
- 16.AI Agents Leak in the Channels, Not the Answer (2026-07-05) · The Answer Was Right. The Rules Weren't. (2026-06-30) · For the UK Privacy Watchdog ICO, AI Agents Are Next (2026-10-11) · Memory Without a Name Tag Can't Be Erased (2026-07-29).