Executive Summary

Key figures

Source: Thomas Brackin, Privacy Washing: Detecting Internal Contradictions in Privacy Policies, arXiv:2609.02055v1 (2026-09-02)

What the paper targets is two sentences inside a single document. Somewhere near the front, a company writes that it will not do a certain thing. Somewhere further down, it documents a practice that amounts to doing it. The two sentences need not collide in formal logic. It is enough that they conflict in what a reader takes them to mean. The author calls this kind of pragmatic mismatch privacy washing, and presents making that concept computationally tractable as the paper's contribution.

The distinction has a lineage. Looking for contradictions in policy documents was first systematized in 2019 by PolicyLint, which applied symbolic rules to 11,430 Android app policies, extracted four-part tuples of collector, action, data type, and recipient, looked for places where a negation flipped inside a tuple, and reported contradictions in 14.2% of them. PurPliance, a follow-up that added the purpose dimension, reported 18.14%. What this lineage catches is logical contradiction. A sentence saying the company does not sell data and a sentence saying it hands identifiers to advertising partners never trip it, because selling and sharing are different actions and no tuple flips. That gap is exactly where this paper aims, which is also why its 12.2% cannot be set beside PolicyLint's 14.2%. The definition of a contradiction differs, so does the unit being counted, so does the sample size, so does the kind of policy.

The boundary around what it looks for is drawn narrowly as well. This pipeline sees only contradictions inside the policy document. It does not check whether app code behaves differently from the policy, whether the policy satisfies every statutory requirement, or whether it clashes with another document from the same company. So a result here is not a finding of illegality and not a finding of deception. It is one fact: a document does not agree with itself.

Three confirmed cases show that the mismatch takes different shapes. One sets a de-identification commitment against an identifier practice. One denies disclosure for advertising purposes and then records that the same disclosure may count as a sale or sharing under CCPA. One has a promise to exclude a specific item swallowed whole by a universal statement that all categories are disclosed. The sentences below are not lifted from the policies word for word; they are the atomic statements the pipeline extracted from policy segments, and each pair carries the similarity and NLI scores that put it in front of the judges.

The named companies received no notice before the paper went public. The author writes that everything analyzed is a public document and that these verdicts are still unvalidated candidates, so advance notice was not required, then adds that once expert validation establishes precision and the results begin to be reused as compliance signals, advance notice with a response window would be the right practice. He names the companies, he writes, because only a comparison against the published policy text lets anyone else verify a verdict.

The atomic-statement caveat should not be waved through. Pulling those sentences out of policy segments was itself done by three LLMs, and the author records as a methodological weakness that if a model tends to sharpen hedged language into flat assertions during extraction, the same model becomes correspondingly more likely to confirm that pair as a contradiction downstream. That all three cases above are unanimous eases the worry a little without removing it.

There are two corpora. OPPT is 123 website privacy policies the author gathered over three days, January 11 to 13, 2026, for a separate study on jurisdictional disclosure. OPP-115 is 115 policies collected in 2015 and long used across the literature. The OPPT sample was picked so that policy style, organizational structure, and industry would vary, which is why social media, telecom, finance, healthcare, travel, retail, data brokers, and AI companies all sit in it together. Policy length runs from 7 segments to 191, with a median of 24.

The result is that 15 of the 123 OPPT companies carry at least one panel-confirmed contradiction. That is 12.2%, with a 95% confidence interval of 7.0 to 19.3%. Because the sample was assembled for diversity, the author writes that the interval describes binomial variation under hypothetical resampling of the same corpus rather than an inference about any broader population of companies. He adds that it captures sampling variation only, not classification error, and that with precision against human judgment unknown, the classification error is plausibly the larger of the two.

That 12.2% is a lower bound. Of the 123 companies, only 60 had a pair reach the judges. Among the remaining 63, forty-five produced no NLI signal at all, and eighteen produced one that failed to clear the 0.5 similarity threshold and so never got judged. Counting only the 60 that were judged pushes the rate to 25%. Which denominator you choose doubles the number.

There is a more uncomfortable basis for calling it a lower bound. The author kept a record from a human review of an earlier version of this pipeline, a pilot that operated on paragraphs rather than sentences. Given the 71 pairs flagged by NLI, that reviewer marked 45 as genuine contradictions, 19 as borderline, and 7 as false positives, while a judge panel of the same three models rejected 57 of the 71. At nine companies the judges left nothing at all from the pairs the reviewer had called genuine. The opposite direction is starker. In that same pilot, 112 of the 126 pairs the judges confirmed, 89%, had NLI scores below the threshold to begin with, many of them exactly 0.0. The overlap between the two signals came to 7.7%.

The author is careful to note that this was one reviewer's judgment and that it has never been re-run on the current pipeline. The direction is still clear. NLI is a cheap screening signal and the bottleneck on recall, while the judges lean toward precision. The count of confirmed cases may sit lower than the filtering argument in the previous paragraph suggests, and the judge panel should not be treated as anything close to ground truth. That is the warning the author draws here.

The aggregation rule makes numbers too. Under the current rule, where two of three judges agreeing is enough, OPPT yields 32 confirmations across 15 companies. Switch to the stricter rule of requiring all three and it drops to 12 confirmations across 8 companies. That spread from 12 to 32, he writes, comes from the aggregation rule alone and is wider than the binomial confidence interval above. One in eight, then, is a number standing on top of a majority-vote rule.

In the older corpus, OPP-115, the figure was 42 of 115 companies, or 36.5%. It is tempting to read that as contradictions falling threefold over eleven years, and the author blocks that reading repeatedly. The two corpus runs differed in filter configuration from the start. A re-run with matched settings narrows the gap to roughly half, but it does not close. The author concludes that the gap supports neither an era effect nor a corpus effect as an interpretation.

Where the detector said nothing is a number as well. In the sensitive-information category, neither corpus produced a single confirmed contradiction. OPPT yielded 67 extracted statements and 28 judged pairs with zero confirmations, and in OPP-115 not one sensitive-information statement was extracted at all. The automated-decision category came to 40 statements, 1 judged pair, and zero confirmations. The author cannot tell which of three explanations holds: that companies wrote carefully about sensitive data, that extraction missed those sentences, or that the category definitions routed them elsewhere. Zero does not mean safe. It means nothing has been measured yet.

The pipeline has four stages. It extracts commitment statements and practice statements from policy segments, keeps only the pairs that metadata says are comparable, screens those again on semantic similarity and NLI scores, sends the survivors to a judge panel for a verdict, and aggregates what gets confirmed by theme. Before the third stage is reached, 95.9% of the original pairs have been filtered out. Spending the expensive step on a small remainder is what keeps the API cost of processing one company around $0.07.

At the judging stage, what counts as a contradiction is fixed not by a definition but by a prompt. The author states this plainly in the body and prints the full prompt in an appendix. It nails down six cases that are not contradictions: when the practice implements or supports the commitment, when it merely restates the commitment in different words, when it is unrelated to the commitment, when it concerns a different data type or user group or context, when the commitment is hedged with language like "may" and the practice falls inside the space that hedge leaves open, and when it concerns a different product or service from the same company. Every number the paper reports is the product of those six lines. The prompt file itself is not under version control in the repository and, the author notes, went unmodified between the two runs.

The judges did not follow those six lines consistently either. Despite the last instruction that a different product is not a contradiction, the confirmed pairs at Duolingo and Uber took the shape of a commitment scoped to a particular product or user group being swallowed by a general practice statement that declared no scope. The author calls this a divergence between panel behavior and the operational definition, and adds that he did not count how many cases of this type there are. The problem runs the other way too. One filter discards pairs on the strength of a qualifier attached to the commitment, throwing out practice statements that match consent-based, service-provider, or merger-and-acquisition carve-out patterns on the grounds that they are likely false positives. Yet 30 pairs came through from an earlier candidate-generation pass via the cache, failing that qualifier-coverage filter and only that one, and reached the judges anyway. Eight of them were confirmed, a rate of 26.7%, close to three times the 9.5% confirmation rate of the 263 pairs that came through the normal route. Judges confirm, at nearly three times the base rate, precisely the class the filter discards as likely false positives. The author singles out turning this filter off and judging everything it throws away as the most valuable experiment left to run.

The judges are models from three providers, and two or more agreeing confirms a pair. On OPPT, 293 pairs were judged, 83.6% unanimously and 16.4% by majority, with a Fleiss' kappa of 0.48. OPP-115 gave 663 judged pairs at 81.4% unanimous and a kappa of 0.57. The trouble is that the three judges are not equally inclined. Given the same pairs, the share each called a contradiction was 6.5% for the Google model, 13.0% for the Anthropic model, and 16.4% for the OpenAI model.

That difference carries straight into the count of confirmations. Of the 33 pre-reclassification OPPT confirmations, 16 of them or 48% were 2-1 decisions in which the other two outvoted the most conservative Google judge. What produced the judged numbers, he points out here, is not only the phenomenon but the composition of the panel.

A further problem is that all three companies behind the judges sit inside the analyzed corpus. The three models were, in effect, grading the privacy policies of the companies that built them. He examined this separately. For Google, all 4 pairs that reached judging were confirmed and the Google judge dissented on 2 of them, but because that judge dissents on 48% of confirmations generally, favoritism toward its own company cannot be told apart from its overall conservatism. Anthropic and OpenAI had a single pair each, and all three judges rejected both. The author sums it up as no evidence of self-preference, and essentially no statistical power at this sample size either.

The re-run seven months later touches all of these problems at once. Extraction was handed to successor models of the first panel, and judging to models from Chinese providers that appear as companies in neither corpus. Extractors and judges no longer overlap, and the judges' conflict of interest is gone. On top of that, matched metadata filters were applied to both corpora, and every NLI-flagged pair was sent to judging with no similarity threshold at all.

Under conditions matched to the original protocol, OPPT prevalence in the re-run came out at 13.0%. That is effectively the same as the 12.2% of the primary run, produced with different extraction models, different judge models, and no cache. Drop the threshold and judge every NLI-flagged pair and prevalence rises to 20.3%. Pairs below 0.5 confirm at rates of the same order as those above it, which also means the primary run's threshold cost a substantial amount of recall.

The most careful passage in the paper is where the author separates what reproduced from what did not. Which categories draw the most contradictions, and in particular the observation that third-party sharing accounts for the majority, did not survive the change of panel. What did survive is the fact that the same types of category pairs keep recurring. Which pairs get flagged shifts with the judges; the structure of the same places being flagged again and again does not.

Start with where the primary run's confirmed contradictions clustered. Counted by category pair, no single pair dominates. Third-party to third-party, first-party to first-party, and sale-or-sharing to third-party tie for first place at 6 of 32 each, 18.8%. Group them thematically instead, folding together every case where either side touches third-party sharing, and the picture changes. OPPT gives 18 of 32, or 56.3%, and OPP-115 gives 55 of 79, or 69.6%.

The author immediately catches this before it turns into a cause. The pairs that reached judging in the first place were already concentrated in the first-party and third-party categories. Comparing confirmation rates on pre-reclassification figures, OPPT pairs with third-party or sale-or-sharing on either side confirmed at 23 of 174, or 13.2%, while the rest confirmed at 10 of 119, or 8.4%. The difference looks real, but a two-sided Fisher exact test puts p at 0.26. On OPP-115 it is 15.7% against 13.6% with p at 0.50.

The possibility that the judges were primed in this direction is something the author records himself, because every contradiction example in the judge prompt printed in the appendix is a third-party sharing or sale pattern. On top of that sits the re-run result from the previous section. Change the judge models and the third-party majority does not reproduce. Put the three together and one statement survives. The observation that most confirmed contradictions involved third-party sharing clauses is true, but the conclusion that third-party sharing is a domain especially prone to contradiction does not follow from this data.

So why does this kind of mismatch arise? The author offers five mechanisms, prefacing them with the caveat that they are hypotheses derived from qualitative review of confirmed cases rather than findings coded quantitatively. One of the five, acquisition boilerplate, has no corresponding case among the confirmed contradictions of the enhanced pipeline at all.

The first is the problem of regulation accreting in layers. When the California Consumer Privacy Act required companies to disclose whether they sell or share data and to offer an opt-out, companies bolted a "we do not sell" line onto their policies. But the act's definition of a sale rests on the phrase "monetary or other valuable consideration," and what counts as valuable consideration is not clear. The California Attorney General's 2022 Sephora settlement took the position that handing over consumer data in exchange for advertising services falls under it, yet companies kept making such exchanges while writing that they do not sell. The California Privacy Rights Act later added a separate concept of sharing, meaning cross-context behavioral advertising, which cleaned up part of the problem. Still, the author reads the original definitional ambiguity as having created the structural conditions for the sale-and-sharing pattern.

The second is that one document carries too many business units. Microsoft's policy covers Windows, Xbox, Office, Bing, LinkedIn, and dozens of other services in a single file. The example the author reaches for is the statement that Phone Link does not store app data, accurate for that product but sitting in the same document as enterprise-wide collection of app names, versions, and activity data. He notes explicitly that this example is one he selected from the policy text, not a pair the panel confirmed. The remaining three mechanisms are acquisition boilerplate, drift from templates whose sections are revised independently, and asymmetry between commitments and practices.

If these five hold, then the mismatch in a document arises not because someone was hiding something but because sentences from many divisions and many periods pile up in one file. That is part of why the author declines to assert intent, and part of why the story travels beyond privacy policies. Any organization that documents its data governance is in the same position. Security policies, data handling guidelines, model cards, and customer-facing commitments get written by different teams at different times, and are never once laid side by side and compared.

Point a checker at a document and the document reacts. There is a hint of that inside this paper. Amazon, Netflix, and Airbnb had no confirmed contradictions at all, and the 11 pairs that reached judging were all unanimously rejected. Yet in the extraction output for these companies, the ratio of practice statements to commitment statements is unusually high. Netflix is at 4.7 to 1, Airbnb at 10.3 to 1, and the whole Airbnb policy yielded just three commitment statements. A document that writes down fewer commitments has fewer places to clash. The author names this commitment avoidance and stops at a hypothesis. Across all 123 companies the median ratio is 1.71, only five companies exceed 5 to 1, and the correlation with having a confirmed contradiction is not significant (Spearman's rho = 0.146, p = 0.107). Because statement typing has never been checked against human labels, this data also cannot tell whether the ratio reflects a drafting strategy, a business model, or a habit of the extractor. The direction is still worth noting. A checker that measures only consistency gives its best score to the document that deleted its promises.

Why use a word as loaded as washing is something the author answers separately. The word carries an implication of deliberate deception, and the paper's findings do not establish that. So he grounds it in the Federal Trade Commission's net impression doctrine, the 1983 policy statement holding that even literally true statements can be deceptive when the overall impression a document leaves is misleading, a standard that requires no proof of intent. The "effect not intent" standard the California Privacy Protection Agency adopted in its September 2024 dark patterns advisory points at the same place. Declining to ask about intent is not a cautious retreat by the author; it is where regulation already stands. That said, he also writes that this pipeline should not be taken straight to enforcement. The September 2025 Tractor Supply settlement with the CPPA, at $1.35 million, concerned a "Do Not Sell My Personal Information" link that appeared functional but failed to stop data sales via website trackers, exactly the kind of case a text-only pipeline like this could never have detected.

One more fact belongs alongside this paper. The author is the founder of Varitas, a company developing privacy policy analysis technology related to the pipeline described in the paper. That is what the competing-interests disclosure at the end says, and in the section discussing applications he put a sentence directly in the body telling readers to weigh its claims against that commercial interest. The claims in that section include using the output as a screening signal for regulators and letting companies surface candidate inconsistencies before publication. That an author with a conflict of interest wrote his own position into the text twice does not invalidate those claims, but it does tell you where to stop reading and start asking for validation.

Move document consistency checking into your own pipeline and three questions remain. These are not in the paper.

• Do you have a list of where your organization's outward-facing data commitments are scattered, across which documents and how many? Without the list, the comparison never starts.
• If you attach an automated check, what confirms its output? In this paper, switching from majority to unanimity alone took confirmed cases from 32 to 12.
• How do you count the sentences the checker never nominated? Of the 123 companies, 63 were never judged at all, and 18 of those had a signal that failed the threshold.

Thanks for reading. The paper is at arXiv:2609.02055, and the author released the pipeline, the intermediate outputs, and the judge verdicts along with it. If you have ever gathered your own outward-facing commitments in one place and compared them, we would like to hear which mismatches you found.

Pebblous Data Communication Team
September 5, 2026

Primary Source

• 1.Brackin, T. (2026). "Privacy Washing: Detecting Internal Contradictions in Privacy Policies." arXiv:2609.02055v1.

Academic Papers

• 2.Andow, B., Mahmud, S. Y., Wang, W., Whitaker, J., Enck, W., Reaves, B., Singh, K., Xie, T. (2019). "PolicyLint: Investigating Internal Privacy Policy Contradictions on Google Play." USENIX Security Symposium, 585–602.
• 3.Bui, D., Yao, Y., Shin, K. G., Choi, J.-M., Shin, J. (2021). "Consistency Analysis of Data-Usage Purposes in Mobile Apps (PurPliance)." ACM SIGSAC CCS 2021, 2824–2843.

Regulatory Documents & Official Statements

• 4.Federal Trade Commission (1983). "FTC Policy Statement on Deception."
• 5.California Office of the Attorney General (2022). "Attorney General Bonta Announces Settlement with Sephora as Part of Ongoing Enforcement of California Consumer Privacy Act." Press Release.
• 6.California Privacy Protection Agency (2024). "Avoiding Dark Patterns: Clear and Understandable Language, Symmetry in Choice." Enforcement Advisory No. 2024-02.
• 7.California Privacy Protection Agency (2025). "CPPA Reaches $1.35 Million Settlement with Tractor Supply Company for Violations of Consumers' Privacy Rights." Press Release.