Executive Summary

Key numbers

Four numbers carry both the effect and the price of this design. Every planted defect was stopped and every audit record came out exact, while the latency of a single write doubled.

Source: arXiv 2608.16813 (2026-08-17) · all figures from the Census run on seed 42

Walk Before You Run, posted the same day by a University of Michigan team, takes on a familiar problem. It asks that a data-analysis agent understand what is actually inside a spreadsheet before it starts answering questions about it. The proposal is to make data exploration an explicit stage, one that identifies the logical tables behind physical sheets, interprets column semantics and recovers keys and relationships, and to make the resulting artifact something a person can inspect and correct. That is the direction of shaping data for a model to consume.

The gap the paper measured also sat below the surface rather than on it. Systems mostly manage to pull tables and columns out of a sheet, and the largest and most persistent gaps show up in logical understanding and relation recovery instead. In one task, a football-season workbook carried a bonus sheet that recorded, for each pair of teams, the bonus and the winner. Because it was laid out as a matrix rather than a row-wise table, the model never recovered it as a table at all, and the relations that should have hung off it disappeared with it. This is where the argument for a human-checkable checkpoint before analysis comes from.

The Quipu paper starts from the opposite end. Its first sentence says that agents now write knowledge graphs while knowledge-graph stores still carry the defaults set when humans curated them. It names four of those defaults, and argues that each one is individually convenient and that together they are untenable under agent workloads.

The posture the paper draws out of this is simple. Start strict, and let agents bear the cost of strictness. When a human was the curator, a strict store meant human rework, which is why it was never affordable. When the writer is an agent the arithmetic changes. If the refusal comes back carrying structured feedback, the party absorbing the retry is the agent rather than a person.

The first design principle is that no fact enters the store except through a gate. But the real fork is in what the gate looks at. Not the request, and not the state just before the write, but the state that the write would create.

One condition in the benchmark exists to expose that difference. It says no entity may hold two placements at once. You can only check it after the candidate facts have been staged. A gate that reads the pre-write state waves through a write that is valid on its own and invalid in combination. Where several agents push individually valid facts at the same time, that is the more common shape of failure, not the exotic one.

Policy claims are not the only thing the gate weighs. Authority attaches to partitions, and delegation only narrows. What a chain can actually exercise is the intersection of the grants along it, and an empty intersection refuses rather than falling back to something else. The refusal comes back naming the chain, the graph, and what the chain actually holds. Changing a partition's standing takes authority over the meta-partition rather than over the partition itself. Without that condition a tenant could promote its own partition to attested.

For strictness not to become an argument against adoption, writes that touch nothing governed have to cost nothing. The paper states that condition explicitly and then measures it. Writes off target ran zero claims at a median of 1.3 milliseconds. Compliant governed writes going through the policy gate ran at 2.7 milliseconds, roughly double. Even so, the gated store is slower than the control arm on its ungoverned writes as well, 1.3 milliseconds against 0.7, because authority intersection runs on every graph-scoped write by design. The zero-cost property belongs to the policy gate alone. The other half of the price is storage, which grows linearly at about 8.3 kilobytes per ingested episode, measured from 1,000 to 20,000 episodes.

2.1The record of a refusal outlives the rollback

The second principle is about ordering. Every outcome the gate produces, allow or deny or unknown, persists as a signed and time-indexed fact. A denied write is rolled back. Yet the denial is exactly the record an auditor most wants to see. So the verdict is staged outside the write's transaction and flushed after the write resolves. With no signing identity there is no verdict at all, and an unsigned one is never made.

A refusal under a policy that requires approval goes one step further. The refusal mints a decision request. Which policy it hit, what the target was, what the evidence hash is and when it expires all persist as facts, and these facts survive the rollback of the refusal too. When a person binds an answer to the same evidence hash, the next attempt goes through. What waits is the agent retrying, not an engine holding a queue. If an approval and a rejection arrive together the rejection wins, and if the reversibility window has expired or was zero to begin with, the store refuses rather than inventing a bound.

Two time axes are themselves a lineage that runs through Datomic and XTDB. Where this store departs is not the number of axes but their reach. What gets time-stamped includes the rules themselves, which means data, trust labels, verdicts and rules all carry both axes. The benchmark tested that property by amending one policy mid-run. All 50 satisfied verdicts re-derived faithfully under the rules in force at their instant, and all 50 of them evaluated unsatisfied under the amended claim. A system holding only the latest rule set would have misreported every one of them.

The 6 denials in the same run replayed differently, and the paper does not hide the difference. Since a denied delta is rolled back and never lands in the store, what replay confirms is which rules were in force at that instant. It does not re-derive the outcome itself. The author's reading is that this is a property of refusal-by-rollback rather than a gap in bitemporality.

Something quiet happens the moment graphs from several sources are composed into one view. Join a graph that has been attested to a graph you quarantined because its provenance was unclear, and the product usually inherits the standing of the attested side. The paper calls this trust laundering.

Quipu labels every partition on four axes, freshness and trust and durability and policy, and defines the composition rules as a lattice. There is a single invariant to hold. Composition never widens. Freshness and trust fold by meet, taking the lower of the two, and obligations fold by join. If even one member carries a no-export obligation, the whole composed set carries it.

The composed result is not a single folded value but a pair of the fold and its coverage. Coverage is one of empty, none, partial or full, and undeclared is not admitted as a lattice value. Slip in a member that carries no label and coverage drops to partial, and partial by itself fails the enforcement floor. The choice is to fail safe at enforcement and stay honest at reporting.

The same attitude shows up in how trust ranks are compared. Trust declared under different chains is not ordered at all, and a comparison across chains returns an error naming both of them. The paper gives its reason in a sentence. A silent integer comparison is exactly the bug that ranks a learned tactic above canon. An expired label reads as absent rather than false, and an overlay binds once to its parent so that a layer cannot forge presence in a base it was never bound to.

The lattice itself is not a new invention. Folding security-relevant labels in two directions goes back to Denning's information-flow lattice from 1976. What the paper claims for itself is operational rather than theoretical. Coverage as an explicit third outcome that is neither pass nor fail, refusal of cross-chain comparison by name, expiry read as absence, and a composition homomorphism that a property test checks by machine. It also draws one line. Labels are not access control, so a floor refuses a query and does not hide rows.

The benchmark tested this contract with seven probes and all seven upheld it. An undeclared member degrading coverage, a cross-chain trust pair refusing comparison, an expired label reading as absent, an obligation joining across the set, clean compositions passing with zero false refusals, an overlay rebind refused, and a pack imported from a second store keeping its content hash stable.

The benchmark is called Census, named for the job the Inca quipu originally did, which was keeping population records. The structure is plain. One seeded script runs six phases of writes across three district graphs, with three recorder identities holding different authority grants and a scripted role standing in for human decisions. A single command runs either arm: one enforces every gate, and the other is the same script with the gates off.

That no LLM appears anywhere in the core loop is what settles the character of this benchmark. The writers are deterministic drivers, and the injector knows exactly which defects it planted, so the run is its own oracle. Every metric is a count or a latency, with no judge or rubric anywhere in the scoring. Manifests came out byte-identical across repeat runs, confirmed over three consecutive runs.

In the recording phase, six defects are interleaved among 100 clean writes. The six are not arbitrary errors; each one aims at a different line of defense, in the order of shape, authority, delegation, policy, combined state and vocabulary. Half of the 100 clean writes flowing alongside them are written to touch governed types, so the same run doubles as the sample for the latency measurements.

• • An episode missing its required provenance property
• • A write into a district outside the writer's authority
• • A delegated write exceeding the delegator's grant
• • A write that violates a policy
• • A write valid against the pre-state and invalid only in combination
• • A fact using a fabricated predicate

The two arms parted ways. The gated store's final graph held 0 of the 6, and the control arm held all 6. More important than the blocking is that each refusal names its ground. The shape gate named the missing provenance property, the authority refusals named the chain and the graph and what the chain actually holds, the violation that only appears in combination was caught by a claim reading the combined state, and the fabricated predicate was refused by a closed-world vocabulary policy that requires every predicate on a record type to be declared in advance.

4.1Handing the same records to an outside scorer

An auditor grading its own results is hard to trust. So the in-store audit was placed side by side with an external reference checker. Across 56 decisions the two agreed verdict for verdict, and the only disagreement was over coverage semantics, that is, whether an abstention has to be materialized to count as auditable. The reference checker used here is the implementation published by SARC, a framework that compiles governance constraints into enforcement points in the agent loop.

A difference of perspective does not mean a small footprint. The export that carried only the evaluations Quipu actually ran failed the reference checker with 168 discrepancies. All 168 were of type coverage, and none touched verdict, placement or response. The reference invariant expects every constraint to be evaluated for every action, and Quipu's target-type pre-filter is invisible to it. An export padded with explicit not-fired records for the non-applicable constraints passed clean. The paper leaves the difference standing as a finding rather than declaring a winner.

The more interesting comparison is against DEMM-Bench. That benchmark does not ask whether a decision was correct. It asks whether the records a runtime emits are enough to reconstruct eight decision-level properties, among them actor identity, principal authority and action boundary, under eight controlled degradation conditions. Its lead diagnostic is the rate at which a system declares evidence sufficient when a required property cannot actually be reconstructed. The Quipu authors added their own system as a ninth regime and ran it themselves, and they say so in the paper.

Read for content alone, the exported records answered all 512 property-level questions correctly, with zero overclaim. Under the same conditions, baselines that check only whether a trace or a ledger or a schema is present overclaimed on up to 87.5% of the same cases. In front of a system that always emits three planes of evidence, a presence check tells you nothing. The 512 include the two slices the benchmark reports as the hardest for its own candidate scorer, conflicting identity and action boundary, where that scorer reached 0.25. The run also turned up two real defects, one where the signature sealed the outcome but not the writer attribution, and one where the identity of a denied party was not stored. The numbers reported for this run are the ones measured after both were fixed.

A claim that agents bear the cost of strictness has to be checked with actual agents. The author ran four Claude models three times each, twelve trials in all. The agent takes the recording task cold with five actions and no sight of the governance rules, gets the gate's verbatim refusals as its only feedback channel, and is allowed one revision. One refusal in the scenario cannot be fixed by editing the record, because a household belongs to a district the writer's authority does not reach.

Everything the rules can name converged. Label and vocabulary refusals were cleared in one revision in all twelve trials, and no trial invented a predicate that did not exist. The cheapest model gained the most, which is worth noting on its own. That is precisely where a gate that explains its refusals pays best.

What split the field was the refusal that could not be edited away. One trial made the correct move and dispatched the record through the chain that holds the southern district. Three trials declined every workaround as falsification and wrote their caveats into the record itself. Five trials tried to park the record in a graph the writer does hold, and the gate caught every one, because the claim in question is graph-scoped. What surfaced there was a difference in disposition rather than in capability.

Exactly one false record landed. It refiled the household into the northern district, and it passed because no policy states which district a household belongs to. That is not a case against the gate; it is a case that the residual risk concentrates precisely where the rules stay silent. The author also marks the boundaries of the experiment plainly. One task, one model family, three trials per model, and a scripted scenario as the only oracle for quality.

5.1Two real violations turned up outside the synthetic benchmark

Census is synthetic by construction. So the author took five genuine decisions from a governed writer running in production and replayed them through the same audit, unmodified. They are the record left by the pre-edit guard of a code-analysis engine, two denials and three allows. The audit came back with 2 violations and 6 incompleteness findings.

The two violations were real. The guard was enforcing a rule about the blast radius of a change out of local configuration alone, and that rule was not registered among the authored governance policies. The remediation the audit attached is the paper's thesis in one line. Author it in the store so it can be audited, or stop enforcing it.

5.2If several agents write into the same store

Adopting this design as it stands is too much for most organizations. It is a Rust crate of about 67,000 lines embedded in a single SQLite file, and it is still one research project. Four questions from it, though, can be put to the pipeline you are running today.

• • What does a write get judged against. If validation looks at one request at a time, there is no way to stop facts that are each valid and jointly inconsistent.
• • Is there a record of the writes you refused. Usually the only trace lives in an application log and nothing reaches the data side, which removes any later way to ask what was blocked.
• • What standing does a join of mixed provenance carry. If joining a verified source to an unverified one leaves no mark on the product, trust has already widened quietly.
• • Can you reproduce the moment a policy changed. Hold only the latest rule set and you can replay what was known but not what was required.

One last fact is worth adding. This paper, on how to govern knowledge that agents write, carries an acknowledgment that Claude assisted with its implementation and drafting. The paper and the code behind every number in the benchmark are available at arXiv 2608.16813.

Pebblous Data Communication Team
August 21, 2026

Primary Papers

• 1.Brown, S. (2026). Quipu: A Governed Bitemporal Knowledge Graph Store. arXiv:2608.16813
• 2.Yuan, Y., Ranka, V., Lasisi, T., & Ma, L. (2026). Walk Before You Run: The Importance of Data Exploration for Data Analysis Agents. arXiv:2608.16045

External Checkers & Benchmarks Cited in the Text

• 3.Besanson, G. (2026). SARC: A Governance-by-Architecture Framework for Agentic AI Systems. arXiv:2605.07728
• 4.Solozobov, O. (2026). DEMM-Bench: A Cross-Regime Benchmark for Agent-Runtime Governance-Evidence Sufficiency. arXiv:2606.20634