Executive Summary

Key Figures

Source: Lamaakal (2026), arXiv:2608.23893, Tables III and IV and Appendix B

Picture a single payment sitting in a transaction log. The amount, the timestamp and the counterparty account are all recorded exactly as they happened. Then the in-house threshold for flagging fraud risk drops from the top 25 percent of amounts to the top 40 percent. At that moment the normal label attached to this payment loses its basis. Not one character of the data was edited.

The paper opens from the same place: "Nothing about a previously stored transaction needs to change for its correct label to become different; what changes is the definition used to interpret that transaction." While data quality work mostly points at missing values, typos and outliers, the thing that invalidates labels in production is often one revised line of policy.

Existing concept drift research infers changes of this kind after the fact, from observed values or prediction error. Performance sags, and the system concludes that something must have moved. A rule revision is a different situation. The organization already knows who changed which condition, when, and how. Rediscovering that information from the data is the redundancy the author objects to.

The default response in the field is simple enough. Relabel the entire history under the new criteria and retrain the model. In the paper's benchmark that route took 993 seconds on average to complete an update. It buys 92.8 percent accuracy at the price of sweeping the full collection once per revision.

The proposed procedure starts by laying out the rule before and after the revision as two predicate graphs. Which attribute carries which threshold, and how the conditions are joined logically, becomes explicit structure. The difference between the two graphs is then extracted and compressed into a typed delta. Whether a threshold moved down, a predicate was inserted, an AND became an OR, or a relation path changed, all of that is separated here.

The next step is the heart of the method. Every historical record carries provenance describing which conditions its label rested on. Matching the delta against that provenance splits the entire historical collection three ways: records whose previous label can be proven still valid, records whose label can be recomputed deterministically by rerunning the new rule, and records the rule alone cannot decide.

The first branch is where the actual savings live. Records land there not because they look probabilistically safe to skip, but because the revision provably cannot propagate to their labels. Local invariance at a blocking node in the predicate graph establishes that before the record is set aside. The paper calls this a provenance stability certificate. In an ablation that removed the certification step and left the rest of the procedure intact, affected-record recall barely moved at 98.4 percent while the reprocessed share jumped from 14.7 percent to 38.9 percent. What the certificate does is not change which labels get reassigned. It safely takes off the table the records that never needed a second look.

Records that can be reassigned automatically are handled by rerunning the new rule, and only the undecidable ones go to a person. The revised labels and the certified stable records are then used together to repair the deployed model locally rather than retraining it from scratch. Every rule version, predicate graph, provenance state and model state along the way is retained by version.

The author draws a line in the appendix around how far the certificate reaches. The theory guarantees the correctness of the certification rule: while the assumptions hold, a record declared stable really is invariant. Real implementations that compress provenance or reconstruct dependencies approximately can break those assumptions, and at that point affected-set recall "becomes an empirical property of the implementation rather than a mathematical consequence of the certificate."

The evaluation runs on RuleShift-Bench, a benchmark the author built. Mobile money transfers at 6.36 million records (PaySim), US census data at 299,000 (Census-Income), network flows at 2.54 million (UNSW-NB15) and a citation graph at 169,000 nodes (ogbn-arxiv) each received seven types of revision, producing eight rule versions per dataset. Threshold changes, predicate insertion and deletion, and logical rewrites were applied to all four. Relation path changes were applied only to UNSW-NB15 and ogbn-arxiv, the two with relational structure.

Three rows in the comparison table sit next to each other on the same reprocessing budget: a random 14.7 percent, the 14.7 percent the model is least certain about, and the 14.7 percent lineage points to. The budget is identical and only the selection differs, yet how many genuinely changed labels each one caught diverged sharply.

Accuracy followed the same order: 88.8 percent for random, 90.1 percent for uncertainty, 92.3 percent for lineage. One more split is worth pulling out here. A setting that used lineage to pick candidates but still retrained the whole model the old way took 515 seconds. The rest of the drop, down to 179 seconds, came from the stage that repairs the model locally. Narrowing the candidates and shrinking the training each contributed roughly half.

The same table carries the conventional drift responses. A sliding window that retrains on recent data reached 88.5 percent accuracy, an online update that absorbs revisions sequentially 87.9 percent, and replay that mixes in historical samples 89.2 percent. ADWIN, which triggers retraining when it detects a performance change, stopped at 87.6 percent, and it reexamined 28.4 percent of the data, nearly double the proposed method, while recovering only 66.0 percent of the affected records. The approaches that work out what changed by reading the data afterwards process more and catch less.

Broken out by dataset, the savings are uneven, and record counts and timings do not track each other. The citation graph ogbn-arxiv is the smallest of the four at 169,000 nodes, and full rework on it took 742 seconds, nearly five times longer than the 154 seconds for Census-Income, which holds almost twice as many records. Its reexamined share was the highest too, at 16.6 percent.

In a citation graph one condition travels outward along neighbors and multi-hop paths. Edit a single threshold and the range of records that edit can reach is far wider than in tabular data. What sets the cost is not the size of the dataset but the structure the changed condition flows through.

Not every revision is machine executable. Some need expert interpretation, some depend on an attribute that has not arrived yet, some run into messy relational information, and some require outside knowledge that the data never represented. The paper sets these records aside as an ambiguous region and hands them to people. It does not ask about the whole region, though. It selects a portion by uncertainty and representativeness and queries only that.

Which records you choose to ask about turns out to decide the outcome. With an annotation budget of a single item, selecting within the scope of the revision returned 78.6 percent Macro-F1, against 75.2 percent for selection by global model uncertainty and 72.4 percent for random. Raising the budget to 32 items kept the ordering intact, at 89.2, 85.8 and 83.5 percent.

Policies in production do not only move forward. A relaxed threshold gets reversed after one incident, and a temporary measure expires and reverts to the earlier definition. The author ran a separate experiment on a sequence where the rule changes three times and then returns to its first definition. The setting with versioned memory recovered 90.6 percent accuracy at the point of return. Its first pass on that same rule had been 91.5 percent, so the gap is 0.9 points. Replay, which retrains by mixing historical data back in, landed at 88.7 percent for a gap of 2.5 points, reexamined more data at 18.0 percent, and took more than twice as long at 315 seconds.

When the returns keep coming, the gap widens. The author ran a longer sequence in which four definitions cycle back across twelve revisions. Versioned memory scored 90.5, 90.7, 90.8 and 90.9 percent on the successive returns to the first definition, closing in on the 91.4 percent it recorded the first time around in that sequence. Handling all twelve by complete retraining accumulated roughly 202 minutes of maintenance time, replay roughly 62 minutes, and versioned memory roughly 30 minutes. The more often rules travel back and forth, the wider the gap grows between the organizations that kept a history and the ones that did not.

The author ran a separate set of experiments on the conditions under which his own method breaks, and put them in the appendix. The design pushes the affected scope of a revision from 5 percent up to 90 percent and measures update latency along the way. At 5 percent affected with loose dependencies the speedup exceeded eight times, and at 15 percent it stayed above six. By 70 percent it had fallen below two times under several conditions, and at 90 percent it came out close to full recomputation. Selective maintenance is a tool for narrow revisions.

The second condition bears more directly on data teams. Rerunning the same experiment while erasing provenance a piece at a time, affected-record recall held around 97 percent as long as lineage was complete. Below 80 percent availability a fork appears. A conservative implementation has to widen the candidate set sharply to avoid missing anything, while an approximate reconstruction of dependencies starts to lose recall. At the point where only 40 percent of lineage remained, the approximate implementation recovered 82.9 percent. Missing provenance cannot simply be read as an unchanged dependency.

The other two conditions run in the same direction. When the share of records that cannot be decided automatically climbs to 80 percent, four annotations reach only 82.5 percent Macro-F1 and it takes 64 to get to 88.4 percent. Lineage narrows the region where human effort pays off, but it cannot eliminate the information a fundamentally non-executable definition demands. Increasing graph path depth from one hop to six raises update latency by roughly 3.05 times.

None of this is free, either. The maintained state, covering provenance and indexes, predictor version information, affected-data metadata and rule graphs, comes to about 6.1 percent of the training data size with a single rule version and about 10.2 percent at twelve. By dataset it ranges from 8.9 percent on PaySim to 15.1 percent on ogbn-arxiv. Growth stays mild across versions because unchanged provenance structures and persistent rule components are shared between them. At twelve versions that works out to a reduction of more than eight times against keeping twelve independent copies.

So there are gradations between keeping a history and keeping none. The author compared the granularity at which provenance gets stored. Record-level storage is the cheapest and the weakest at localizing the affected region. Going all the way down to tuple and path level brings affected-record recall close to 98 percent at the cost of higher storage and retrieval. In between sits predicate level, storing only which condition decided this label, which held recall around 95 percent and balanced best. Provenance granularity is not a question of having it or not having it but a systems parameter to be tuned.

The paper is a preprint with no conference or journal venue attached yet. The original is at arXiv:2608.23893.