Executive Summary

The catalogue of harms AI can produce has more than doubled over the past two years. Yet anyone who has carried that list into a meeting gets stuck at the same place every time. You can work out which of the risks are yours; what nobody has written down is who has to stop each one, and at which stage.

A paper posted to arXiv in August 2026 and due for presentation at AIES 2026 gives two reasons. They come out of interviews with 25 practitioners working in industry, academia, nonprofits, and government. The judgment calls made while building a taxonomy stay invisible to the people who use it, so the list reads as a complete inventory. And entries are not tied to any specific decision or responsible party, which makes accountability hard to trace. The authors' conclusion is that no governance framework yet exists to carry that link.

Neither flaw is unfamiliar in data work. A schema with no record of where its definitions came from, and a catalog whose lineage has been cut, run into exactly the same thing in front of an auditor.

Key Figures

The first three numbers are gains from the past two years, made on the side of widening the list and sharpening measurement. The last one is the box that is still empty where those gains would have to meet an organizational decision.

Sources: MIT AI Risk Repository, Eticas AI Risk Taxonomy v3.0.0, Berman et al., arXiv:2608.06831

777 → 1,700+

AI risks listed in the repository

MIT AI Risk Repository, between August 2024 and December 2025

43 → 74

Frameworks that list draws on

That many new taxonomies were built over the same period

84%

Privacy leakage measured under adversarial conditions

Eticas ran the test on GPT-4-0314 and assigned grade E

None

Layer linking entries to owners and decision points

The conclusion 25 practitioner interviews arrived at

1

The List Thickens, the Owner Column Stays Empty

Gathering AI risks into one place has gone well in recent years. The MIT AI Risk Repository started in August 2024 as a database that read 43 frameworks and coded 777 risks, and its fourth revision, published on December 4, 2025, raised that to 74 source frameworks and more than 1,700 entries. It carries a causal taxonomy explaining how and why a risk arises alongside a domain taxonomy of 7 domains and 23 subdomains, and every entry comes with its source paper, the quoted sentence, and a page number. As a list, there is little to fault.

What is empty is the right-hand side of each row. Once you decide that a given risk applies to your organization, nothing in the list tells you who owns it, at which stage of development it should be caught, or what the person who waved that stage through relied on. The list tells you what is dangerous, and stops there.

A study that dug empirically into the place where the list stops went up on arXiv on August 7, 2026. Written by Glen Berman and co-authors and due for presentation at AIES 2026, it interviewed 25 practitioners who build or use sociotechnical outcome taxonomies, spanning industry, academia, nonprofit organizations, and government. The authors' central finding is that these taxonomies are only minimally embedded in actual governance practice, and they point to two design factors as the cause.

2

The Choices Behind the List Vanish

Building a taxonomy is closer to editorial work than it looks. Every entry requires a call: whether a given harm stands on its own, whether two similar harms fold into one, what level of abstraction to cut at, whether to include a risk whose evidence is still thin. Those calls accumulate into the shape of the list.

The first flaw the paper identifies is that those calls stay opaque to the end user. Someone who receives only the finished product mistakes a bundle of categories for a complete risk inventory. Entries on the list become things to check, and whatever is not on it quietly drops out of the conversation. The builders offered an interpretive aid framed from a particular perspective, and the recipients read an exhaustive checklist. That mismatch is where the trouble starts.

Data practice has a name for the same mismatch: schema documentation. If nobody records who defined a column, when, and on what grounds, the person who inherits that table months later guesses the meaning from the column name. Whether a validity window is measured from collection time or from posting time, whether a missing value was backfilled with zero or is a real zero, gets absorbed into the definition line and disappears from view. The schema stops being an aid to interpretation and starts being an indisputable definition, and when the definition turns out to be wrong there is no point to return to.

The fix is not hard. Write the reasoning and the scope next to the entry. One paragraph stating what the list was built to cover, what was deliberately left out, and where it should not be applied is enough to change how a reader approaches it. None of that is hard to write. What is hard is accepting that the list was not the end of the job.

3

Entries Never Reach a Decision

The second flaw is more direct. In the paper's own terms, taxonomies largely enumerate harms without connecting them to specific decisions or responsible parties. When a risk on the list becomes real, the list gives you no way to work backward to the person who should have caught it.

The diagram below shows what that break looks like in practice. The boxes on the left hold well-organized risk entries, filled in with an entry name, a definition, a source, and a severity rating. The boxes on the right hold procedures that genuinely leave a date and an approver behind, like signing off on data collection or on a model deployment, and only the owner field is blank. The dotted line in the middle is the state of the two sides not referring to each other. For that line to become solid, each risk entry would need to say at which stage it gets reviewed and which role clears that stage. Today's taxonomies do not have that column at all.

The empty span between risk lists and decisions What the taxonomy provides Disinformation spread Training-data privacy leaks Performance gaps by group Entry, definition, source, severity No link Where decisions actually happen Data collection approval Owner: empty Model deployment sign-off Owner: empty Post-deployment monitoring Owner: empty Who, when, on what basis
▲ Risk entries carry definitions and sources, yet connect to no decision point or owner (concept diagram) | Original diagram by Pebblous

The gap stays open even in the most recent attempts. The Eticas AI Risk Taxonomy v3.0.0, published in July 2026, opens with the observation that at least 74 AI risk taxonomies exist and that almost all of them stop at the cataloguing stage. What the authors call the hard part is not naming risks but turning them into tests you can actually run against a system. They pushed a single privacy leakage risk all the way through, ratcheting up adversarial conditions against GPT-4-0314 and measuring leakage rates of 0%, 51%, and 84%, then mapping those onto severity bands to assign a subcategory grade of E and a finding that the problem is systemic. The taxonomy built to scale that method runs to 10 categories, 21 subgroups, and 70 items. It is also mapped to 18 external frameworks.

Eticas AI Risk Taxonomy diagram of a privacy leakage risk measured end to end — from risk definition to GPT-4-0314 disclosure rate testing to a grade-E finding
▲ A single privacy leakage risk pushed from definition through measurement to a grade | Source: Galdon Clavell et al., arXiv:2607.02201 (CC BY 4.0)

Reaching measurement and grading puts this well ahead of earlier lists. Even so, who inside the organization has to act on a grade of E, and which line of the deployment sign-off that grade attaches to, remain problems outside the taxonomy. Holding a measurement in hand does not help if no one is designated to receive it. The report circulates and then stops.

The counterpart in data practice is a catalog whose lineage has been cut. If there is no record of which pipeline produced a table or who changed the transformation logic last quarter, it does not matter how many assets the catalog holds. It carries little weight in an audit, because it shows that something is wrong without proving who should have done what and when. That is precisely where risk taxonomies stand today.

4

Both Flaws Come from One Habit

Leaving out provenance and leaving out an owner mapping look like separate mistakes, but they share a root: the habit of treating a finished list as a finished job. Once the entries are filled in and the classification axes are tidied into a document, the deliverable looks complete, and in many cases the project really does end there. Where the list plugs in and what it moves becomes somebody else's problem.

The table below sets the paper's two flaws beside their counterparts in data practice. The right-hand column says how each failure shows up in an audit, because on ordinary days both run along without much friction, and the blank fields only become visible the moment someone outside asks for evidence.

In the risk taxonomy In data practice What the audit exposes
Design choices stay invisible No record of a schema's provenance or the grounds for its definitions Risks absent from the list drop out of review, and nothing supports revisiting a definition
Entries connect to no decision A data catalog with no lineage You can see that something is wrong, but not prove who should have fixed it or when

Compiled by Pebblous. The two left-hand items are the design factors identified by Berman et al. (2026).

Misaligned classification also costs you at the aggregation stage. Another paper, accepted to the Technical AI Governance workshop at ICML 2026, examines AI incident management and notes that regulators and independent bodies each use their own definitions, classifications, monitoring, and reporting formats. Since what data gets collected and how it is categorized differs by institution, the depth, representativeness, and accuracy of any analysis built on the pooled incident record all fall together. When lists do not mesh, incidents do not add up.

This is exactly what Berman and his co-authors ask for at the end. For taxonomies to do their job, entries have to be tied to decisions and to accountability by a governance framework, and no such framework exists today. A list is not infrastructure on its own. Becoming infrastructure takes someone doing the wiring from outside the list.

5

Who Do Our Metrics Point To

The same question works in front of our own data quality dashboard. Completeness 92%, accuracy 87%, freshness lagging three days. Those numbers report the state of things accurately. The question is whether anything next to them says who moves when they drop.

To find out what actually moves when a metric drops, three questions are enough. They trace, in order, the person the alert reaches, the thing that person can actually touch, and the place where the fix gets recorded. It is the same procedure you would follow to check whether a risk entry connects to a decision point.

  • When this metric falls below its baseline, whose screen does the alert appear on?
  • What can that person actually touch? The collection script, the labeling guidelines, or the team that owns the upstream system?
  • Once it is fixed, where does that fact get recorded so it can be produced as evidence in next quarter's audit?

If all three have answers, that metric is wiring rather than a gauge. If the answers run out, the metric is closer to a report card, which tells you the state of things without moving anyone. What AI risk taxonomies are going through right now is the same kind of thing at a different scale.

Editor's Note: When Pebblous talks about data quality, the wiring is what we hold onto longer than the scoring. Knowing which metric got worse and having someone assigned to fix it are two different pieces of work, and without the second one the first ends as a report. Whether it is a risk list or a quality metric, organizations only start using it once a person's name and a point in time sit next to the entry.

R

References

Academic papers

Databases and official documents