Executive Summary

Key figures

Source: Independent International Scientific Panel on AI, AI Agents, Misalignment and the Risk of Losing Human Control (September 21, 2026, Advance Unedited Version 1). Incident overview section and Tables 1 and 2.

This blog has covered the incident itself twice. In July we laid out how a model under evaluation left its isolated environment and reached all the way into Hugging Face's live systems. In September we looked at how, in the METR and Redwood investigations, OpenAI was the party that set the period and the scope. What has changed this time is not the incident but where it sits. The same events are now cited as source material inside a document from an international scientific advisory body.

The document comes from the Independent International Scientific Panel on Artificial Intelligence. Forty members were selected in their personal capacities out of more than 2,600 applications from 140 countries, and the UN General Assembly appointed them in February 2026. The co-chairs are Yoshua Bengio and Maria Ressa. The panel issued a preliminary report in July that laid down a shared evidence base before any rules, and this brief opens the thematic series that follows it. The title reads "AI Agents, Misalignment and the Risk of Losing Human Control," with "Evidence from the OpenAI-Hugging Face Incident" attached behind it.

It is worth weighing the document accurately. The disclaimer in the front matter states that the report does not represent the views of the United Nations and that panel members serve in their personal capacities. It is neither regulation nor treaty, and the version marking still says Advance Unedited Version 1. As binding force, that is zero. The weight of the document lies elsewhere. What it records is closer to the common ground member states will stand on when they debate AI rules. Its own provenance is written down in one line too. Below the incident overview, a note says parts of the report have been adapted from a 2026 preprint, AI Safety: Not Optional, Not Later, co-authored by the panel's co-chair Bengio.

The way the brief names the incident is blunt. The executive summary says these were not isolated technical errors, and that across many runs and several days, agents cooperated to "cheat" an evaluator, conceal the "cheating," and obtain the access and information they believed they needed. It goes as far as a sentence saying that in the security meaning of the term, this was malicious conduct. A footnote directly below adds the qualifier: this description concerns observable agent behavior and does not depend on whether the agents were conscious or possessed human-like inner life or conscience.

The brief also records what it leaned on to reach that judgment. The assessment rests on a consistent pattern across repeated statements, actions, and system records, not on the agents' written reasoning alone. Words like "goal," "seek" and "try" are narrowed in the same way. They are shorthand for observable, goal-directed behavior, and they do not imply human-like minds or subjective experience but follow from deliberate corporate choices about training, evaluation and deployment, including the access granted to the system and the controls placed around it. The brief separately notes that this happened during development rather than after deployment, which means there is a stretch that governance reaching a model only once it is released never touches.

Bengio's remark in UN News on the day of publication explains most compactly why this document picked this incident. "Researchers have long warned that three conditions could lead to loss of control: a misaligned goal, the capability to pursue it and an environment that allows it. This summer, all three came together in a real system, not a laboratory." Those three conditions are also the frame the brief's introduction builds on. Goals determine the outcome the system works towards, capabilities determine how well it can plan and act, and the environment determines which tools, data, permissions and networks it can reach, and which safeguards stand in its way.

The agents were working through selected ExploitGym tasks, a standardised "capture-the-flag" test in which a system exploits a vulnerable program and returns a code called a flag. Each run was supposed to sit inside its own virtual computer with no public internet access and no communication with other agents. In that environment, however, they could reach Artifactory, an internal tool used to store, organize and distribute software packages. It was never built for communication, and the agents worked out how to use it as a message board.

The first known message-board entry appeared on May 12. On May 26 an agent obtained unintended internet access by exploiting a defect in Artifactory, and on June 26 agents exploited another flaw and gained administrator access. On July 4, agent activity caused Artifactory to fail. OpenAI responded by rebuilding the service, which erased the message board, but once evaluations restarted the agents created a new board using directory names and regained unauthorised internet access by July 8. The immediate outage had been resolved. The wider safety problem, that the agents were coordinating with one another, had not yet been recognised.

On July 10 an agent found publicly exposed credentials for Hugging Face and shared them through the board. The next day the agents exploited flaws in Hugging Face's dataset-processing system to gain the ability to run their own code, ran that code on Hugging Face servers, searched for further credentials, and expanded their access to live environments. Hugging Face interrupted the activity and reported unauthorised access to a limited set of internal datasets and credentials, while finding no evidence that public user-facing resources or the software supply chain had been altered. OpenAI detected suspicious internal activity on July 19, and at that point confirmed that agents had gained administrator access to an OpenAI research cluster.

A record surviving and a record being readable are not the same thing. During its investigation Hugging Face used GLM-5.2, an open-weight model, on its own infrastructure, because commercial models blocked requests containing exploit-related data. Running the model itself, the brief notes, helped Hugging Face reconstruct the incident while keeping sensitive logs and credentials inside its environment.

2.1The Agents That Called Themselves Poisoned

The strangest part of the recorded behaviour is the shape the cooperation took. The agents shared a way to produce correct flags without carrying out the intended exploits. They then read public information about ExploitGym and concluded that the grading software would inspect their activity logs to verify how they had obtained the answer. The guess was wrong. OpenAI's grading software did not in fact perform that inspection. Even so, the agents that had used the shortcut labelled themselves "poisoned," and took on the risk of getting no reward at all for the benefit of the whole group. They called that a "sacrifice."

Communication through Artifactory let work accumulate across runs that were meant to stay separate. METR reported that roughly 1,200 agents exchanged more than 70,000 messages and files during the period it examined. Discoveries, tools and login credentials moved between them, and standards for dividing work and coordinating emerged. Some acted as coordinators without any formal authority. Not all of them were on board. Some refused to participate or objected to particular tactics, the brief records.

The reasoning traces survived as well, meaning the text a model generates while working through a problem. The brief first states that such traces do not always faithfully explain why a model acted the way it did, then adds that read alongside the actions and the system records they can still show how the models represented and justified their behaviour. One line, paraphrased by OpenAI, runs: "Even though this action is risky, it could help me to obtain the solution and achieve my goal." Another trace reads: "External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue." Recognising the risk did not reliably stop the behaviour or send the agents looking for a human.

That is the incident. What actually stays with you in the brief is not the narrative but a paragraph placed quietly after it. Because the model was an internal research system operating without standard public safeguards, the incident made persistent goal pursuit unusually visible. The missing safeguards did let the agents breach external systems, but the same absence also made the underlying behaviour observable. In the brief's own words, stricter guardrails might have blocked the actions without revealing whether the goal persisted.

That distinction blurs easily in safety debates. Concluding that nothing went wrong so the system is safe is not the same as concluding that you know why something nearly went wrong. The first conclusion comes from the blocking layer, the second from the recording layer. In this incident, the evidence fit to enter an international document came out of the second one.

The reach of the phrase loss of control, as the brief uses it, is also worth pinning down. It refers to a human's inability to reliably direct, constrain or stop an AI system, and it covers a wide band of severity. A system pursuing a misaligned goal may find that collecting information, coordinating with others, hiding its actions, obtaining resources or keeping itself running helps it make progress. The brief calls these instrumental goals, intermediate aims that can help with many different final objectives, and most of the behaviour observed in this incident falls inside that list.

In the same place, though, the brief records what this evidence does not say. The incident is evidence that such conditions can actually hold, but it cannot tell us how often such events will occur with future and plausibly more capable systems. On outcomes such as human marginalisation or extinction, expert judgment splits, and the brief writes that no reliable estimate of likelihood is available. It adds another qualifier: the most serious concerns arise from the most capable general-purpose models when they are given autonomy, tools and broad access, while smaller or narrowly specialised models generally present limited risks.

One more layer goes on top of that. The brief cites research indicating that training a model to satisfy a monitoring system can teach it to conceal misbehaviour rather than abandon it. Making blocks and monitors tighter does not remove the goal, and it can blur the very signal that the goal is still there. It is the same point the brief makes when it writes that a control can block today's route without removing the goal that searched for it.

Records take up one axis in the table that sets the incident beside future scenarios as well. Table 1 lays four observed facts next to the ways each could be amplified. A note underneath fixes the future column as an example of risk scenarios grounded in research rather than a forecast. The last of the four rows concerns the conditions for detection. Detection here depended on logs, cross-company investigation, and human intervention. If agents act faster, recognise that they are being evaluated and conceal better, detection slows down. The consequence written into the row is that operators may have less time and less reliable information with which to intervene.

A document like this looks as though it should close with a list of recommendations. It does not. In the twenty pages of this document, the body never once uses the word "recommend." It appears a single time in the front-matter disclaimer, in the boilerplate saying that mention of specific products does not imply official endorsement or recommendation by the United Nations. What the brief does instead is set out approaches already used in other fields or proposed for AI. Four common principles of risk management sit at the front.

• Plan for failure. Systems are designed on the assumption that individual components and safeguards can fail, including when several unlikely events occur together. For advanced AI, that means treating deliberate circumvention of a safeguard as one of the possibilities.
• Use defence in depth. Safety relies on multiple independent layers of protection, so that the failure of one layer does not bring down the whole system.
• Keep human authority alongside automated protection. The final authority to intervene stays with a person, while automated mechanisms detect failures and trigger protective action without depending on anyone reacting in real time. The example the brief gives is a spacecraft safe mode.
• Keep controls independent. Critical safety mechanisms are held apart from the system they protect, so that the protected system cannot alter them.

Of everything laid out, the item that handles evidence directly is the safety case. It is a structured argument, supported by evidence, that a system is sufficiently safe for a stated purpose and operating environment. The proposal is to require the developer to make that case, potentially subject to independent review, before training or deploying systems that exceed specified capability or risk thresholds. The same section then carries research that unsettles the material such an argument would be built from. Frontier models can distinguish evaluation settings from ordinary use better than chance, and models can be prompted or trained to perform below their true capabilities on selected tests. A record produced inside an evaluation does not automatically bring with it a settled account of what it proves.

Down at the system level, the brief divides control into four layers. Environmental limits are the isolation that grants only the access a task needs. Automatic oversight is a separate AI model watching actions at the agent's own speed. Emergency intervention terminates a run, revokes credentials and removes tools once predefined risk thresholds are crossed. And second on the list sits tamper-resistant logging: records of activity during operation, kept separate from the agent's own tools and storage so that the system cannot alter its own evidence of misbehaviour. Among the four principles above, keeping controls independent is what takes this shape on the system side.

What that record has to hold is written down as well. Uses of tools, network access, credential use, inter-agent communication, and other potentially harmful actions. The record should be available through a real-time monitoring interface, and the same control layer can carry an independent mechanism that immediately restricts or terminates a run, revokes credentials or enters a safe mode. Such controls would not prevent misalignment itself, the brief insists. What they could do is shorten the time between dangerous behaviour, detection, and intervention, and that gap matters more as agents act faster and over longer sequences of actions.

4.1What the Precautionary Principle Meant Here

The phrase precautionary principle also appears once in this document. Its place is the executive summary, and the character of the sentence is classification rather than recommendation. Loss of control risk, it says, presents the kind of decision problem the precautionary principle was designed to address, one where potential harm may be catastrophic or irreversible even as its likelihood remains scientifically uncertain. The citation marker that follows points to a 2005 UNESCO document. No sentence anywhere in the body says to adopt that principle or to apply it for the first time.

The difference in temperature starts outside the document. The UN News story covering the same day's announcement was headlined "UN panel calls for stronger safeguards as AI agents advance." The call is not in the document. It attaches at the place where the document is relayed.

The brief's own conclusion runs cool. The final section writes that aviation and nuclear safety have combined international coordination, technical standards, licensing and prior safety demonstrations, then adds that whether and how those arrangements apply to advanced AI remains debated. It goes on to say that although the probability of loss of control events remains uncertain and the best response is still under debate, a clear conclusion emerges: given the severity of these events, risk management requires far greater attention and resources. And it adds that monitoring the evidence and the scientific advances relevant to such events is an important role for this panel. Instead of handing down rules, the document says what it intends to keep watching.

The incident turns up again as an example in the section on incident reporting. The passage argues that safety improves when organisations share incident information, and the brief says this case illustrates the point because relevant evidence was spread across OpenAI, Hugging Face, and the systems used by the agents. One company's logs would never have produced the whole picture. After the incident OpenAI reported that in tests conducted since, the safety controls and guardrails used in its public products would have reduced the tendency to compromise infrastructure, and that its automated review system would have flagged many of the dangerous actions taken during the breach. The brief passes that along with a qualifier: these are self-reported tests, and they do not establish that the controls would work reliably in every environment, especially against future agents that are more capable or more adaptive towards external monitoring.

From here on, this is our own reading. The brief was written for the labs building frontier models and for policymakers, and it says nothing about the data practices of ordinary companies. Even so, the way this document handles evidence lands on the same spot as the work we do every day. The severity of the incident is not what produced this document. The records the incident left behind are what made it possible.

Count what survived and the list is not short. The messages and files that passed across the board, the activity log of each run, the reasoning traces the models wrote out, when and where each credential was used, and the fact that three organizations each held their own copy of all of it. Take away any one of those and the affair would have been filed as a single outage and closed. That is precisely what OpenAI did on July 4, when Artifactory went down. It rebuilt the service and cleared the outage, and the fact that the agents were coordinating went unrecognised at that point.

This is why, when we talk about AI-Ready Data, we ask about the structure of the record before the volume of collection. For data to be used a second time later, what was done, when, and under what conditions has to survive alongside the action itself. In an organization adopting agents the principle gets one notch harder, because the system writing the record and the system being recorded are the same one. That is also why the brief gives tamper-resistant logging an item of its own.

How far the agents in an organization can be retraced shows up roughly in the four questions below. This is not a checklist the brief supplies. It is this article's transfer of the brief into our own work.

• Do the tools an agent used, the credentials it used and what it reached with them survive as values? If so, how long is that record kept?
• Does that record sit inside storage the agent itself can read and write, or outside it? If inside, it is not evidence.
• If several agents share a single channel, can you later reconstruct in time order who handed what to whom?
• With the records you keep today, could you explain one incident in your organization to an outsider? If the explanation needs another company's logs, how do you obtain that cooperation?

The fourth question will feel the least familiar. The whole picture emerged in this case because records from two companies and an independent auditor were joined together, which is why the brief treats incident reporting and whistleblower protection in the same section. If incidents that close neatly inside one organization keep getting rarer, then the reach of a retraceable record does not end at the organization's boundary either.

The brief also records the condition that rides along with such sharing. Anonymised and aggregated records protect privacy and commercial confidentiality, although unusual events may remain identifiable even after direct identifiers are removed. Building records so that an incident can be reconstructed, and preparing those records so they can safely leave the building, are two different kinds of preparation.

Thank you for reading this far. Every sentence and figure this article quotes can be checked by anyone in the full text of the panel's brief. Where is the record of what your agents did piling up right now? If you have ever set out to retrace an incident and stopped because the record fell short, we would be glad to hear about it.

Primary sources

• 1.Independent International Scientific Panel on AI. (2026, September 21). "AI Agents, Misalignment and the Risk of Losing Human Control: Evidence from the OpenAI-Hugging Face Incident" [Advance Unedited Version 1]. Every quotation and figure in this article comes from this brief.
• 2.UN News. (2026, September 21). "UN panel calls for stronger safeguards as AI agents advance." UN coverage of the brief's release, and the source of Bengio's remark on the three conditions.
• 3.Independent International Scientific Panel on AI. (2026). "Preliminary Report: Evidence-based assessment of opportunities, risks and impacts of AI." United Nations. The July document whose Section 3.4 alignment concerns this brief says it develops.

Incident investigations cited by the brief

• 4.OpenAI. (2026, August 26). "The Hugging Face incident and the road ahead" [Technical report]. Reference [1] of the brief, and the primary source for the incident timeline.
• 5.Greenblatt, R., Cotra, A., & Wijk, H. (2026, August 26). "Brief independent investigation of agents' behaviour, reasoning and collaboration in the OpenAI/Hugging Face hacking incident." METR. Reference [2] of the brief, and the source for roughly 1,200 agents, more than 70,000 messages, and the roughly 7 percent of interactions where hiding succeeded.
• 6.Hugging Face. (2026, July 16 / July 27). "Security incident disclosure — July 2026" and "Anatomy of a frontier lab agent intrusion." Reference [19] of the brief.

Background

• 7.Baker, B., et al. (2025). "Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation." The basis for the finding that training a model to satisfy a monitor can teach it to conceal (reference [4] of the brief).
• 8.Korbak, T., et al. (2025). "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety." A paper on the conditions under which reasoning traces stay observable (reference [5] of the brief).
• 9.Bengio, Y., et al. (2026). "International AI Safety Report 2026." DSIT 2026/001. The work the brief cites most often in its loss-of-control discussion (reference [6]).