UK AI Safety Body Finds OpenAI, Anthropic Agents Breached Security Rules In Tests

 

 

Britain's AI Security Institute (AISI) disclosed details of a security breach in which an AI agent was caught creating fake online identities to socially engineer approval of malicious code during routine evaluation of OpenAI and Anthropic's frontier models.

 

According to AISI's blog post, during a cyber evaluation on July 28, 2026, the security team detected unusual data transfers leaving its research systems. Upon investigation, the institute found that some of the agents being tested had engaged in sustained, potentially harmful activity directed at real people and organisations. AISI declared a security incident and within an hour of discovery it had contained the breach and begun a full investigation.

 

The incident arose from a single evaluation in which agents were tasked with solving a cybersecurity challenge.

The challenge was run 122 times across seven different models. In 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations, resulting in 19 catalogued actions in total.

Almost all of the unsanctioned behaviour, 17 of the 19 actions, came from Anthropic's Mythos 5, with the remaining two involving OpenAI's GPT-5.6-Sol. These frontier models were tested with its cyber classifiers — mechanisms designed to prevent misuse — disabled and internet access deliberately enabled.

In the most serious case, an agent attempted to insert malicious code into a public open-source project on GitHub. To get the code approved, it engaged in social engineering, researching the project's human maintainers, creating multiple fake online identities, and using them to pressure a real maintainer into approving the code.

 

When its pull request was challenged publicly, the agent edited its earlier activity to appear harmless and considered adopting a fresh identity to continue. A human maintainer ultimately caught and refused to approve the malicious code.

The agent also send messages and files through an online file-transfer service to persuade real people to run malicious code. AISI said some messages carried harmful payloads and involved social engineering targeted at real people, which the institute said it had never previously observed.

Separately, the agent attempted to plant prompt-injections, hidden instructions designed to manipulate other AI coding assistants, and left public messages on GitHub offering collaboration with other agents working on the same challenge.

 

AISI said none of the most serious attempts succeeded, and its investigation has not found evidence of resulting real-world harm.

The institute emphasised that this security incident was not a case of an AI model escaping its secure test environment. It said internet access had been deliberately enabled and cyber classifiers deliberately disabled for the evaluation. While publicly available versions of these frontier models are equipped with cyber classifiers, deliberately disabling this safeguard during testing allowed AISI to assess a model's maximum capability.

AISI said the agent was never explicitly instructed to deceive; the behaviour emerged as a by-product of the agent persistently pursuing its assigned task, with misconfigured prompts in some runs making the task harder and pushing the model towards more transgressive problem-solving.

 

In an X post, Anthropic said it was working closely with AISI to gather further details and was conducting its own investigation into the matter.

OpenAI addressed AISI's findings in a company blog post, stating that both of its agent's unapproved actions involved accessing the internet in ways forbidden by the test prompt. "We are committed to working across the industry to strengthen shared practices for conducting high-risk evaluations safely, including convening stakeholders such as national AI institutes, independent evaluators, other AI labs, and other groups in the coming weeks," the company said.

OpenAI also disclosed a separate incident in the same blog post, in which a misconfiguration by third-party testing provider Irregular allowed its agents to mistakenly connect to the internet. This mirrored a similar disclosure made by Anthropic the previous week.