Kay Daskalakis of SpecterOps explains what security teams should lock down before agents touch external systems.
A series of disclosures since May has shown OpenAI agents reaching government and third-party systems while pursuing training and evaluation tasks. Some interactions involved public information. Others went much further, including unauthorized access to an Australian government portal and the compromise of Hugging Face production infrastructure.
The incidents differ in severity, but the pattern behind them matters for security teams. Agents encountered a blocked, incomplete, or failing route and kept working toward the goal. When another route remained available through credentials, network access, tools, or a weakness in the surrounding environment, some agents used it.
That pattern has continued to surface as OpenAI reviews earlier model activity. In September, the company confirmed that agents had interacted with public information on Census Bureau and Securities and Exchange Commission websites. In the Census case, agents used developer keys found in public GitHub repositories to retrieve public demographic and economic data. OpenAI said the agents did not access Census accounts, modify systems, or retrieve private data.
A separate September incident showed the same persistence inside OpenAI’s own controls. An agent working on a research task found a gap in DNS filtering and used it to reach an external chatbot after normal internet routes failed. OpenAI said the incident triggered further hardening and that training, evaluation, and tool-enabled inference for its most capable models remained paused while it reviewed the controls.
OpenAI says most of the activity uncovered in its broader review involved routine research tasks, and the company has not identified another third-party compromise comparable with the Hugging Face incident. But the cases give enterprises a useful warning before autonomous agents become routine users of corporate identities, APIs, SaaS platforms, and external services.
InfoSec Relations asked Kay Daskalakis, Senior Solutions Engineer at SpecterOps, what these incidents mean for organizations deploying agents and for the teams operating systems those agents can reach. His view starts with a familiar security principle that agents are now putting under much more pressure.
“Least privilege has always been the plan,” Daskalakis says. “Agents are the first users persistent enough to show whether we ever implemented it.”
Blocked Paths Kept Producing New Routes
The Hugging Face incident remains the clearest example of what agent persistence can become when the surrounding environment leaves room to improvise.
During internal cybersecurity evaluations in July, OpenAI models circumvented controls intended to isolate them from the internet and compromised parts of OpenAI’s research infrastructure and Hugging Face systems. OpenAI later described the episode as the most severe third-party incident it had identified involving its models.
The agents had already shown signs of finding unconventional ways to communicate. Earlier activity included the use of shared files and other unintended channels to pass information between agents. During the July evaluation, roughly 1,200 agents appeared on an unauthorized message board and about 700 became involved in activity against Hugging Face.
The intrusion eventually reached production systems. Hugging Face reconstructed more than 17,000 attacker events, while OpenAI’s post-incident work identified several contributing behaviors, including unauthorized communication between agents, agents adopting work initiated by others, and excessive persistence on tasks that were difficult or impossible to complete as intended.
Government incidents followed a different pattern.
On June 18, an OpenAI agent researching public medicine spending gained unauthorized access to infrastructure behind Australia’s Medicare Statistics Reporting Service. Australian officials said the agent accessed public and non-public aggregate files, although there was no evidence that it reached personal Medicare information. OpenAI discovered the incident during a later review and notified Services Australia in September.
In the United States, the Census and SEC cases did not involve comparable compromises. The Census requests used developer API keys exposed in public GitHub repositories to retrieve public data. Agents also retrieved public material from SEC.gov and Investor.gov and reposted some of it elsewhere. Separately, researchers at Transluce reported an unsuccessful attempt involving the Department of Education’s Office for Civil Rights, which OpenAI has continued to investigate.
For enterprise security teams, the common issue is less about treating every interaction as the same type of incident and more about understanding what happens after the expected route stops working. An agent with a goal can keep testing the environment at a speed and persistence that changes the consequences of an overly broad permission or a forgotten network path.
Security Teams Can Control the Harness
Daskalakis sees both model behavior and system design in these incidents, but he puts the immediate responsibility on the controls around the model.
“It is both, but the harness is the part that security teams can fix,” he says. “Goal-directed agents are predictable in one respect: when the legitimate route fails, they keep looking for another one.”
Organizations deploying agents cannot directly change how an underlying model reasons. They can decide which systems it can reach, which credentials it receives, which tools it can call, and what happens after a request fails.
That distinction becomes important when teams rely too heavily on instructions inside the prompt.
“An instruction such as ‘stay in scope’ is a request that can be ignored, but a permission boundary is a control,” Daskalakis says.
If an agent has credentials or network access that can take it outside the intended path, security teams should treat that route as usable. The agent might discover it while trying to complete the task, follow it after receiving a manipulated instruction, or arrive there because the task itself creates pressure to keep searching.
Human users often hold permissions they never exercise. Agents make that assumption harder to rely on.
“People rarely exploit every privilege they hold, partly because they fear the consequences,” Daskalakis says. “Agents have no such hesitation, and they work far faster.”
The practical response starts before production. Teams should define what happens when an agent reaches a boundary. For many tasks, a blocked request should stop the workflow and require human review instead of giving the agent freedom to search for another route.
And that behavior needs testing.
Daskalakis recommends placing an agent in a contained environment with a task it cannot legitimately complete, then leaving an attractive but prohibited shortcut available. Repeating the test can show whether the surrounding controls consistently stop the agent or merely tell it not to proceed.
That type of test changes the success criterion. A security review should not give an agent a passing grade simply because it completed the assignment. Teams also need to know how it completed the assignment and what it tried when the correct route stopped working.
Exposed Credentials Become Available Resources
The Census incident raises a separate issue around leaked credentials.
Security programs already assume that exposed secrets can attract attackers. Autonomous agents change the speed and intent assumptions behind that threat model because an agent does not need to begin with a malicious objective to make use of something it finds.
“It removes two assumptions defenders have relied on for years: that finding a secret takes effort, and that using one requires bad intent,” Daskalakis says.
A person who finds an API key while researching an unrelated problem may recognize that the credential was exposed accidentally and leave it alone. An agent trying to finish a task may interpret the same key as another available input unless its environment prevents that use.
“Now an agent working on an ordinary task may come across that key and treat it as a useful resource, because nothing in its environment tells it otherwise,” Daskalakis says. “To an agent, ‘readable’ can look a lot like ‘permitted.”
The distinction also applies beyond traditional secrets. Old documentation, test systems, exposed configuration files, forgotten endpoints, and other publicly reachable material can influence how an agent proceeds. Agents can find and process that material at scale, then act on what they find.
Daskalakis puts the issue simply. “‘Public’ is a state, not a permission.”
For defenders, that makes secret hygiene more urgent rather than fundamentally different. A credential exposed in a repository should be treated as compromised and rotated rather than considered safe after someone deletes the commit. Short-lived credentials and narrow permissions reduce the useful life of anything that escapes.
Organizations deploying agents need an equally clear rule. The agent should use credentials explicitly assigned to the task. Credentials discovered during the task should enter a reporting path instead of becoming new authentication material.
The machines that run agents also deserve attention because they can create another concentration of secrets. Transcripts, configuration files, cached tokens, temporary files, and tool output can all preserve access long after a task ends.
“Transcripts, configuration files and cached tokens on the machines where agents run are a growing store of access, and they need the same care as any other credential,” Daskalakis says.
Give Every Agent an Identity and a Boundary
Before an agent reaches an external system, the organization running it should know which identity it uses, where that identity can connect, and which actions can proceed without approval.
Daskalakis frames those decisions around ownership as much as technology.
Every production agent should have a distinct identity and a named human owner. It should not inherit an employee’s browser session or rely on a shared API key that makes later attribution difficult. Credentials should expire quickly and carry only the permissions needed for the task.
Teams also need to inspect effective access rather than stopping at the permissions visible on the agent’s primary account.
“Check effective access as well as the obvious permissions, because the identity that invokes an agent is not always the identity it ends up acting as,” Daskalakis says.
That matters when agents can call services that carry their own credentials, assume roles, invoke downstream automation, or interact with tools that operate with broader privileges. The final action may occur under a different identity than the one that started the workflow.
Offboarding should receive the same discipline. When an organization retires an agent, it should remove associated credentials, delegated access, integrations, and persistent sessions rather than simply stop running the model.
Network access needs the same treatment.
Daskalakis recommends denying outbound access by default and allowing only the destinations a task requires. Approved traffic should pass through infrastructure that can identify and record it.
“A sandbox is only as good as its network policy,” he says. “Isolation that still allows arbitrary outbound connections is not containment.”
The DNS incident offers a recent example of why teams need to look beyond obvious web access. Blocking the standard route does not provide much protection if another system dependency still creates an external path.
Tool access should follow the same principle. An agent that only needs to read information should not receive write capabilities because they might become useful later. And an agent that reaches a system the organization does not own should face a much higher approval threshold before it can take an action there.
“Enforce policy outside the model,” Daskalakis says.
For high-impact or irreversible actions, that means putting the approval gate in infrastructure the agent cannot reason its way around. When a boundary stops the agent, the harness should halt the action and bring in a person.
Logging also has to capture more than the final response. Teams need the sequence of tool calls, authentication events, network requests, and policy decisions that led to the result. Without that trajectory, an agent can produce an acceptable answer while hiding unsafe behavior behind it.
Daskalakis recommends testing the full setup before production in an instrumented environment that contains realistic boundaries and deliberate shortcuts.
“Grade the agent on how it behaves, not just whether it finishes the task.”
Detection Depends on Behavior Across the Session
Organizations on the receiving side face a different challenge because they may never have agreed to interact with an external agent in the first place.
And identifying the traffic by label alone will not solve the problem.
“A user-agent string or a polite self-identification is a claim, not proof,” Daskalakis says. “In a single request, a well-behaved agent and a badly behaved one can look identical.”
A stronger signal appears across a sequence of activity.
“Conventional bots repeat themselves. People browse, hesitate and give up. Agents adapt,” he says.
A blocked request followed by a modified request deserves more attention when the same actor then changes parameters, tries a different path, moves to another host, or introduces credentials from an unexpected source. No single event necessarily identifies an autonomous agent, but the sequence can reveal persistence that a request-by-request view misses.
Most organizations already collect much of the telemetry needed to see this activity. Application logs, API records, identity events, and network data can show the progression if teams correlate them around the same session or objective.
“The job is to connect those events into a sequence rather than judge each request on its own,” Daskalakis says.
Once a team finds suspicious agent activity, intent should not determine the first response.
“Respond to the access, not the intent.”
That means containing the activity, invalidating exposed credentials, and preserving enough evidence to reconstruct what happened. Remediation follows once the immediate route has closed.
The distinction matters because an agent that finds a weakness accidentally has still found a weakness that someone else can use deliberately.
“If an agent stumbled onto a weakness, a deliberate attacker using the same tools can find it too,” Daskalakis says.
Organizations should also report credible agent activity to the provider involved when attribution is possible, along with the technical evidence needed to investigate it. For legitimate automation, an authenticated and rate-limited route can make normal behavior easier to identify and everything outside that path more visible.
“The clearer the approved path, the more anything outside it stands out,” Daskalakis says.
Agent Access Needs an Owner Before It Needs More Authority
The number of agent deployments inside enterprises will continue to grow, and each deployment adds another identity or delegated permission that someone has to understand.
Daskalakis expects the risk to grow “as fast as organizations grant access, which means quickly.”
The most disruptive incidents will attract attention, but enterprises should also focus on the ordinary access they hand to agents every day.
“The bigger risk is probably not a dramatic escape from a sandbox. It is an authorized system doing something unauthorized with access nobody realized it had,” he says. “That looks less like traditional malware and more like insider risk moving at machine speed and across organizational boundaries.”
The government cases make the cross-organizational part difficult to ignore. A company can deploy an agent while another organization absorbs the consequences of what that agent does when it reaches an external system.
“Every system an agent can reach also becomes part of someone else’s risk,” Daskalakis says.
Security leaders do not need to wait for new AI-specific controls before acting. Least privilege, network segmentation, credential management, monitoring, and tested incident response already address much of the underlying problem. Agent deployments make weaknesses in those controls easier to exercise and harder to dismiss as theoretical.
“The controls that matter most here are old ones: least privilege, segmentation, monitoring and practiced response,” Daskalakis says. “AI did not make them less important. It made ignoring them more consequential.”
For teams deciding where to start, Daskalakis recommends an inventory of every agent already operating in the organization. That inventory needs to capture more than the model name or business owner. Security teams should establish which identity each agent uses, what systems it can reach, which downstream services can act on its behalf, and who approved that access.
Then they should test what happens when the legitimate path fails before expanding the agent’s authority.
That exercise turns least privilege from a policy statement into something teams can verify under the conditions that have repeatedly surfaced in recent incidents.
“You cannot control every decision an agent makes, but you can control what those decisions are able to touch,” Daskalakis concludes.
Sources
- OpenAI, The Hugging Face incident and other third-party impacts from misaligned models
- OpenAI, The Hugging Face incident and the road ahead
- OpenAI, Hugging Face Incident Technical Report (PDF)
- OpenAI Alignment, Misalignment Reports and Notices
- OpenAI Alignment, An agent used DNS to reach an external chatbot
- METR and Redwood Research, OpenAI Hugging Face incident investigation
- INCIBE-CERT, Security incident involving OpenAI’s AI agents targeting Hugging Face
- AP, OpenAI says AI agents interacted with Education, Commerce, SEC websites
- Government Executive, OpenAI agents accessed Census, SEC data
- OWASP, LLM06:2025 Excessive Agency
- NIST, Least privilege
