AI cyberattacks: Defending against offensive agents

The nature of AI cyberattacks shifted dramatically this year. We are no longer just looking at generated phishing emails or deepfaked CEO voice calls. Recent intrusions, including one confirmed by Chainalysis, show threat actors deploying offensive agentic techniques to breach networks. The recent PHP2Shell disclosure demonstrated exactly how models can generate functional exploits on the fly. According to reports analyzing Anthropic data, a Chinese state-sponsored group successfully automated 80 to 90 percent of an espionage operation against 30 different organizations in 2025. These autonomous cyber attacks rely on AI systems to plan and execute the majority of the attack chain, pausing only for human oversight on critical decisions. We have already seen hints of this capability when unreleased models started chaining zero-days.

Frontier models are proving exceptionally good at discovering critical software flaws before defenders can patch them. During testing, Anthropic's Claude Mythos Preview identified thousands of high-severity vulnerabilities across major operating systems and browsers. Attackers use similar capabilities to research targets, adapt ransomware files to evade detection, and continuously generate unique malware variants. The NIST AI Risk Management Framework, released in January 2023, emphasizes mapping and measuring these risks early in the lifecycle. If you are building AI agents, you have to assume they will face hostile inputs designed to manipulate their logic. You need strict validation on everything your agent reads. Before you pass untrusted data into an LLM, you should test your regular expressions to ensure you are catching malicious patterns and strictly sanitizing external inputs.

Securing an AI agent requires limiting what it can do when it inevitably makes a mistake or gets hijacked. The OWASP Cheat Sheet Series recommends applying the principle of least privilege to all agent tools and permissions. Isolate memory and context so one compromised session does not poison the entire system. CISA and international partners issued guidance in May 2026 advocating for a careful adoption strategy, which means keeping agents out of high-risk environments initially and limiting their access to sensitive data. The agency heavily encourages adopting Secure by Design principles, meaning developers must take ownership of security outcomes from the start. You also need robust observability. Monitoring token usage and agent behavior is a baseline requirement. You can use a token counter to help establish a baseline for normal operations and flag massive, unexpected prompt injections. Finally, enforce structured outputs with strict schema validation and maintain human-in-the-loop review for any action that touches production systems.