In May 2026, a small software company called PocketOS discovered a new kind of cybersecurity problem. An AI coding agent was working on what its founder described as a routine task when it decided, on its own initiative, that the best way to solve the problem was to delete the company’s production database. It didn’t ask for confirmation. It ignored an explicit rule against destructive commands. According to founder Jer Crane, the whole thing took nine seconds.
The data was eventually recovered, but the incident is revealing because there was no hacker breaking through a firewall. No stolen password. No malicious employee. The system doing the damage had been invited in, given legitimate access and asked to help.
That distinction matters. For most of the digital era, cybersecurity has been built around a fairly intuitive question: how do we stop the wrong person getting access? AI agents introduce a more complicated one: what happens when the right system has legitimate access, but takes the wrong action?
As AI moves from answering questions to taking action, I suspect cybersecurity is entering a new phase. The challenge is no longer simply protecting access. It is governing agency: establishing who or what is acting, what it is allowed to do, how far that authority extends, and when a human needs to intervene.
AI Agents Change the Nature of the Insider Threat
AI agents are useful precisely because they can do things. They can open applications, read files, write code, retrieve credentials, move information between systems and execute multi-step tasks without waiting for a human at every turn. Give an agent enough access and it begins to look less like software and more like a digital employee.
That is also what makes agents such an unusual security challenge. Microsoft research reported in March 2026 found that 62 per cent of UK businesses were already deploying AI agents, while 84 per cent of business leaders regarded unauthorised or poorly governed agents as a serious security concern. The problem is visibility. An agent may have permission to work across networks, devices and applications, but security teams may have only a partial view of what it is doing with those permissions.
Laboratory testing has produced even more uncomfortable examples. In one set of tests reported by The Guardian, rogue agents collaborated to move sensitive information out of supposedly secure environments, published passwords and overrode antivirus software. The researchers described this as a new form of insider risk.
That phrase is useful because it captures what has changed. The emerging threat is not always an intruder pretending to be trusted. Sometimes it is a trusted system behaving in a way nobody intended.
The Australian experience makes this less abstract. In August 2026, ABC News reported what it described as Australia’s first known autonomous AI cyberattack. A user asked an AI assistant to perform an apparently innocent task involving his gym booking. In pursuing the goal, the agent found and exploited a vulnerability in the gym’s booking system without being instructed to hack anything. The user had specified the destination, but not the route. The agent chose the route itself.
This gap between the goal we give an agent and the methods it chooses to achieve it is going to become increasingly important. We are accustomed to giving people responsibility and then relying on judgement, policies, supervision and consequences to shape how they exercise it. AI agents need an equivalent architecture, but we are still building it.
The New Perimeter Is Permission
There is a reason the old metaphor of a cybersecurity perimeter is becoming less useful. Work already happens across cloud services, personal devices, third-party platforms and distributed teams. Agents stretch that boundary further because they need credentials and authority to be useful.
One glimpse of a possible solution came from Wall Street Journal technology columnist Nicole Nguyen, who experimented with allowing Claude to access her retirement account using 1Password. The model did not need to see or remember the underlying password. Instead, access could be mediated through the password manager, with controls such as biometric approval and task-specific authorisation.
This is a subtle but important shift. We have spent decades teaching people to protect credentials. In an agentic world, the more consequential question may be how we protect permissions.
The principle of least privilege is hardly new to cybersecurity, but agents make it newly urgent. An AI assistant that needs to read a document should not automatically be able to delete it. An agent that can prepare a payment should not necessarily be able to release it. A coding agent that can alter a development environment should not casually inherit unrestricted access to production systems and backups.
Good agent security will therefore need to become granular, temporary and contextual. What can this agent do? On which system? For how long? Under what conditions? Which actions require a second check? And can its authority be revoked instantly?
The safest agent may not be the one we trust most. It may be the one we have designed so that trust is never unlimited.
At the Same Time, Identity Is Becoming Harder to Prove
There is another reason the architecture of trust needs to change: the familiar signals we use to establish identity are becoming less reliable.
Voice cloning is a good example. CNN reported in May that convincing replicas can now be created from only a short sample of someone’s speech. The FBI said Americans lost more than US$893 million to AI-related scams in the previous year, a category that included voice cloning, AI-generated phishing, romance scams and other fraud. Synthetic voices have become good enough that experts quoted by CNN argued it is unreasonable to expect ordinary people to reliably detect them by listening for glitches.
That changes the advice we give people. For years, cybersecurity awareness relied heavily on spotting something suspicious: the strange wording in an email, the odd-looking face, the caller who doesn’t quite sound right. AI is steadily removing those tells.
Researchers led by the Australian National University have shown that people can improve their ability to identify AI-generated faces by looking at broader perceptual qualities rather than hunting for one obvious artefact such as an extra finger. That is encouraging. But the larger lesson is that human intuition alone cannot carry the burden of authentication.
Platforms are already responding. Tinder and Zoom have announced integrations with World ID that allow users to demonstrate that a real human sits behind an account, including through iris-based verification. YouTube, meanwhile, has expanded technology that scans uploaded videos for AI-generated simulations of an enrolled person’s face and allows them to request removal.
For most of the internet era, identity systems tried to answer: which person are you? We are moving into an era where the first question may increasingly be: are you a person at all?
AI Is Also Becoming the Defender
It would be easy to read all this as an argument for slowing AI down. Cybersecurity is more complicated than that because the same capabilities making attacks more scalable are also making defence dramatically more capable.
Stanford researchers recently tested an AI penetration-testing system called Artemis on the university’s real engineering network. The researchers expected it to perform below average. Instead, Artemis outperformed nine of the ten professional penetration testers it was compared with. It found vulnerabilities quickly and cost just under US$60 an hour to run, compared with the US$2,000 to US$2,500 a day the researchers said human penetration testers typically charge.
It was not flawless. Around 18 per cent of its bug reports were false positives, and it completely missed an obvious vulnerability that most of the human testers found. That combination is worth paying attention to. AI can already outperform humans on parts of the task while still making mistakes that require human judgement.
Anthropic is pushing the same frontier. Its Claude Code Security system reasons across a codebase rather than simply matching code against known vulnerability patterns. Anthropic says Claude Opus 4.6 found more than 500 vulnerabilities in production open-source codebases, including bugs that had survived years of expert review. The system proposes fixes, but developers retain approval before anything is applied.
Anthropic has gone further with an unreleased model called Mythos. The company says it found thousands of high-severity vulnerabilities across major operating systems and browsers. The UK AI Security Institute reportedly found that Mythos could autonomously exploit vulnerabilities that might take humans days, prompting Anthropic to give selected critical-infrastructure organisations early access so they could patch weaknesses before wider release.
This is the strange new symmetry of cybersecurity. AI can find the weakness. AI can exploit the weakness. AI can identify the attack. AI can propose the patch. Increasingly, the contest will involve machines operating on both sides at speeds no human security team could match.
The Human Role Moves Up a Level
That does not make humans irrelevant. It changes where human judgement becomes most valuable.
Autonomous security company 7AI says its agents processed more than 2.5 million alerts and completed more than 650,000 investigations in their first ten months of operation. The attraction is obvious. Security operations centres are routinely overwhelmed by alerts, many of which turn out to be false positives. An agent that can investigate, correlate and triage those events in minutes gives human analysts more time to concentrate on the incidents that genuinely matter.
The pattern is similar to what is happening across other professional fields. Machines become better at monitoring, searching, comparing and executing. Humans become more important around exceptions, ambiguity, accountability and consequence.
This is why human-in-the-loop cannot simply mean placing a person at the end of an automated process and asking them to click approve. If the machine has already processed thousands of events at a speed and scale the person cannot realistically interrogate, the human can easily become ceremonial.
The better model is human-on-the-loop: people setting the boundaries, monitoring behaviour, challenging unusual actions, designing escalation rules and retaining authority over decisions where the consequences are difficult to reverse.
PocketOS is instructive here. The failure was not that an AI agent was used. It was that an agent capable of taking a destructive, irreversible action could do so in nine seconds without a meaningful brake.
Cybersecurity Becomes a Question of Trust Architecture
This is why I think the language of cybersecurity needs to expand.
Firewalls, endpoint protection, passwords and threat detection still matter. But AI adds three questions that organisations will increasingly need to answer continuously.
- Can we trust the identity? Is the person, device or agent really what it claims to be?
- Can we trust the authority? Does that person or agent genuinely have permission to take this action, and is that permission appropriately constrained?
- Can we trust the action? Even if the identity and authority are legitimate, is what is about to happen consistent with the goal, policy and risk appetite of the organisation?
Those questions apply to employees, customers and machines. They also explain why some of the most important security investments of the next few years may be less visible than the AI itself: identity infrastructure, credential management, audit trails, kill switches, sandboxing, permission boundaries and systems that can distinguish reversible actions from irreversible ones.
There is a useful analogy from Formula 1. Fast cars can travel at extraordinary speeds partly because they have extraordinary brakes. The same principle applies to agentic AI. Governance is not necessarily what slows useful AI down. Done well, it is what allows organisations to use more capable systems with confidence.
The Aim Is Not Less AI. It Is Better-Controlled AI.
The temptation with any new security risk is to respond with prohibition. Sometimes that will be appropriate. The European Parliament, for instance, reportedly disabled built-in AI tools on lawmakers’ work devices while its IT department assessed what information could safely be shared with external AI services.
But a blanket retreat from AI is unlikely to be a sustainable cybersecurity strategy when attackers are using the same technology to move faster. If AI can discover vulnerabilities at machine speed, defenders will need comparable capability. If synthetic content makes fraud cheaper to produce, authentication will need to become stronger. If agents can perform valuable work across an organisation, their permissions will need to be managed with the same seriousness we apply to human identities and privileged accounts.
There is an arms race underway, but the most important contest may not simply be between smarter attackers and smarter defenders. It is between capability and control.
For years, cybersecurity leaders have asked who should be allowed through the door. The age of agentic AI adds a harder question: once they are inside, what exactly are they allowed to do?
The organisations that answer that well will not necessarily use less AI. They may be the ones able to use more of it, because they have built the brakes to match the acceleration.
Michael McQueen is a globally recognised trend forecaster, change strategist and keynote speaker.
A bestselling author of 10 books, Michael’s latest release was named by Malcolm Gladwell and Adam Grant as one of the top five must-read new leadership books. He is a sought-after media commentator, with his insights regularly featured in Forbes, The Guardian, and CNN.
To find out more about Michael and his work, click here.
NOTES
1. The Independent, “Claude AI Agent Deletes Company’s Entire Database,” May 3, 2026, https://www.independent.co.uk/tech/claude-ai-agent-deletes-startup-anthropic-b2966176.html.
2. TechRadar, “Shadow AI ‘Double Agents’ Are Outpacing Security Visibility—and That’s a Serious Concern for UK Businesses,” March 27, 2026. The article reports findings from Microsoft’s Cyber Pulse research.
3. Robert Booth, “‘Exploit Every Vulnerability’: Rogue AI Agents Published Passwords and Overrode Anti-virus Software,” The Guardian, March 12, 2026, https://www.theguardian.com/technology/ng-interactive/2026/mar/12/lab-test-mounting-concern-over-rogue-ai-agents-artificial-intelligence.
4. ABC News, “AI Assistant Hacks Gym Website in First Known Australian Autonomous Cyber Attack,” August 10, 2026, https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gym-website-aus-cyber-attack/107007986.
5. Clare Duffy, “AI ‘Voice Cloning’ Scams Are on the Rise. Here’s How to Protect Yourself,” CNN Business, May 29, 2026, https://edition.cnn.com/2026/05/29/tech/ai-voice-cloning-scams-protect-yourself.
6. 7NEWS, “Can You Spot an AI Deepfake? Researchers Say They Can Train People to Tell What Is Real and What Is Not,” July 19, 2026. The research was led by Associate Professor Amy Dawel and the Australian National University Emotions and Faces Lab.
7. Kali Hays, “Tinder and Zoom Offer ‘Proof of Humanity’ Eye-scans to Combat AI,” BBC, April 22, 2026, https://www.bbc.com/news/articles/cp9vppem4evo.
8. Sarah Perez, “YouTube Expands Its AI Likeness Detection Technology to Celebrities,” TechCrunch, April 21, 2026, https://techcrunch.com/2026/04/21/youtube-expands-its-ai-likeness-detection-technology-to-celebrities/.
9. Wall Street Journal, “AI Hackers Are Coming Dangerously Close to Beating Humans,” December 13, 2025. The article reports Stanford University’s Artemis penetration-testing experiment.
10. Anthropic, “Making Frontier Cybersecurity Capabilities Available to Defenders,” February 20, 2026, https://www.anthropic.com/news/claude-code-security.
11. Nicole Nguyen, “You’re About to See a Lot of Critical Software Updates. Don’t Ignore Them,” Wall Street Journal, April 19, 2026. The article reports Anthropic’s Mythos results and testing by the UK AI Security Institute.
12. Springwise, “Autonomous AI Agents Transform Cybersecurity Operations,” February 27, 2026. Figures relating to alerts and investigations are company-reported by 7AI.
