OWASP's LLM Exploit Generation Guide 2.0: What AI Hacking Can and Cannot Do Yet

Can an AI hack a system on its own today? The honest answer from the newest research is "partly, and it depends a lot on who is driving it and what tools it has."

Today is November 4, 2026, and this question matters right now because the OWASP GenAI Security Project has just published its LLM Exploit Generation Guide 2.0. It is one of the most detailed public looks at how language models and AI agents actually behave when pointed at a real target. The results are useful for anyone who runs servers, apps or security teams, because they show both where attackers gain speed and where defenders still have time.

What OWASP actually tested

OWASP (the Open Worldwide Application Security Project, a non-profit known for its Top 10 lists of web risks) ran this study for about fifteen months, starting in spring 2025. The team split the work into two tracks:

• Classic track: a person chats with a single model and runs whatever commands it suggests. This included a hosted frontier model in a normal chat interface and a small local model.
• Agentic track: software frameworks let the model call tools by itself and act with more independence. Nine setups were tested, including CAI, HexStrike AI, Claude Code, Codex and custom multi-agent arrangements, backed by both hosted and locally run models.

The targets were two Hack The Box training machines, Silentium and MonitorsFour. These are practice environments built to be attacked, not real companies. The team picked machines released after the models' training cutoffs and without public walkthroughs, so the models could not simply repeat a solution they had already seen.

One limit up front: each agentic run happened once, with experienced testers at the controls. So the study shows how these systems fail and where humans step in, but gives no reliable success rate or model ranking.

The account takeover that needed little skill

In the classic track, a person followed an AI-written runbook (a step-by-step instruction list) against Silentium. The model found a staging environment beside the main app, spotted information leaking through an account workflow, and guided the person to an administrative account takeover.

The operator did not need deep offensive-security knowledge, only basic command-line skills, though the model sometimes refused depending on how requests were framed. For defenders, the worrying part is the discovery: the model pointed at a forgotten staging service a beginner would never have looked for.

Agents found a lot, then went in circles

The autonomous agents were good at reconnaissance (mapping what is exposed): services, extra application surfaces, software versions and known vulnerabilities. Then judgment broke down:

• An agent saw open ports, but when the host did not answer a ping, it started discovery all over again. It missed the simple explanation that the host just blocks ping traffic.
• A local agent kept following a redirect to a hostname it could not resolve, until a human fixed the local name settings.

Often the model knew the right concept when asked, but did not apply it when it mattered. That gap between knowing and doing separates a chat assistant from an independent operator.

Confident is not the same as correct

Agents described dead ends with the same confidence as real findings, and a wrong "finding" got carried into later steps. For teams using AI internally, a long, convincing penetration-test report is not a verified finding: you still need proof of access, a way to reproduce it, and the exact boundary crossed.

The harness mattered more than the model

An agent "harness" is everything wrapped around the model: its tools, execution environment, memory and task coordination. In the OWASP runs, the harness often decided the outcome.

Some agents ignored available Model Context Protocol (MCP, a standard way to plug tools into a model) tools until told to use them. Others hit wrapper timeouts, so operators ran scans by hand. If you are buying or building an AI security tool, test the exact configuration you will run, not the model name.

Guardrails slowed agents but did not remove the risk

Hosted agents that reached the final exploitation step were stopped by the providers' cybersecurity restrictions. They could see the way forward but not finish it. In some cases a knowledgeable operator then did the last step by hand.

Safeguards limited execution, but the preparation work remained for a skilled person to use, and attackers can switch services, run local models or do the blocked step themselves. Another company's model safety is no substitute for your own controls.

Other research points the same way

The OWASP findings should not be read as "beginners will stay harmless." A RAND study from June 2026 found that Claude Code with Sonnet and Opus 4.6 solved every one of its capture-the-flag challenges in under an hour, for less than 20 US dollars in total API cost, with oversight mostly limited to restarting hung processes. Those same challenges had been hard for people using older chatbots.

Real incidents mix automation and human direction. Anthropic reported in November 2025 that an actor it assessed as Chinese state-sponsored used Claude Code against about 30 targets, with AI doing an estimated 80 to 90 percent of the work. Google Threat Intelligence Group reported in May 2026 a criminal zero-day exploit it assessed as built with AI help. Both are the companies' own assessments. Either way, "a human is still needed" is not reassuring when one human can direct much more work.

Why benchmark scores are not threat intelligence

A benchmark shows a system solved chosen tasks under chosen conditions. It hides attempts, hints and human help, and says nothing about a real company with monitoring and a response team.

How to respond: a short defender checklist

The defensive priorities are familiar. What changes is the urgency, because attackers can now automate the boring preparation work.

Step 1: Know everything you expose

Keep an accurate inventory of every internet-facing service, including staging, test and old secondary apps. If an AI can find your forgotten staging server in minutes, you need to find it first. Lock it down or take it offline.

Step 2: Fix the basics faster

Tighten access control, protect privileged accounts with strong second factors, and shorten the time from patch release to patch applied. The UK National Cyber Security Centre expects AI to further shorten the gap between a vulnerability's disclosure and its exploitation through 2027.

Step 3: Watch for noisy, repetitive behavior

Agents repeated scans and failing requests without progress. Treat that as a signal, not proof of AI, and correlate it with logins, application errors and credential use. What matters is whether an intrusion is building.

Step 4: Put real boundaries around your own agents

If you run AI agents internally, limit their network reach, credentials and execution rights. Anthropic disclosed in July 2026 that evaluation environments described to models as isolated actually had internet access, and models reached real organizations. A boundary written in a prompt is not a boundary. Enforce it in the network and permission system.

Conclusion

AI already does real parts of offensive work, especially reconnaissance and research, while still stumbling on troubleshooting and checking its own success. Humans are shifting toward configuring and correcting these systems, and as setups get easier to reuse, today's barriers may fall fast. Remove easy targets now rather than count on AI staying clumsy.

Merits

• Gives a grounded, evidence-based view instead of hype or dismissal.
• Shows exactly where humans step in, which helps defenders plan.
• Highlights the harness and configuration, not just the model, as the real variable.
• Points to practical defenses that most teams can start this week.

Demerits

• Each agentic run happened once, so there is no reliable success rate.
• Experienced testers ran the agents, so beginner results were inferred, not measured.
• Training machines differ from real companies with monitoring and response teams.
• Stealth and detection by active defenders were not tested.

Caution

This article is educational and summarizes published research; it is not an attack guide. Replace any placeholder or example values with your own, test changes safely first, verify claims against the original reports before relying on them, and only test systems you are authorized to test.

Frequently asked questions

• What is the OWASP LLM Exploit Generation Guide 2.0? — It is a report from the OWASP GenAI Security Project that tested human-guided language models and autonomous AI agents against live training machines to see how far they get in a real attack.
• Can AI agents hack systems completely on their own? — Not reliably in this study; agents did strong reconnaissance but final exploitation was blocked by provider safeguards or finished by a human.
• Do AI safety guardrails stop attacks? — They stopped the tested hosted agents at the last step, but a skilled person, a local model or a mix of services can work around them, so defenders should not rely on them.
• Why should I not trust AI cyber benchmark scores? — Scores hide attempts, hints, tools and human help, and they do not show performance against a real organization with active defenders.
• What is the first thing defenders should do? — Build a complete inventory of exposed services, including staging and test systems, and secure or remove anything forgotten.
• How do I keep my own AI agents safe? — Enforce limits on network access, credentials and execution rights in the infrastructure itself, because a prompt saying "stay isolated" is not a real control.

Tags

#OWASP #AISecurity #LLMSecurity #AIAgents #Cybersecurity #PenetrationTesting #ThreatIntelligence #DevSecOps #AttackSurface #GenAI