Less than one in ten of cybersecurity pros trust AI testing tools to find vulnerabilities, with over three-quarters say their AI vulnerability scanning tools missed critical flaws

Confidence in AI Security Testing Collapses

Less than one in ten cybersecurity professionals trust AI testing tools to find vulnerabilities, with over three-quarters saying their AI vulnerability scanning tools missed critical flaws. Fully automated testing is being replaced with a hybrid model, as "elite human expertise remains foundational."

Cobalt's 2026 State of Pentesting Report reveals that confidence in fully automated AI testing collapsed from 29% in 2025 to just 9% this year. A total of 78% of respondents saw automated tools miss critical vulnerabilities. LLM flaws proved particularly complex, with mean time to resolve (MTTR) rising from 19 to 36 days and most issues left unresolved.

Hybrid models surged to 47% adoption, as experts stress that automation should complement, not replace, elite human expertise in uncovering business logic risks.

Report Methodology and Key Findings

As the world praises Mythos and the Chinese rush to create their own variant, a report painting an entirely different picture comes from Cobalt. The cybersecurity company just published the Cobalt State of Pentesting Report 2026, based on two comparative surveys conducted in 2025 and 2026.

Polling around 450 cybersecurity professionals, Cobalt wanted to measure how confident the cybersecurity community is in automated AI testing for vulnerabilities. The answer: not that much.

• Last year, just below a third (29%) relied entirely on AI automation for testing.
• This year, the figure dropped to 9%.
• Cobalt suggests the key reason for such a steep drop is that 78% saw fully automated scanning tools missing critical vulnerabilities.
• Another key reason is the complexity of the AI attack surface the scanners are testing.

Context-Dependent Vulnerabilities

Roughly one in three findings from an AI pentest are rated "high-risk" - which is 2.7 times the average of conventional software. At the time of analysis, less than two-fifths (38%) of LLM vulnerabilities were fixed, while 62% remained open. Mean time to resolve (MTTR) for AI/LLM security issues rose from 19 days to 36 days.

"LLM vulnerabilities are deeply context-dependent and invisible to tools that lack an architectural understanding of the application," said Andrew Obadiaru, CISO of Cobalt. "To close the validation gap, automation should be deployed exactly where it excels, but elite human expertise remains foundational to uncovering and remediating the most complex business logic risks."

Shift to Hybrid Model

It took the cybersecurity community less than a year to almost completely abandon fully automated AI testing and replace it with a hybrid model - something around 47% said they now prefer. This model has surged 22% year-over-year, while the percentage of organizations using automation for low-risk environments also increased to 47%.

"While the industry is rightfully excited about the potential of Mythos-class tools, unguided algorithms are inherently prone to returning even more false positives and costly false negatives than the automated scanners we have today," continued Obadiaru.