Penetration test vs. vulnerability scan
What scanners catch, what they miss, and when you need a person. A plain-English comparison with the research behind it.
Last reviewed October 2026
The short answer
A vulnerability scan is automated and finds known weaknesses quickly. A penetration test is done by people, who confirm what’s exploitable and look for the flaws tools struggle with, like broken authorization and business logic. You may need both, at different times.
What is a vulnerability scan?
A scanner checks your systems against a large library of known vulnerabilities and misconfigurations: missing patches, outdated software, weak settings, and common web flaws. It runs on a schedule, covers many systems at once, and produces a list of possible issues, ranked by severity. Someone still has to decide which results are real and which matter.
What is a penetration test?
A penetration test is an authorized attempt to break in, led by testers. They map how your system works, try what an attacker would try, confirm what’s actually exploitable, and connect small issues into a path to your data. The result is a report with evidence, reproduction steps, and specific fixes.
What scanners do well
Scanners are fast, cheap to repeat, and good at finding known vulnerabilities and common misconfigurations across many systems at once. That makes them the right tool for checks between tests, and for catching a known exploited vulnerability before it lingers. CISA’s Known Exploited Vulnerabilities catalog lists more than 1,700 vulnerabilities with reliable evidence of in-the-wild exploitation. CISA strongly recommends all organizations prioritize remediating them.[1]
What scanners miss
Authorization
A 2025 peer-reviewed ACM CCS study says reliable broken-access-control detection remains limited; in its benchmark, a commercial scanner with an authorization extension caught 54% of 57 unauthorized-modification flaws.[2] MITRE’s CWE-862 (Missing Authorization) rates automated static analysis ‘Limited,’ noting difficulty with ‘custom authorization schemes,’ and says ‘manual analysis is required’ to determine whether missing authorization violates business logic.[3] The OWASP Top 10:2025 keeps Broken Access Control at #1, and OWASP notes its data is ‘largely limited to what the industry can test for in an automated fashion.’[4]
Business logic and workflows
OWASP’s Web Security Testing Guide says business-logic flaws ‘cannot be detected by a vulnerability scanner’ and that automating business-logic abuse cases ‘is not possible and remains a manual art.’[5] A 2014 peer-reviewed NDSS paper noted that application-specific logic flaws ‘remain outside the scope of most of the existing tools and still need to be discovered by manual inspection.’[6] A peer-reviewed 2025 NDSS paper says web scanners ‘still perform poorly or even fail’ at discovering deeper app states behind multi-step workflows like checkout, ‘leaving potential vulnerabilities undetected.’[7]
Older studies, same pattern
In a peer-reviewed 2010 study, all 11 web scanners missed 8 of 16 planted flaws, including a coupon logic flaw; students with average security skills found 15 of 16.[8] In a 2010 IEEE S&P study, eight commercial scanners caught none of the planted second-order SQL injections and averaged 15% on stored XSS; only textbook reflected XSS topped 32.5%.[9] In a peer-reviewed 2021 study (IEEE S&P), six open-source and academic scanners found no confirmed XSS on six modern apps, including WordPress and PrestaShop; only the authors’ scanner did.[10]
These studies tested older tools, a single test application, or a single bug class, so read them as a pattern rather than a measurement of today’s scanners.
Manual vs. automated penetration testing
NIST SP 800-115 notes most penetration tests seek combinations of vulnerabilities granting more access than any single flaw, and manual analysis can identify new or obscure vulnerabilities scanners may miss.[11] NIST SP 800-53 Rev. 5 (CA-8) says penetration testing ‘goes beyond automated vulnerability scanning,’ is conducted by teams ‘with demonstrable skills and experience,’ and provides ‘a more in-depth analysis.’[12] In a 2022 peer-reviewed case study on one web app, static analysis found the most vulnerabilities, exploratory manual pentesting found more severe ones, and each technique found issues others missed.[13]
That last study is the best summary: tools for breadth, people for depth.
Which do you need?
- A customer, auditor, or insurer asked for a penetration test: a penetration test, not a scan.
- You want to catch known vulnerabilities between tests: recurring vulnerability scanning.
- You’re shipping a new app, API, or AI feature: a penetration test before launch.
- You’ve never tested before: start with a penetration test of your most important system, and add scanning after.
Use both
Scanning and penetration testing answer different questions, so they work best together: scans catch known issues between tests, and a pen test finds what scans can’t reason about.
Not sure which you need?
We reply to every request within one business day. Every engagement is a fixed fee after a free scoping call.