Definitions from NIST
NIST SP 800-115 defines vulnerability scanning as a technique used to identify hosts, host attributes and associated vulnerabilities.
The same publication defines penetration testing as security testing in which evaluators mimic real-world attacks in an attempt to identify ways to circumvent the security features of an application, system or network.
The differences side by side
| Aspect | Vulnerability scan | Penetration test |
|---|---|---|
| Who does it | A tool, configured by a person. | Skilled testers, who also use tools. |
| Goal | Identify known weaknesses across a target. | Show what an attacker could achieve by exploiting and chaining weaknesses. |
| Exploitation | Identifies and reports. It normally does not exploit. | Attempts exploitation within agreed rules. |
| Depth | Broad and shallow. Strong on known patterns and misconfiguration. | Narrower and deeper. Strong on business logic, access control and chained flaws. |
| Repeatability | High. Suitable for every release or on a schedule. | Point in time. Usually repeated periodically or after major changes. |
| Output | A list of findings with severity, which needs triage for false positives. | A narrative report with evidence, impact and reproduction steps. |
| Preparation | Target, credentials if scanning behind login, and permission. | Scope, rules of engagement, test accounts, a time window and written authorisation. |
What a vulnerability scan leaves to you
- Triage. Scanners report false positives, and someone must confirm what is real.
- Anything that needs understanding of your app, such as whether one customer can see another's data.
- Areas the scanner could not reach, such as pages behind login if you did not give it credentials.
- Fixing and re-scanning. A report changes nothing until findings are closed.
What a penetration test leaves to you
- Everything outside the agreed scope and time window.
- Every change you ship after the test ends. The report describes the app on the days it was tested.
- Remediation and a retest to confirm the fixes.
- Choosing a competent tester and agreeing rules of engagement. NIST SP 800-115 treats planning and authorisation as part of the process.
Which one do you need right now
- Before launch with no security work done yet: start with automated scanning and the manual checks below. They are quick and catch the common problems.
- A customer, regulator or contract asks for a penetration test: a scan report will not satisfy that request. Ask what scope and tester independence they expect.
- You handle payments, health data or other sensitive data at scale: plan for both, with scans running continuously and a penetration test periodically.
- After a major rebuild or a new auth system: scan at once, and consider a focused test on the changed area.
What to check whichever you pick
- You have written permission for every system in scope, including third-party hosting where their policy requires it.
- Testing runs against staging with test data where possible.
- Access control is tested with at least two accounts and a logged-out visitor.
- Each finding has an owner, a fix and a retest.
- Reports are stored securely, since they describe how to attack your app.
- The activity repeats. One clean result does not cover next month's code.
Frequently asked questions
Is a vulnerability scan the same as a penetration test?
No. NIST defines scanning as a technique to identify vulnerabilities, and penetration testing as evaluators mimicking real-world attacks to circumvent security features. A scan is automated and broad. A penetration test is human-led and deep.
Do I need a penetration test before launching my app?
Not always. It depends on your data, your customers and any contracts or regulations that apply. Many small apps start with scanning and manual access control checks, then commission a penetration test when a customer or compliance need requires it.
Can an automated tool do a penetration test?
Tools assist testers, and some products automate parts of exploitation. NIST's definition centres on evaluators mimicking real attacks, which includes judgement about business logic that automated tools handle poorly. Ask any vendor exactly what is automated.
How often should I run a vulnerability scan?
There is no single rule. A sensible pattern is on every release or on a regular schedule, because new code and newly disclosed vulnerabilities both change the result. Follow any frequency that a standard or contract sets for you.