Field Notes / Fundamentals
Vulnerability Assessment and Penetration Testing: What VAPT Buys You, and How to Avoid Paying for a Scan
Vulnerability assessment and penetration testing is one engagement with two halves. The assessment tells you what is present and known-vulnerable. The test tells you what an attacker reaches, chains and takes. Here is how to scope a VAPT engagement so you get both, and how to spot the automated scan that arrives wearing a penetration test cover page.
01 What VAPT actually means
APT is a procurement label. The methodology underneath it belongs to NIST, and it is more precise than the acronym suggests. NIST SP 800-115 breaks a penetration test into four phases: Planning, Discovery, Attack and Reporting, with a loop that runs from Attack back into further Discovery each time the testers gain new access NIST SP 800-115 .
Count the phases the supplier actually delivered. Vulnerability analysis sits inside Discovery, phase two of four. A supplier who stops there has sold you part of one phase and named the invoice after the whole sequence. Ask which phase the report ends in. The answer is usually visible from the table of contents.
Attack is where the money goes, and NIST is blunt about why. "Most penetration tests involve looking for combinations of vulnerabilities on one or more systems that can be used to gain more access than could be achieved through a single vulnerability" NIST SP 800-115 . No scanner produces that sentence about your estate. Here is the shape it takes in a cloud-hosted product. A public storage bucket holds a build artefact. The artefact contains a CI token nobody rotated. The token can push a container image. The image runs with a role that reads the production database. Four findings, none of them rated critical on its own, one route to every customer record you hold. A tool that scores each asset in isolation has no way to draw that line.
02 Two halves, two questions
The clearest comparison in the industry was written by the PCI Security Standards Council in 2017 and it has not aged. Its Penetration Testing Guidance sets the two activities side by side across purpose, timing, method, reporting and duration PCI SSC Penetration Testing Guidance . The table below follows that structure.
| Vulnerability assessment | Penetration test | |
|---|---|---|
| Question answered | What is present and known-vulnerable? | What can be reached, chained and taken? |
| Method | Automated, signature and version driven | Manual process run by skilled testers, tools in support |
| Duration | Seconds to minutes per scanned host | Days or weeks, depending on scope and environment size |
| Output | A list of unvalidated weaknesses with severity scores | Validated findings with a demonstrated exploitation path |
| False positives | High rate, flagged in NIST SP 800-115 | Removed by exploitation before the finding is written |
| PCI DSS v4.0.1 control | Requirement 11.3, at least every three months | Requirement 11.4, at least every 12 months and after change |
Look at the duration row again, because it decides most disputes. The PCI SSC measures a scan in "several seconds to several minutes per scanned host" and a penetration test in engagements that "may last days or weeks depending on the scope of the test and size of the environment" PCI SSC Penetration Testing Guidance . When a quote prices effort by IP address rather than by tester-day, you are reading a scan. Our own penetration testing engagements are scoped in operator days for exactly that reason.
03 What a scan can never find
Some flaw classes are not hard for scanners. They are closed to them. OWASP states it flatly in the Web Security Testing Guide: a business logic vulnerability "cannot be detected by a vulnerability scanner and relies upon the skills and creativity of the penetration tester" OWASP WSTG 4.10 . Automated tools struggle with context, so a person has to run these tests. A tool can confirm that a checkout process works. It cannot know that applying a refund twice should be impossible.
OWASP puts a number on the ceiling in the Application Security Verification Standard: "automated tools and online scans are unable to complete more than half of the ASVS without human assistance", and business logic and access control testing "is only possible using human assistance" OWASP ASVS 4.0.3 . Half of the recognised verification requirements for a web application sit outside what any scanner can reach.
That matters more once you see which flaw class sits at number one. Broken access control took the top spot in the OWASP Top 10:2021 with the most occurrences of any category in the contributed dataset: over 318,000 occurrences across 34 mapped CWEs and 19,013 CVEs, at an average incidence rate of 3.81% and a maximum of 55.97% OWASP Top 10:2021 A01 . The most commonly encountered serious category in application security is also one OWASP says needs a human to verify. A scan will pass an application whose most likely route to compromise it was never able to look at.
AI systems have opened the same gap again, one generation later. A scanner has no concept of prompt injection, tool misuse, or an agent persuaded to act outside its permissions, which is why AI penetration testing exists as separate work, and why our AI security testing targets the model, the retrieval pipeline and the tools it can call rather than the server underneath them.
04 A CVSS list is not a decision
Most scan reports rank findings by CVSS and stop. FIRST, the body that owns CVSS, rejects that use of its own score: "CVSS is designed to measure the severity of a vulnerability and should not be used alone to assess risk." The Base Score "should be supplemented with a contextual analysis of the environment", and a real prioritisation system should weigh factors outside the scope of CVSS entirely, such as exposure and threat FIRST CVSS v3.1 User Guide .
Then look at the volume. NIST's National Vulnerability Database held 385,855 CVE records on 3 September 2026 NIST NVD CVE API , and 49,972 of those were published in calendar 2025 NIST NVD, 2025 publication windows . CISA's Known Exploited Vulnerabilities Catalog, its authoritative list of weaknesses confirmed exploited in the wild, held 1,694 entries on the same day CISA KEV Catalog . Fewer than one CVE record in two hundred appears on the list CISA has confirmed attackers are using.
| Figure | Value | Source |
|---|---|---|
| CVE records in the NVD | 385,855 (3 September 2026) | NIST NVD, CVE API 2.0 |
| CVE records published in 2025 | 49,972 | NIST NVD, CVE API 2.0, 2025 date windows |
| Entries in CISA KEV | 1,694 (version 2026.09.02) | CISA |
| Breaches starting with vulnerability exploitation | 31% | Verizon DBIR 2026 |
Read those numbers together and the job becomes obvious. Scanning gives you breadth you cannot get any other way, and NIST is clear that it matters for catching weaknesses before an attacker does. The scanner just cannot tell you which of its thousands of rows is the one that ends your quarter. NIST says the same about the scores themselves: assessors should determine the appropriate risk level for each vulnerability rather than accept the levels a scanner assigns, and scanning "is unable to address the overall risk level of a scanned network" NIST SP 800-115 . A human decides. The tool counts.
05 Spotting the scan in disguise
Ask one question. Did a person attack this, or did a tool enumerate it? You can answer that from the report structure alone, before you read a single finding. The PCI SSC states the standard directly: "merely reporting lists of vulnerabilities is not helpful in this endeavor and does not meet the intent of the penetration test", and the report should "clearly communicate what was tested, how it was tested, and the results of the testing" PCI SSC Penetration Testing Guidance .
The Council also publishes the report outline that meets that intent: executive summary, statement of scope, statement of methodology, statement of limitations, testing narrative, segmentation test results, findings, tools used, and cleanup instructions. Use it as your acceptance checklist. Three absences settle the question on their own.
- Testing narrative present: what the tester did, in what order, and what happened when they did it. A scan cannot produce one.
- Statement of limitations present: what could not be reached, what was excluded, and why. Silence here means nobody hit a wall, which means nobody pushed.
- Every serious finding carries an exploitation path, evidence, and the business impact. A CVE plus a CVSS score with no proof is scanner output.
- Effort quoted in tester-days against a defined scope, not in hosts or IP addresses scanned.
- At least one finding that chains two or more weaknesses together. Chaining is the signature of a human.
- Findings a scanner cannot generate: business logic abuse, access control bypass between user roles, authorisation flaws in the API.
- Named testers with relevant experience. PCI SSC states qualifications "cannot be met by certifications alone".
- Cleanup instructions for test accounts, uploaded files and any artefacts the testers left behind.
One more tell sits in the pricing. A genuine penetration test is priced on human time against an agreed scope, and the supplier will argue with you about scope before they quote, because scope drives days. A scan repackaged as a test is priced per asset and quoted within an hour of your first email. Both can be legitimate purchases. Only one of them satisfies Requirement 11.4, and only one finds the flaw class OWASP says needs a person.
06 How to scope one properly
Scope a VAPT engagement around the two questions rather than around an asset count. Start by writing down what you already know from scanning, so the testers do not spend paid days rediscovering it. Then define what a successful attack would look like in your business: reading another customer's data, moving money, reaching the production database from a low-privilege account. Those targets turn a test into evidence.
Then set the two clocks running at their own speeds. Scanning tracks the 49,972 CVE records that landed in 2025 and the ones still arriving this quarter. Testing answers the only question those records cannot: whether any of them connect into a route through your business. Retest what you fixed. A finding closed without verification is a finding you hope is closed.
Buy the engagement on the report you want to be holding at the end of it. Ours carries a testing narrative, a stated set of limitations, and a proven path behind every serious finding, because CREST-certified operators exploited each one by hand before it was written down. The free audit is where that starts: a map of your exposed attack surface, the route in we would take first, and the scope your regulator or your largest customer will actually accept. Your red team. Within reach. Get your free audit.
References
Sources
- PCI Security Standards Council. Information Supplement: Penetration Testing Guidance, v1.1. September 2017. listings.pcisecuritystandards.org
- PCI Security Standards Council. PCI DSS v4.0.1 Self-Assessment Questionnaire D for Merchants, Requirements 11.3 and 11.4. Document Library, SAQ category (v4.0.1 SAQs published October 2024; retrieved 3 September 2026). pcisecuritystandards.org
- NIST. Special Publication 800-115: Technical Guide to Information Security Testing and Assessment. nvlpubs.nist.gov
- OWASP. Web Security Testing Guide: Introduction to Business Logic Testing (WSTG 4.10). owasp.org
- OWASP. Application Security Verification Standard 4.0.3, Using the ASVS. raw.githubusercontent.com
- FIRST. CVSS v3.1 User Guide, Section 2.1: CVSS Measures Severity, not Risk. first.org
- CISA. Known Exploited Vulnerabilities Catalog, version 2026.09.02 (retrieved 3 September 2026). cisa.gov
- NIST. National Vulnerability Database, CVE API 2.0, all-time record count (retrieved 3 September 2026). services.nvd.nist.gov
- Verizon. 2026 Data Breach Investigations Report, findings summary (retrieved 3 September 2026). verizon.com
- OWASP. Top 10:2021, A01 Broken Access Control. owasp.org
- NIST. National Vulnerability Database, CVE API 2.0, publication-date windows across 2025 (four quarterly queries summed: 12,412 + 12,205 + 12,207 + 13,148 = 49,972; retrieved 3 September 2026). services.nvd.nist.gov