Measurement OSV npm feed, September 2026 150 advisories sampled
The reachability ceiling
Reachability analysis is sold as the cure for vulnerability alert fatigue: it tells you whether your code actually calls the broken function. I measured how often an npm advisory names a function at all. The answer is about one in six, and when there is nothing to name, the tooling does not go quiet. It tells you that you are safe.
Every scanner answers the easy question
Run npm audit on a real project and you get dozens of alerts. Almost all of them are for
packages you pulled in transitively and code you never touch. The scanner knows the vulnerable
version is in your tree. It has no idea whether you go anywhere near the vulnerable part.
Reachability analysis is the answer the industry settled on. Snyk, Endor Labs and Arnica all sell it. The published research is encouraging: one recent framework reports a 78 to 89 percent reduction in false positives once you know which function is vulnerable.
That result rests on a quiet assumption, which is that the advisory tells you which function. For Go it does. OSV records it in a structured field, and Rust does the same. For npm, that field is simply absent, so every vendor rebuilds the mapping in private. I set out to build an open one, and ran into the assumption instead.
How often is there a function to name?
- 25 name a callable function
- 124 name none
- 1 undecidable
The debate about reachability for JavaScript has been a debate about precision. An
OpenJS thread ran for months on how
to name an export unambiguously: packages have zero, one or many entry points, ESM exposes a value
under any number of names, and a package without an exports field makes every file reachable.
Those problems are real. They also only arise for the sixth of advisories that name something. For the other five, no syntax helps, because there is no function in the description to point at.
What the other five in six look like
GHSA-35jh-r3h4-6jhm lodash reachability can answer
lodash versions prior to 4.17.21 are vulnerable to Command Injection via the template function.
A named export. A tool can search your code for calls to it.
GHSA-hrpp-h998-j3pp qs nothing to search for
qs before 6.10.3 allows attackers to cause a Node process hang because an __proto__ key can be used.
A key in an object. There is no function called __proto__.
GHSA-hf5h-hh56-3vrg uws nothing to search for
Affected versions of uws do not properly handle large websocket messages when permessage-deflate is enabled, which may result in a denial of service condition.
A configuration option. Whether you are affected depends on a setting, not a call.
GHSA-77q4-m83q-w76v browserify-hmr nothing to search for
Versions of browserify-hmr prior to 0.4.0 are missing origin validation on the websocket server.
Runtime behaviour of a service. There is no library API involved at all.
Option keys, query operators, property names, crafted inputs, HTTP headers, and the behaviour of servers and command line tools with no importable API. Roughly five advisories in six describe one of these rather than a function you call.
The failure is silent
A tool that cannot answer could say so. In practice the machinery does something worse. It searches for a call, finds none, and reports that you are not affected. I know because I built one and it did exactly that.
A worked example
GHSA-5mrr-rgp6-x4gr, marsdb: In the DocumentMatcher class, selectors on $where clauses are passed to a Function.
My extractor read DocumentMatcher and $where as symbols.
Linguistically correct, semantically wrong: $where is a key in a query object,
not something you can call. No call pattern can ever match it.
Meanwhile, in OWASP Juice Shop, routes/chat.ts line 149:
db.reviewsCollection.find({ $where: 'this.product == ' + productId })
Concatenated user input reaching a $where sink. Juice Shop documents a NoSQL
injection challenge on exactly this code.
my tool reported NOT REACHABLE and the output was indistinguishable from a win
That is the shape of the problem. A false positive wastes an afternoon. A false negative of this kind removes a live vulnerability from the queue and looks like the tool working well. Nothing in the output distinguishes we checked and you are fine from we had nothing to check with.
What would fix it
Not a better call graph. The engines are good, and open ones already exist. The gap is in the advisory data, and it needs two things.
Symbols where they apply. Go and Rust already publish them in OSV. Roughly 1,170 npm advisories could carry the same field today.
An explicit marker where they do not. This matters more. An advisory with no function data currently looks identical to one where no function applies, so every consumer decides for itself what the absence means, and the convenient reading is that there is nothing to reach. Making that state nameable is what turns a silent false negative into an honest cannot determine.
Method and limitations
150 advisories drawn with a fixed seed from the 7,020 live GHSA npm records in the OSV feed, withdrawn records excluded. Each was classified twice, independently, on reshuffled batches. Agreement was 92 percent with Cohen's kappa of 0.81.
- Classification was done by language models reading advisory prose, not by expert adjudication. That is why it was run twice, and why every per-advisory call is published rather than only the total.
- Only the first 420 characters of each advisory were read. A longer description might name a function the excerpt omits, which would bias the result toward the larger figure.
- The Juice Shop example was verified by reading the source, not by running the exploit.
- The first round of classification produced 84.7 percent and was not reliable enough to publish: per-batch rates varied from 8 to 18 percent, and two near-identical advisories were classified oppositely. The rubric was conflating does this name something callable with is calling it sufficient to be exploited. Only the first is a question reachability answers.