OWASP LLM Top 10, coverage map¶
LLMSecTest maps directly to the OWASP Top 10 for LLM Applications (2025).
The ten risks split into two testing modalities and LLMSecTest is explicit about which applies to
a given target, the live, authoritative map is llmsectest --check.
- Black-box, testable by sending inputs to your running app (
--target app:<url>). - White-box, needs your application's internals (dependencies, RAG/vector store, resource limits, model/data provenance) and is covered by dedicated modules.
Each category also carries a representative CVSS v4.0 base score (worst-case for the class),
reported as the SARIF security-severity of its findings.
All ten categories ship a probe or scanner and all ten now have a deep-dive page, linked from the table below. Each page says what the category's probes send, what its oracle matches, and, the part a coverage table cannot carry, what a clean result does not tell you.
| Category | Modality | CVSS v4.0 | Status today |
|---|---|---|---|
| LLM01 Prompt Injection | black-box | 9.2 Critical | ✅ probes |
| LLM02 Sensitive Information Disclosure | black-box / white-box | 9.2 Critical | ✅ probes |
| LLM03 Supply Chain | white-box, requires --repo |
9.5 Critical | ✅ scan |
| LLM04 Data and Model Poisoning | white-box, requires --model-scan |
7.1 High | ✅ scan |
| LLM05 Improper Output Handling | black-box / white-box | 9.9 Critical | ✅ probes |
| LLM06 Excessive Agency | black-box / white-box | 10.0 Critical | ✅ probes |
| LLM07 System Prompt Leakage | black-box | 8.7 High | ✅ probes |
| LLM08 Vector and Embedding Weaknesses | black-box, requires --app-canary and/or --app-rag-poison (RAG); white-box, requires --vector-store |
7.1 High | ✅ probes + scan |
| LLM09 Misinformation | black-box | 5.3 Medium | ✅ probes |
| LLM10 Unbounded Consumption | black-box | 8.7 High | ✅ probes |
No silent gaps
All ten categories run on every invocation. Each ships a real probe or scanner; a category that
needs an input it wasn't given (a repo, a model path, an app marker) appears as a skipped test that
says exactly what it needs, never silently absent. LLMSecTest will not claim coverage a target's
modality didn't exercise. Run llmsectest --check for the current state.
With LLM04, LLMSecTest now covers the complete OWASP LLM Top 10 (2025), 10/10. The two
white-box categories run from a path you provide: LLM03 (supply chain) scans the project's dependency
manifests with --repo <path> (see the LLM03 deep-dive); LLM04 (data and model poisoning)
scans the project's serialized model files with --model-scan <path>, flagging load-time code-execution
in pickle/PyTorch artifacts (see the LLM04 deep-dive). LLM08 (vector & embedding weaknesses)
ships three dimensions for RAG apps: retrieval exposure and indirect injection via a poisoned
retrieved document, both black-box, plus the white-box embedding-inversion-exposure scan of a persisted
store with --vector-store <path> (see the LLM08 deep-dive); LLM09 (misinformation) ships
black-box confabulation probes (see the LLM09 deep-dive). What remains is depth: embedding
poisoning, multi-tenant isolation, inversion itself, plus a classifier refusal oracle.
Testing a real application (black-box)¶
When you point LLMSecTest at a running app (--target app:<url>, or the run_app_scan API on the app's
system prompt), it tests only what black-box access can reach. It reports the rest, never
as a silent pass:
The two entry points reach the same ten categories at different depth
With every developer input supplied, the run_app_scan API sends 23 cases, of which LLM01 is
one injection technique and LLM09 is one confabulation probe. The CLI runs the packaged
pytest suite, where LLM01 arrives as the five authored injection techniques plus the eight
built-in red-team behaviours and LLM09 as four confabulation probes, for 38 delivered
attacks. Both paths print the same 10-category coverage map, so an exercised LLM01 or LLM09 row
from the API path means the category was asked once. AppScanResult.outcomes carries the exact
set a given run sent.
- LLM01 (prompt injection), LLM05 (improper output handling), LLM09 (misinformation) and LLM10 (unbounded consumption) transfer with no setup: the attack-side marker (or, for LLM09, a guaranteed-nonexistent entity) lives in the attack, so the app needs to reveal nothing for a finding to be unambiguous. (LLM10 uses a bounded repetition-flood probe against an app, an explicit finite repeat count, above the flood threshold yet a short reply. So a flooding app is flagged without the unbounded model-mode prompts ever risking a runaway generation against an uncapped endpoint.)
- LLM07 (system-prompt leakage), LLM02 (sensitive disclosure), LLM06 (excessive agency) and
LLM08 (vector & embedding weaknesses) light up once you tell LLMSecTest what a leak looks like, the
app's own system prompt, a known secret it holds, its privileged action signatures, or (for a RAG app)
a confidential canary planted in its retrieved corpus (
--app-canary) and/or the marker a poisoned retrieved document emits (--app-rag-poison). Without that, they are reported as not exercised with the reason, rather than passed vacuously. - The white-box categories are likewise surfaced as not-exercised against an endpoint unless you supply
their artifact path: LLM03 (supply chain) runs from the repo (add
--repo <path>to scan the dependency manifests alongside the endpoint probes) and LLM04 (data and model poisoning) runs from the model files (add--model-scan <path>). LLM08 runs from the persisted store with--vector-store <path>, which measures how much of the corpus a reader of the store recovers with no inversion, alongside its two black-box dimensions, retrieval exposure and indirect injection via a poisoned retrieved document (see the LLM08 deep-dive). Embedding poisoning, multi-tenant isolation and inversion itself remain not-exercised.
Every scan prints a coverage footer accounting for all ten categories, so the report never overstates what was tested.