Skip to content

Getting started

Install

pip install llmsectest

LLMSecTest is a pytest plugin and a CLI. The core install is dependency-light (just pytest); provider SDKs are optional extras you install only if you target them:

pip install "llmsectest[openai]"   # also: [anthropic], [huggingface], or [all]

A local model needs no extra and no API key — see Test your running app.

Your first run (zero config)

With no target, LLMSecTest runs against a built-in offline demo app so you get a real report immediately — no keys, no network:

llmsectest                 # scans the offline "vulnerable" demo app → shows findings
llmsectest --target demo-defended   # the hardened demo app → passes cleanly

This writes a SARIF report under results/ and prints a summary. Because findings are pytest failures, the process exits non-zero when the target is vulnerable — exactly what you want in CI.

Targets

A target is what you point LLMSecTest at. Choose one with --target:

Target What it is
app:<url> Your running application's HTTP endpoint — the faithful way to test an app (its own system prompt, guardrails, RAG and tools are in the loop). See the guide.
ollama:<model> A local model via Ollama — no API key, no paid calls (e.g. ollama:gemma4:e2b-it-q4_K_M).
openai:<model> / anthropic:<model> / huggingface:<model> A hosted model (needs the matching extra + API key in the environment).
demo / demo-defended Offline deterministic demo apps (no network).

See what's covered

llmsectest --check          # the OWASP coverage map + each category's test modality
llmsectest --list-probes    # the red-team corpus that ships today

--check is the source of truth for coverage — it shows which categories are black-box (testable against your endpoint now) and which are white-box (need app internals and land per milestone).

Next