Skip to content

Getting started

These pages describe v0.3.0

The site is built from main, so it can describe a version newer than the one pip install llmsectest gives you. Run llmsectest --version to see what you have. The changelog says what arrived when.

Have a running app and a minute?

Quickstart goes straight from pip install to a rendered report against your own HTTP chat endpoint. This page is the wider tour: every target type, the extras and the coverage map.

Install

pip install llmsectest

LLMSecTest is a pytest plugin and a CLI. The core install is dependency-light (just pytest); provider SDKs are optional extras you install only if you target them (append the extra to the URL):

pip install "llmsectest[openai]"   # also: [anthropic], [huggingface], or [all]

A local model needs no extra and no API key, see Test your running app.

Your first run (zero config)

With no target, LLMSecTest runs against a built-in offline demo app so you get a real report immediately, no keys, no network:

llmsectest                 # scans the offline "vulnerable" demo app → shows findings
llmsectest --target demo-defended   # the hardened demo app → passes cleanly

This writes a SARIF report under results/ and prints a summary. Because findings are pytest failures, the process exits non-zero when the target is vulnerable, which is what you want in CI.

Targets

A target is what you point LLMSecTest at. Choose one with --target:

Target What it is
app:<url> Your running application's HTTP endpoint, the faithful way to test an app (its own system prompt, guardrails, RAG and tools are in the loop). See the guide.
ollama:<model> A local model via Ollama, no API key, no paid calls (e.g. ollama:gemma4:e2b-it-q4_K_M).
lmstudio:<model> A local model via LM Studio's OpenAI-compatible server (default localhost:1234), no API key, no paid calls. Set LMSTUDIO_BASE_URL to override the port.
openai:<model> / anthropic:<model> / huggingface:<model> A hosted model (needs the matching extra). openai and anthropic read OPENAI_API_KEY / ANTHROPIC_API_KEY and fail fast without one; huggingface reads HF_TOKEN if set but runs without it against a public model.
demo / demo-defended Offline deterministic demo apps (no network).

See what's covered

llmsectest --check          # the OWASP coverage map + each category's test modality
llmsectest --list-probes    # the red-team corpus that ships today

--check is the source of truth for coverage. It shows which categories are black-box (testable against your endpoint now) and which are white-box (LLM03 via --repo, LLM04 via --model-scan, both shipping today).

Next