garak
10,125The LLM vulnerability scanner.
Activity band details
Activity band A to F: a bin of a score that adds GitHub stars (log scale), a push to the upstream repository in the 12 months before `as_of`, the number of declared integration surfaces and, where the catalogue records them, citing papers and benchmark evaluations. It measures popularity and activity. It is not a level of evidence: no band means a paper cites the tool or that it was measured. Recency is judged as of 2026-11-22.
Computed band: D
(raw score 4.6).
Shown band: D
.
Facets:
documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_push=2stars=1.6
Source of truth: pipeline/maturity/score.py
Description
NVIDIA garak is an open-source LLM vulnerability scanner: applies a library of probes (jailbreaks, prompt injection, data leakage, malware generation, misinformation) against any LLM endpoint and reports per-probe attack success rates. Models the NVIDIA NeMo Guardrails adversarial test plane.
Why it's included
Canonical LLM-vuln-scanner: complements llm-guard by running at assessment time rather than at request time.
Addresses failure modes
These tags are the catalog's claim about what garak is built to catch or prevent. A tag is not a test result. SAICA-Bench has not measured garak, so none of these tags is tested here.
- Fabrication — Generating reference to non-existent APIs, packages, or symbols.
- Supply-chain attack — Slopsquatting, typosquatting, compromised MCP servers.