Skip to main content

Tools

The LLM vulnerability scanner.

Detection Post-generation Full HITL AI-based Stable Activity band D Environment
Activity band details

Activity band A to F: a bin of a score that adds GitHub stars (log scale), a push to the upstream repository in the 12 months before `as_of`, the number of declared integration surfaces and, where the catalogue records them, citing papers and benchmark evaluations. It measures popularity and activity. It is not a level of evidence: no band means a paper cites the tool or that it was measured. Recency is judged as of 2026-11-22.

Computed band: D (raw score 4.6). Shown band: D .

Facets: documented_in=0evaluated_on=0integration_surfaces=1paper_citations=0recent_push=2stars=1.6

Source of truth: pipeline/maturity/score.py

Description

NVIDIA garak is an open-source LLM vulnerability scanner: applies a library of probes (jailbreaks, prompt injection, data leakage, malware generation, misinformation) against any LLM endpoint and reports per-probe attack success rates. Models the NVIDIA NeMo Guardrails adversarial test plane.

Why it's included

Canonical LLM-vuln-scanner: complements llm-guard by running at assessment time rather than at request time.

Addresses failure modes

These tags are the catalog's claim about what garak is built to catch or prevent. A tag is not a test result. SAICA-Bench has not measured garak, so none of these tags is tested here.

  • Fabrication — Generating reference to non-existent APIs, packages, or symbols.
  • Supply-chain attack — Slopsquatting, typosquatting, compromised MCP servers.

Techniques

llm-probesattack-success-rate-metricsdetect-generators

View raw YAML on GitHub →