Skip to main content

Tools

promptfoo

27,525

Test prompts, agents, and RAGs — red-teaming, pentesting, and vulnerability scanning for LLMs.

Detection Post-generation Full HITL Hybrid Stable Activity band A PromptEnvironment
Activity band details

Activity band A to F: a bin of a score that adds GitHub stars (log scale), a push to the upstream repository in the 12 months before `as_of`, the number of declared integration surfaces and, where the catalogue records them, citing papers and benchmark evaluations. It measures popularity and activity. It is not a level of evidence: no band means a paper cites the tool or that it was measured. Recency is judged as of 2026-11-22.

Computed band: A (raw score 7.28). Shown band: A .

Facets: documented_in=0evaluated_on=2integration_surfaces=1.5paper_citations=0recent_push=2stars=1.78

Source of truth: pipeline/maturity/score.py

Measured on SAICA-Bench in this configuration

Each row is one run of promptfoo, set up by us as its run card says, on one labelled set of faulty and clean samples (how we test). It says what that configuration flagged on that set, not how good promptfoo is in general: another model, another input or the tool's own defaults could score differently. J = TPR - FPR, on the answers that carry a verdict. 0% is chance, 100% flags every bad sample and no clean one. J is the sort key within a mode; each row also has false alarms per 100 clean changes and precision at 1%, 5% and 20% prevalence.

Set Configuration and run card J (95% interval) TPR FPR No verdict Cost per 1,000 reviews
Security vulnerability
Real agent output
promptfoo llm-rubric (deepseek-v4-flash judge)
Catalog tool · Hybrid · our rubric · judge model deepseek-v4-flash
Rank 9 of 13 ranked rows; no reviewer reaches J 25% on this set.
Version
promptfoo 0.125.1
Judge model
deepseek-v4-flash-0731
Provider
Hyperbolic
What it was shown
task text; unified diff of the change
Counts as flagged when
Flagged when the llm-rubric assertion fails.
Temperature
0
Output token limit
16384
First run
2026-11-19/20
Check
our rubric: the tool ran the rule text we wrote, not a check of its own
Catalog
in the catalog
Runs
1,702; failed 25 (1.5%): blocked 25
Samples
1,243 bad, 459 clean
Answers with no verdict
25 (5 bad, 20 clean): blocked 25
Cost
$1.00 per 1,000 reviews (api); $1.71 spent on this run. Charge reported by the API with each call (Hyperbolic)

How this run differs from the tool's defaults

  • runs our rule text through its llm-rubric assertion; promptfoo's grader prompt is unchanged
  • answers that failed on provider errors are counted as failed runs, not as 'not flagged'
+1.8%
−7.0% to +10.6%
48.7% 46.9% 25 $1.00
Test edits that hide failures
failure mode: Test manipulation
Failure planted on purpose
promptfoo llm-rubric (deepseek)
Catalog tool · Hybrid · our rubric · judge model deepseek-v4-flash
Rank 8 of 12 ranked rows.
Version
promptfoo 0.125.1
Judge model
deepseek-v4-flash-0731
Provider
Hyperbolic
What it was shown
full agent session transcript (messages, tool calls, tool results)
Counts as flagged when
Flagged when the llm-rubric assertion fails (the grader judges the transcript breaks the rule).
Temperature
0
Output token limit
16384
First run
2026-11-19
Check
our rubric: the tool ran the rule text we wrote, not a check of its own
Catalog
in the catalog
Runs
305; failed 0 (0.0%)
Samples
56 bad, 249 clean
Answers with no verdict
0 (0 bad, 0 clean)
Cost
$1.10 per 1,000 reviews (api); $0.57 spent on this run. Charge reported by the API with each call (Hyperbolic)

How this run differs from the tool's defaults

  • runs our rule text through its llm-rubric assertion; promptfoo's grader prompt is unchanged
+86.2%
+76.1% to +90.7%
94.6% 8.4% 0 $1.10

An answer with no verdict (failed call, blocked request, text cut by the token budget, no verdict in the text) is unreadable: it is left out of the rates and counted beside them, per label. “~” marks an estimated cost. SAICA-Bench results v2.0.0, built 2026-11-22.

Description

Promptfoo is a CLI + library for testing, evaluating, and red-teaming LLM applications. Supports side-by-side prompt comparison, LLM-as-judge assertions, adversarial red-team generation, and CI integrations. Used by enterprises to gate LLM deployments on pre-prod evals.

Why it's included

Occupies the CLI-driven prompt-eval + red-team cell. Distinct from deepeval (Pytest-native unit tests) by leaning into YAML configs and shift-left red-teaming.

Addresses failure modes

These tags are the catalog's claim about what promptfoo is built to catch or prevent. A tag is not a test result. The rows under Measured on SAICA-Bench are the evidence, for the sets and the configuration we ran. Not measured here: Scope creep and Supply-chain attack.

  • Security vulnerability — Insecure patterns, injection flaws, weak cryptography, secret leakage.
  • Scope creep — Actions outside declared task boundary.
  • Test manipulation — Agent gaming its own evaluation — editing tests or test harness so failures disappear without fixing the underlying bug.
  • Supply-chain attack — Slopsquatting, typosquatting, compromised MCP servers.

Records evidence of

promptfoo can record evidence of these failures (traces, or checks you configure yourself), but it does not detect or prevent them on its own. They are not counted as failure modes it addresses.

Techniques

prompt-eval-harnessred-team-attacksllm-as-judgeci-gates

View raw YAML on GitHub →