Test-manipulation-resistant flow
semgrep rules on test-file diffs plus a promptfoo LLM judge that reads the agent's session. Flags edits that weaken, skip or delete tests.
Description
Two checks for the `test_manipulation` failure mode. Both run after the agent has finished and before its change is merged. Stage 1: semgrep runs project-written rules over the diff of test files and test-runner configuration: a removed assertion, an added skip marker, a weakened matcher. A match asks for human approval. Stage 2: promptfoo runs an `llm-rubric` assertion. A judge model reads the session transcript or the diff together with a rubric that describes test manipulation (assertions rewritten to match wrong output, tests skipped or deleted, the CI job disabled) and answers whether the agent gamed the tests.
Stack
Ordered composition. Each stage consumes the output of the one before it.
-
Stage 2promptfoo
Test prompts, agents, and RAGs — red-teaming, pentesting, and vulnerability scanning for LLMs.
Targets failure modes
- Test manipulation — Agent gaming its own evaluation — editing tests or test harness so failures disappear without fixing the underlying bug.
Notes
Text rewritten on 2026-11-21 to describe the tools in `stack`. The earlier text described aider commits and a coverage gate, which are not in the stack, and said the recipe was anchored in a DAPLab observation; that incident record was retracted (see data/TOMBSTONES.yml). Of the two stack tools the KG tags only promptfoo for test manipulation; Semgrep ships no rules for it, so they have to be written. SAICA-Bench measures the promptfoo stage on its own (an llm-rubric judge on Patronus TRACE sessions) and includes two simple test-edit rules as baselines; the two stages have not been measured together. `evidence_tier` stays `anecdotal`.