Pyright
15,755Microsoft's fast static type checker for Python.
Activity band details
Activity band A to F: a bin of a score that adds GitHub stars (log scale), a push to the upstream repository in the 12 months before `as_of`, the number of declared integration surfaces and, where the catalogue records them, citing papers and benchmark evaluations. It measures popularity and activity. It is not a level of evidence: no band means a paper cites the tool or that it was measured. Recency is judged as of 2026-11-22.
Computed band: A
(raw score 7.18).
Shown band: A
.
Facets:
documented_in=0evaluated_on=2integration_surfaces=1.5paper_citations=0recent_push=2stars=1.68
Source of truth: pipeline/maturity/score.py
Measured on SAICA-Bench in this configuration
Each row is one run of Pyright, set up by us as its run card says, on one labelled set of faulty and clean samples (how we test). It says what that configuration flagged on that set, not how good Pyright is in general: another model, another input or the tool's own defaults could score differently. J = TPR - FPR, on the answers that carry a verdict. 0% is chance, 100% flags every bad sample and no clean one. J is the sort key within a mode; each row also has false alarms per 100 clean changes and precision at 1%, 5% and 20% prevalence.
| Set | Configuration and run card | J (95% interval) | TPR | FPR | No verdict | Cost per 1,000 reviews |
|---|---|---|---|---|---|---|
|
Logic error
Model-written code, not agent output
|
pyright
Catalog tool · Free program · own check · outside design scope
Not ranked: outside the tool's design scope: a type checker looks for type errors, not for wrong results; shown as a floor.
How this run differs from the tool's defaults
|
+0.3%
−0.5% to +1.1%
|
1.5% | 1.2% | 0 | $0 |
|
Fabrication
Failure planted on purpose
|
pyright
Catalog tool · Free program · own check
Rank 6 of 15 ranked rows.
How this run differs from the tool's defaults
|
+85.3%
+81.0% to +89.3%
|
88.8% | 3.5% | 0 | $0 |
|
Obsolescence
Failure planted on purpose
|
pyright against the latest libraries (deprecated, missing attribute or import)
Catalog tool · Free program · own check
Rank 4 of 9 ranked rows.
How this run differs from the tool's defaults
|
+56.8%
+42.0% to +71.2%
|
60.6% | 3.9% | 0 | $0 |
An answer with no verdict (failed call, blocked request, text cut by the token budget, no verdict in the text) is unreadable: it is left out of the rates and counted beside them, per label. “~” marks an estimated cost. SAICA-Bench results v2.0.0, built 2026-11-22.
Description
Pyright is Microsoft's TypeScript-implemented static type checker for Python, designed for low-latency incremental analysis of large codebases. It powers the Pylance language server in VS Code and runs from the CLI in CI, catching type incompatibilities and references to non-existent attributes (the fabrication failure mode) before code ships. Configuration lives in pyproject.toml [tool.pyright] or in a dedicated pyrightconfig.json.
Why it's included
Occupies detection × post_generation × fully_autonomous for logic_error and fabrication in the Python lane, alongside mypy. Distinct because Pyright targets editor-grade incremental performance and ships the language server most agentic IDE integrations rely on; missing it leaves the audit detector blind to a major slice of real Python configurations.
Security notes
Standard dev-tool security posture; Pyright runs locally and analyses source statically without executing it. Released under the MIT license with public source on GitHub.
Addresses failure modes
These tags are the catalog's claim about what Pyright is built to catch or prevent. A tag is not a test result. The rows under Measured on SAICA-Bench are the evidence, for the sets and the configuration we ran.
- Logic error — Correct API use, incorrect logic.
- Fabrication — Generating reference to non-existent APIs, packages, or symbols.
- Obsolescence — Using deprecated APIs, retired packages, or outdated library versions — often because the model's training data pre-dates current releases.