AI codebase audits,
one focus area at a time.
Noxaudit is an open-source CLI that runs focused AI audits across security, testing, dependencies, patterns, performance, docs, and hygiene — then validates, deduplicates, and remembers what you've already decided, so only genuinely new findings surface.
More than a prompt
Raw LLM output is noisy and unstable between runs. Every audit goes through a seven-stage pipeline built to make findings trustworthy enough to act on.
- 01
Gather
Collects the files relevant to each focus area, with an optional cheap-model pre-pass that triages large repos before the expensive audit.
- 02
Audit
Sends focused prompts to Claude, GPT, or Gemini — synchronously or through provider batch APIs at half the price.
- 03
Validate
A second LLM pass reads each finding against the actual source and drops false positives.
- 04
Deduplicate
Normalizes finding titles to canonical forms so the same issue keeps the same identity across runs.
- 05
Score
Assigns confidence from cross-run frequency — findings that keep reappearing rank higher than one-offs.
- 06
Remember
Filters against your decision history, so dismissed or accepted findings don't resurface unless the file changes.
- 07
Report
Writes markdown and SARIF (GitHub Code Scanning), sends notifications, and can open GitHub issues.
Seven focus areas
Each focus area has a specialized prompt that goes far beyond “scan for issues.” Run one, several, or all of them — combined runs share file context and cut token use by up to 80%.
Security
Trace data flow from input to output, identify auth bypass paths, check for OWASP top 10, analyze session and token handling.
Testing
Analyze actual coverage gaps, suggest specific test cases with inputs and outputs, identify untested edge cases and critical paths.
Dependencies
Cross-reference against known CVEs, check license compatibility, identify abandoned or unmaintained packages, flag unused deps.
Patterns
Check for architectural consistency across modules, identify code that violates the project's own conventions and patterns.
Performance
Identify N+1 queries, missing indexes, hot loops, unbounded allocations, missing caching opportunities.
Docs
Diff README and docstrings against actual function signatures, find stale examples, identify undocumented public APIs.
Hygiene
Map dead code paths, find unused exports and imports, identify copy-paste duplication, flag orphaned config files.
Try it
Bring your own provider key. Everything runs locally; findings and decisions live in your repo under .noxaudit/.
pip install 'noxaudit[openai]'
export OPENAI_API_KEY=sk-...
# all seven focus areas
noxaudit run
# or pick a few
noxaudit run --focus security,performance
# dismiss a finding so it stays gone
noxaudit decide abc123 --action dismiss \
--reason "test fixture"CLI
pip install noxaudit — run against any repo with your own provider key.
GitHub Action
Scheduled audits in CI, with SARIF upload to GitHub Code Scanning.
MCP server
Query findings and record decisions from Claude Code, Cursor, or Windsurf.
Benchmarked across 10 models
September 2026: ten current models ran all seven focus areas against the same two repos, at the same commits, as the March run, through batch APIs, for $2.81 total. Quality is scored by hand against six real python-dotenv issues that four or more models found in March.
$0.03
gpt-5-mini for both repos
Still the daily pick six months on: 5 of 6 canary issues, same as March, for the same price.
6/6
Opus 5.5, Sonnet 5.5, Sonnet 4.6
Three models caught every canary issue, as three did in March. Frontier quality got cheaper per finding, not better on the canary.
2/6
gpt-6-luna
Newest isn't always better: a third of gpt-5-mini's price, and it missed four of the six issues.
| Model | Findings | Cost | Consensus | Pick |
|---|---|---|---|---|
| gpt-5-mini | 34 | $0.03 | 5/6 | Best daily |
| gpt-6-sol | 100 | $0.21 | 5/6 | Best deep dive |
| claude-sonnet-5-5 | 67 | $0.39 | 6/6 | |
| claude-opus-5-5 | 120 | $0.79 | 6/6 | Max depth |
Read the write-up →All 10 models →Raw results and methodology on GitHub →
How the hosted version was built
Alongside the CLI sat a managed platform: install a GitHub App, and audits ran nightly on managed inference with a dashboard for triage. It ran on Atriumn's own repositories and is no longer offered — this is how it worked.
GitHub App install
One-click install instead of YAML and API keys. Webhooks register repos; short-lived installation tokens scope each clone to a single repo.
Ephemeral Fly.io workers
A scheduled workflow starts a stopped machine nightly. It clones each repo, runs the pre-pass, submits batch jobs, and exits. A second machine polls every 10 minutes to retrieve finished batches.
Tiered model schedule
gpt-5-mini daily, gpt-5.4 weekly deep dives, Opus monthly — the tiers came straight out of the March benchmark.
Budget enforcement
Every batch is costed on retrieval. At 80% of a repo's monthly budget audits downgrade to the cheapest model; at 95% they stop.
Supabase Postgres + RLS
Findings, audits, decisions, and health history live in tenant tables behind row-level security; workers write through authenticated API routes.
Dashboard and triage
Next.js 16 app on Vercel with health scores, trend charts, audit comparisons, and bulk triage — plus an API-backed MCP server so decisions sync across a team.