agent-audit-kit

AgentAuditKit

The missing npm audit for AI agents.

Static security scanner for MCP-connected AI agent pipelines. It finds misconfigurations, hardcoded secrets, tool poisoning, rug pulls, trust-boundary violations and tainted data flows across 11 agent platforms — and emits the compliance evidence an auditor asks for afterwards.

It runs fully offline. No account, no telemetry, no model in the loop.

Quick start

pip install agent-audit-kit
agent-audit-kit scan .

aak is installed as a shorthand for the same command.

As a pre-commit hook:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/sattyamjjain/agent-audit-kit
    rev: v0.6.23
    hooks:
      - id: agent-audit-kit

In CI:

# .github/workflows/agent-security.yml
- uses: sattyamjjain/agent-audit-kit@v0.6.23
  with:
    severity: medium
    fail-on: high
    upload-sarif: true

The action writes SARIF; upload it with github/codeql-action/upload-sarif to land findings in the GitHub Security tab. Full GitHub Action reference · CLI reference · pre-commit hook.

What it finds

412 rules across 14 security categories:

Every finding carries severity, evidence, a file and line, and remediation. Full detail per rule is in the rule reference.

Why not a hosted scanner

• Offline and deterministic. Your code and secrets never leave the machine, and the same input always yields the same findings — measured at 20/20 identical runs, 0% variance. A scanner with an LLM in the loop cannot promise that, which is what makes CI diffs and audit re-runs stable here.
• Auditor-ready evidence, not just findings. SARIF plus PDF evidence packs mapped to 14 frameworks, a CycloneDX/SPDX SBOM, and an OpenVEX document that joins to it on purl.
• Pin and verify. pin fingerprints a tool surface at approval; verify re-checks it afterwards. That is the only thing that catches a server which behaves until it does not — see the Deadbugz case study.

The hand-adjudicated benign-slice false-positive rate covers MCP server configs only. Instruction files (CLAUDE.md, AGENTS.md and the like) are not in the slice, and an outside report on 930 repositories (#771) is what exposed AAK-AGENT-002 on them, fixed in v0.6.8. Instruction files now have their own benign slice, built from that report's corpus; it is pending adjudication, so it has no rate yet.

What we measured

From the State of MCP Security 2026 report (how to cite):

• 2,303 distinct public MCP configs scanned.
• 50.6% (1,166) declare a remote server with **no authentication**.
• 0 reference RFC 9728 Protected-Resource-Metadata discovery in the config.
• 100% (424/424) of inline-auth remote configs **hardcode a static credential**.

These regenerate from results.json and are asserted in CI, so they cannot drift from the report.

Compliance evidence

PDF and text evidence packs mapped to 14 frameworks, including two already in force that most scanners do not carry: EU AI Act Article 50 (transparency, since 2026-08-02) and Colorado SB 26-189 ADMT (effective 2027-01-01).

agent-audit-kit report . --framework eu-ai-act --format pdf
agent-audit-kit sbom . --format cyclonedx -o sbom.cdx.json
agent-audit-kit vex  . -o vex.openvex.json

Every control row cites a real clause, and a row the scanner cannot evidence says so instead of printing a tick. Full list of the 412 rules mapped to 14 frameworks: compliance reference.

OWASP Agentic Top 10 coverage

Against the OWASP Top 10 for Agentic Applications (Version 2026, December 2025), with OWASP's own titles. A rule is counted when it detects the risk, or the absence of a control the entry prescribes for it.

Per-entry rule lists, generated from the registry: Agentic Top 10 · MCP Top 10 (v0.1, 2025 edition) · Agentic Skills Top 10. Every MCP Top 10 entry has at least one rule, but not evenly; the MCP table labels the thin entries Partial and names their rules. The rule-by-rule NSA MCP CSI and OWASP Agentic view is the standards crosswalk.

MCP Security Index

A public leaderboard of scanned public MCP servers, with per-server grade cards and a 90-day disclosure policy: sattyamjjain.github.io/agent-audit-kit.

Last published snapshot: **2026-10-19** (11 snapshots in [`history.json`](https://sattyamjjain.github.io/agent-audit-kit/data/history.json)). The build fails if this date falls more than 10 days behind, so a stalled index reports itself.

CVE response

Newly disclosed MCP CVEs are triaged and turned into rules as they land, surfaced by an NVD watcher and logged in CHANGELOG.cves.md. The measured disclosure-to-rule latency is published in docs/cve-latency.md and regenerated from the ledger, not asserted.

Under the hood

103 scanner modules: AST-based taint analysis for

Python, and regex dangerous-sink scanners for TypeScript/JavaScript and Rust.

4,684 test functions. 27 CLI commands. Releases are Sigstore-signed

and ship a deterministic rule bundle.

Mechanical fix recipes cover 11 of 412 rules (2.7%), applied by agent-audit-kit fix. That is a scope decision, not a coverage gap: a recipe ships only where the remediation is deterministic and one-line — exactly one correct edit, confirmable from the diff. Everything else stays advisory on purpose, because a fix that needs judgement is a fix that can be wrong silently. Why the rest stay advisory.

Documentation

sattyamjjain.github.io/agent-audit-kit/docs — the full documentation site, built from docs/ by MkDocs and deployed alongside the MCP Security Index.

Getting started · CLI reference · Rule reference · CI/CD · Comparison with other scanners · All documentation

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md. Adding a rule means a rule definition, a scanner, and fixtures in both directions; agent-audit-kit rule lint checks the registry invariants.

Security

Report vulnerabilities per SECURITY.md. AgentAuditKit publishes no fixed CVE-response SLA; it publishes the measured latency instead.

License

Apache-2.0. See LICENSE. Apache-2.0 carries an explicit patent grant, which the organisations that adopt a security tool tend to ask about.