agentpipe-scan

agentpipe

Find CI agents that eat untrusted GitHub events and hold keys. A repo lets a stranger's issue title drive an AI agent step that carries an npm token, and now an issue is a publish. agentpipe finds those chains statically, before anyone opens the issue.

Not an Actions linter. Not gitleaks. It reports preconditions and never exploits.

Part of Agentoffense. The recon front of the line: find the exposure here, close it at runtime with Airlock and Countersign.

pip install agentpipe-scan
agentpipe scan .                          # local clone, no network
agentpipe scan github.com/org/repo        # public workflows via GitHub API
agentpipe scan github.com/org             # whole org, aggregated report
agentpipe prove .                         # canary kit: prove each finding harmlessly
                                          # (token: GITHUB_TOKEN env or --token)

What problems it solves

AI agents moved into two places that hold real power, and both are easy to leave open. agentpipe finds the exposure in each, statically, before an attacker does.

1. A stranger's text becomes your CI's actions ("clinejection"). You wire an AI agent into GitHub Actions to triage issues or review PRs. The workflow triggers on issues or pull_request_target, drops the issue title straight into the agent's prompt, and the job holds an NPM_TOKEN and packages: write. Now anyone who opens an issue is handing instructions to a process that can publish your package or exfiltrate through a bot comment. No one clicked anything. agentpipe traces the whole chain (untrusted trigger to agent step to secret or publish sink) and points at the exact file:line, so you see it as code review, not as an incident.

2. Your own laptop's agent config is wide open. The agent you run locally trusts a config you set up once and forgot. Confirmations are off (bypassPermissions), a filesystem MCP is rooted at / so the agent can read every secret you own, a hook pipes curl | sh, an MCP server installs from an unpinned npx -y pkg (a rug-pull waiting to happen), and a token sits in plaintext in the config. agentpipe local reads only, touches no network, masks secret values, and tells you exactly what to tighten.

Why static and why now. Runtime tools (Airlock, Countersign) stop a bad call in the moment. agentpipe answers the earlier question: where are you even exposed? It runs in seconds on a repo, a whole org, or a machine, needs no agent running and no exploit, and every finding carries what it proves, what it does not, and the one change that closes it. Each report ends with what the scan cannot see, which is where a real pentest begins.

The killer bit: prove

A finding says "preconditions exist". agentpipe prove turns each high/medium finding into a harmless canary kit: a unique token, the exact issue/PR text to paste into a fork you own, the three places to watch (comment, step summary, job log), what a positive result proves and what it still doesn't (that part is the pentest). Every payload is echo/read-only by construction.

The report also groups flat findings into attack chains (entry → amplify → impact, one per job, severity-tagged).

Fix it: agentpipe fix

Finding the chain is half the job. agentpipe fix writes the patch, round-trip so your comments, key order and formatting survive:

agentpipe fix .            # show the diff (dry run)
agentpipe fix . --apply    # write the patches

It only edits where the change cannot break a legitimate workflow: it removes a --dangerously-skip-permissions flag (that just re-enables the confirmation the agent was told to skip), and adds environment: release to a publish job on an untrusted trigger (which requires a human approval before the publish). Anything whose safe form depends on your intent (narrowing permissions:, isolating an untrusted field in the prompt) is printed as advisory, never edited. Breaking someone's CI is worse than staying quiet.

agentpipe local: the laptop half

CI is only half the surface. agentpipe local audits agent configs on the machine itself (~/.claude.json, .mcp.json, cursor/gemini/windsurf configs):

Read-only, no network, secret values never printed.

Clinejection-class demo (issue title → Claude in Actions → npm token):

agentpipe scan examples/clinejection-repo

Run it in your CI (GitHub Action)

Drop agentpipe into a workflow. On a pull request it scans the diff's repo, posts one updatable comment with the chains the PR introduces, and uploads SARIF to the Security tab.

# .github/workflows/agentpipe.yml
name: agentpipe
on: [pull_request]
permissions:
  contents: read
  pull-requests: write      # the sticky PR comment
  security-events: write    # Code Scanning (SARIF)
jobs:
  agentpipe:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: cyberbobas/agentpipe@v0
        with:
          fail-on: high
          comment: true

• One sticky comment. The bot edits its own comment instead of posting a new one per push. Only reachable + high chains go in the headline; everything else folds into a details block. Noise is uninstall, so the default is quiet.
• SARIF to Code Scanning. Every finding shows in the Security tab with real file:line, severity, and a fingerprint GitHub tracks across runs.
• Baseline / diff. On a repo that already carries debt, commit a baseline once and the Action goes silent on it, speaking up only for chains a PR adds:

agentpipe scan . --write-baseline .github/agentpipe-baseline.json   # once, commit it
#  then the Action runs with:  baseline: .github/agentpipe-baseline.json

The same outputs are in the CLI: agentpipe scan . --sarif out.sarif writes SARIF anywhere, and agentpipe comment . posts the sticky comment from any CI that sets GITHUB_TOKEN and GITHUB_REPOSITORY.

Policy: .github/agentpipe.yml

The Action and the CLI both read a per-repo policy file (auto-loaded from the repo, no input needed):

# .github/agentpipe.yml
fail_on: high                 # never | high | medium
comment_on: high              # PR-comment headline floor: high | medium
ignore_paths: ["examples/**"] # findings under these globs are dropped
disabled_detectors: [AWI-15]  # detector ids to silence

A CLI flag overrides the file, the file overrides the built-in default. A repo can loosen its own policy, but the change is a visible line in git history, which is the point.

An org policy (--org-policy <file|url> or AGENTPIPE_ORG_POLICY) sits above a repo's file and can only tighten it: required_detectors a repo may not disable, lock_ignore_paths it may not ignore, and a stricter fail_on wins. A single repo cannot weaken the org's floor with a faceless commit. This is the core of the org-wide App layer (see docs/APP-SPEC.md).

What the scanner flags

Findings carry real file:line of the agent step. A workflow that does not parse is a loud warning, not a silent skip.

Not yet: INS-01..04 (instruction-file hygiene in repos), GitHub Action wrapper, --follow-remote.

What you will not see

A 0–100 score. A badge. Telemetry. A working exploit. An Airlock pitch.

The report ends with what this scan cannot see (that list is the pentest offer).

Verdicts

reachable (trigger + agent + sink) · plausible (untrusted field in the prompt) · adjacent (one piece of the chain). Severity only on the first two.

MIT. Python ≥ 3.11. PyYAML.

python3 tests/run.py           # regression suite (41 checks)
python3 tests/adversarial.py   # parser fuzz, FP/FN matrix, recursion, ReDoS, CLI
python3 tests/matrix.py         # negative matrix per detector, GitHub semantics, live corpus