agent-readiness-scorecard

🤖 Agent Readiness Scorecard

Is your codebase ready for the AI workforce?

agent-readiness-scorecard is a CLI tool that analyzes your Python project to determine how "friendly" it is for AI Agents (like Jules, Copilot, Gemini, or Devin).

AI Agents struggle with:

• Massive Contexts: Large files confuse models and waste token limits.
• Ambiguity: Missing type hints lead to hallucinated parameters.
• Hidden Logic: Complex code flows (high cyclomatic complexity) make it hard for agents to reason about changes.

This tool scores your repo and helps you fix it.

📦 Installation

pip install agent-readiness-scorecard

🚀 Usage

1. Check your Score

Run the scorecard on your current directory:

agent-score .

2. Select an Agent Profile

Different agents have different strengths. Optimize for specific workflows:

• --agent=generic (Default): Balanced checks for readability and types.
• --agent=jules: Strict typing and high documentation requirements (expects agents.md).
• --agent=copilot: Focuses on small file sizes for autocomplete efficiency.

3. Auto-Fix Issues

Bootstrap your repository with the necessary context files and skeleton docstrings:

agent-score . --fix

4. Generate a Markdown Report

Generate a detailed report for your team or CI/CD logs. Control the level of detail with --report-style:

agent-score . --report scorecard.md --report-style actionable

• --report-style=full: Includes all sections and a complete breakdown of every file.
• --report-style=actionable (Default): Focuses on issues; hides passing files and high-coverage type safety rows.
• --report-style=collapsed: (Minimal) Only includes the Executive Summary.

⚙️ Configuration

agent-readiness-scorecard can be configured via pyproject.toml, .agent-readiness-scorecard.json, or CLI flags.

Priority

Settings are resolved in the following order (highest to lowest):

1. CLI Flags (e.g., --agent, --verbosity)
2. Configuration File (pyproject.toml or .agent-readiness-scorecard.json)
3. Defaults

Customizing Output Verbosity

You can control how much visual noise the scorecard generates by adding the verbosity or report_style keys to your pyproject.toml.

Noise-Reduction Configuration

[tool.agent-readiness-scorecard]
# Control CLI output (quiet, summary, detailed)
verbosity = "summary"

# Control Markdown report detail (collapsed, actionable, full)
report_style = "actionable"

[tool.agent-readiness-scorecard.thresholds]
acl_yellow = 10
acl_red = 15
type_safety = 90

# Language-specific overrides
[tool.agent-readiness-scorecard.javascript.thresholds]
acl_yellow = 12
acl_red = 18

[tool.agent-readiness-scorecard.markdown.thresholds]
token_limit = 64000

Verbosity Levels (CLI Output)

Report Styles (Markdown Report)

Used when generating a report via the --report flag.

📊 The Scoring System

Your codebase starts at 100 points. Penalties are applied for:

🛠 Project Structure

To get a perfect score, your project should look like this:

my-project/
├── agents.md          # High-level architecture map for the Agent
├── instructions.md    # Testing/Linting commands for the Agent
└── src/               # Your source code (Typed & Docstringed)

🚀 Beta Features

The beta branch includes upcoming capabilities for enhanced agent-readiness:

• pyproject.toml Configuration: Centralize settings alongside your other tools.
• Prompt Linter (check-prompts): Validate your system prompts against LLM best practices (Role Definition, Few-Shot, etc.).
• Verbosity Control (--verbosity): Choose between quiet, summary (default), or detailed output.
• Enhanced Metrics: Track "Average ACL" and "Average Type Safety" in the summary report.
• CI/CD Automation: Automated badge updates and GitHub Actions workflows for continuous agent-readiness.

🛡 Badges

Show off your Agent-Readiness! Run agent-score --badge to generate an agent_score.svg for your repo.

⚡ CI/CD & Diff Mode

Optimize your CI/CD pipeline by scoring only the files changed in a Pull Request.

Diff-Aware CI Reporting

The provided GitHub Action defaults to Diff-Aware Reporting. This means that in a Pull Request, the scorecard will automatically detect and only score the files you've modified.

Benefits:

• Noise Reduction: Developers only see scores and issues for the code they are actually touching.
• Speed: Analysis is significantly faster on large repositories.
• Project Integrity: While focusing on specific files for scoring, the tool still validates project-wide issues like circular dependencies and global context health.

Local Incremental Analysis

You can use the --diff flag to analyze only the files that have changed compared to a base reference (default: origin/main). This is useful for quickly checking your work before committing.

# Check only changed files (compared to origin/main)
agent-score --diff

# Check changes compared to a specific branch or commit
agent-score --diff --diff-base HEAD^

Manual File Limitation

You can also manually use the --limit-to flag to restrict analysis to a subset of files. You can provide the flag multiple times. If used with --diff, it will analyze the intersection of changed files and the specified files.

# Score only specific files
agent-score score . --limit-to src/core.py --limit-to src/utils.py

📝 Prompt Engineering Linter

Static analysis for your LLM prompts. Ensure your system prompts follow best practices before deploying them to production.

agent-score check-prompts prompts/system_v1.txt

Heuristics Checked:

• Role Definition: Does the prompt establish a persona?
• Cognitive Scaffolding: Are there "Think step-by-step" instructions?
• Delimiter Hygiene: Are instructions separated from data using XML/Markdown tags?
• Few-Shot Examples: Does it include 1-3 examples?
• Negative Constraints: Identifies "Don't" statements and suggests positive alternatives.

---