aieng-bot

aieng-bot

---

AI-powered tool that autonomously fixes CI failures, resolves merge conflicts, and merges GitHub pull requests using Claude AI.

Features

• Fix any PR - Use the CLI to fix CI failures on any GitHub pull request
• Auto-fix - Automatically fixes test failures, linting issues, security vulnerabilities, merge conflicts, and build errors
• Smart classification - Categorizes failures and applies appropriate fix strategies
• Organization-wide automation - Includes workflows to scan VectorInstitute repos daily for bot PRs (Dependabot, pre-commit-ci)
• Observable - Full execution tracing with dashboard analytics
• Transparent - Comments on PRs with status updates

Architecture

┌─────────────────────────────────────────────────────────────┐
│  aieng-bot                                                  │
│                                                             │
│  CLI Usage (any PR):                                        │
│    aieng-bot fix --repo owner/repo --pr 123                 │
│                                                             │
│  Automated Workflows (bot PRs only):                        │
│    Daily scan (00:00 UTC) for Dependabot/pre-commit-ci PRs  │
│    with failing checks across VectorInstitute org           │
└─────────────────────────────────────────────────────────────┘

Quick Start

Setup (in this repository)

1. Create Anthropic API Key

• Get from Anthropic Console
• Add as repository secret: ANTHROPIC_API_KEY

2. Create GitHub Personal Access Token

• Go to Settings → Developer settings → Personal access tokens → Fine-grained tokens
• Configure: Resource owner: VectorInstitute, Repository access: All repositories
• Permissions: contents: write, pull_requests: write, issues: write
• Add as repository secret: ORG_ACCESS_TOKEN

3. Enable GitHub Actions

• Go to Actions tab → Enable workflows

The bot now monitors all VectorInstitute repositories automatically.

How It Works

1. Classification

• Analyzes PR and failure logs using Claude Haiku 4.5
• Categorizes failure type: test, lint, security, build, or merge conflict
• Routes to appropriate fix strategy

2. Fix Loop

• Clones target repository and PR branch
• Loads appropriate AI prompt template for the failure type
• Uses Claude Agent SDK to automatically apply fixes
• Commits and pushes fixes to PR
• Polls CI status, retries on failure (up to max_retries)

3. Merge

• Auto-merges when all checks pass
• Comments on PR with status updates

Automated Bot PR Monitoring (optional, for VectorInstitute org)

• Daily scan at 00:00 UTC for Dependabot/pre-commit-ci PRs with failures
• Automatically dispatches fix jobs for discovered PRs

Configuration

Required Secrets

• ANTHROPIC_API_KEY - Anthropic API access for Claude
• ORG_ACCESS_TOKEN - GitHub PAT with org-wide permissions

Model Configuration

• Classification: Claude Haiku 4.5 (claude-haiku-4-5-20251001) - cost-efficient
• Fixing: Claude Sonnet 4.5 (claude-sonnet-4-5-20250929) - agentic capability
• Override with CLAUDE_MODEL environment variable

Workflows

• discover-and-dispatch.yml - Daily scan (00:00 UTC) for Dependabot/pre-commit-ci PRs with failures
• fix-pr-agent.yml - Per-PR agent workflow (6-hour timeout), can be triggered manually for any PR
• code_checks.yml - Ruff + mypy checks
• unit_tests.yml - pytest suite

AI Prompt Templates (customize for your needs)

• fix-merge-conflicts.md - Resolve merge conflicts with best practices
• fix-test-failures.md - Test failure resolution strategies
• fix-lint-failures.md - Linting/formatting fixes
• fix-security-audit.md - Security vulnerability handling
• fix-build-failures.md - Build/compilation error fixes

Capabilities

Can fix:

• Merge conflicts (dependency files, lock files, code)
• Linting and formatting issues
• Security vulnerabilities (dependency updates)
• Simple test failures from API changes
• Build configuration issues

Cannot fix:

• Complex logic errors
• Breaking changes requiring refactoring
• Issues requiring architectural decisions

CLI Usage

# Install dependencies
uv sync

# Fix and merge a PR
aieng-bot fix --repo owner/repo --pr 123

# Fix with dashboard logging
aieng-bot fix --repo owner/repo --pr 123 --log

# Custom retries and timeout
aieng-bot fix --repo owner/repo --pr 123 --max-retries 5 --timeout-minutes 180

Manual Testing

Trigger via CLI:

# Fix a PR using the CLI (recommended)
aieng-bot fix --repo owner/repo --pr 123

# Or trigger via GitHub workflow
gh workflow run fix-pr-agent.yml \
  --field target_repo="owner/repo" \
  --field pr_number="123"

# Run the bot PR discovery workflow (VectorInstitute org only)
gh workflow run discover-and-dispatch.yml

Trigger via GitHub UI: Actions → Select workflow → Run workflow → Enter parameters

Dashboard

View comprehensive analytics and agent execution traces:

• 📊 Dashboard - Interactive dashboard with:
  • Overview table of all PR fixes
  • Success rates and performance metrics
  • Detailed agent execution traces (like LangSmith/Langfuse)
  • Code diffs with syntax highlighting
  • Failure analysis and reasoning timeline

Features:

• Real-time PR status tracking
• Agent observability (tool calls, reasoning, actions)
• Historical metrics and trends
• Per-repo and per-failure-type analytics
• Sortable/filterable PR table

Authentication:

• Restricted to @vectorinstitute.ai email addresses
• Google OAuth 2.0 sign-in

Monitoring

View activity:

• Dashboard - Comprehensive analytics and traces
• Actions tab - All workflow runs and success/failure rates
• PR comments - Detailed status updates on each PR
• Run summary - PR count and actions taken per run

Debug commands:

# View recent workflow runs
gh run list --workflow=discover-and-dispatch.yml --limit 5

# View logs for specific run
gh run view RUN_ID --log

Documentation

• Setup Guide - Configuration and permissions
• Deployment Guide - Rollout and monitoring

Troubleshooting

See Setup Guide for detailed troubleshooting.

---

🤖 aieng-bot - AI-powered PR maintenance by Vector Institute AI Engineering