a11y-fixer

a11y-fixer

An accessibility remediation agent for React JSX/TSX. It scans a file or directory, sends each component through an LLM (Groq, OpenAI, or any OpenAI-compatible endpoint) with a WCAG 2.0/2.2 + ARIA APG system prompt, validates the result structurally, and then either rewrites the source, emits a patch, or reports for CI.

The validation step is what makes the output safe to apply unattended: model output is parsed as JSX/TSX and compared against the original, and a fix is rejected if it fails to parse or drops an event handler or hook call.

Install

pip install a11y-fixer

That gives you the core agent — structural gate, --write / --patch / --check / --fail-on, all providers, the cache, --since. The optional eslint-jsx-a11y feedback loop (--lint) and the eval harness's axe pass (--axe) run a bundled Node package that isn't in the wheel; for those, work from a clone:

git clone https://github.com/arysarin/a11y-fixer && cd a11y-fixer
pip install -e ".[dev]"        # editable install + pytest
npm --prefix lint install      # only needed for --lint / --axe

Set an API key via environment or a local .env (the variable depends on the provider — GROQ_API_KEY by default, OPENAI_API_KEY for --provider openai):

echo "GROQ_API_KEY=gsk-..." > .env

Usage

# dry run: print diffs, change nothing
a11y-fixer src/

# apply validated fixes in place (writes .bak alongside each changed file)
a11y-fixer src/ --write

# CI: check only what this PR changed
a11y-fixer src/ --since origin/main --check

# write validated fixes to a patch instead of touching the tree
a11y-fixer src/ --patch a11y-fixes.patch

# CI gate: exit non-zero if any file still has an accessibility issue
a11y-fixer src/ --check --json-summary a11y-summary.json

Output modes

--write is ignored when combined with --check or --patch.

Exit codes

• 0 — clean, or fixes applied/emitted successfully
• 1 — --check (or --fail-on) found a gated outcome, or a file errored (API failure, truncated response, unreadable file)
• 2 — bad invocation (missing API key, path not found, --workers < 1)
• 130 — interrupted with Ctrl-C (partial results saved)

Options

Observability

--log FILE appends one JSON object per line as the run proceeds — it survives Ctrl-C and crashes, since each line is flushed immediately:

{"event":"run_start","mode":"CHECK","model":"...","files":42, ...}
{"event":"file_start","path":"src/NavBar.jsx"}
{"event":"lint","path":"src/NavBar.jsx","phase":"initial","violations":2}
{"event":"model_call","path":"src/NavBar.jsx","kind":"lint_round","round":1,"prompt_tokens":970,"completion_tokens":734}
{"event":"file_done","path":"src/NavBar.jsx","status":"would-fix","seconds":7.6,"violations_before":2,"violations_after":0,"repair_attempts":1, ...}
{"event":"run_end","duration_seconds":8.6,"interrupted":false,"scanned":42, ...}

Ctrl-C lets in-flight files finish, saves the cache and prints the summary, then exits 130. A second Ctrl-C bails immediately. Parallel runs sharing a .a11y-cache.json merge on save (each writes only the entries it touched) behind a best-effort lock file, so they don't clobber each other.

Linter feedback loop (--lint)

By default the agent trusts the model to find and fix accessibility issues. --lint closes that loop with a real linter:

1. Run eslint-plugin-jsx-a11y over the file.
2. If it's clean, skip the model entirely (no tokens spent).
3. Otherwise pass the exact violations into the prompt, get a fix, and re-lint it.
4. Repeat up to --lint-rounds (default 2) until clean or give up.
5. The structural gate (parse, handlers/hooks) still runs on the final result.

The run summary reports a11y lint N -> M violations, the JSON summary carries per-file violations_before / violations_after / lint_rounds, and under --check any file left with violations fails CI.

Setup — needs Node.js and a one-time install of the bundled ESLint:

cd lint && npm install        # installs eslint + jsx-a11y + the TS parser
a11y-fixer src/ --lint --check

--lint-config FILE points at your project's own ESLint config instead of the bundled a11y-only one. The bundled rule set is folded into the cache key, so changing it re-checks every file.

Project config

Drop an a11y.toml (or .a11y.toml) in the directory you run from — it's auto-discovered (--config FILE to point elsewhere, --no-config to skip). See a11y.toml.example.

primitives = ["Button", "Link", "Modal"]   # already-accessible components; leave them alone
rules = ["Our <Icon> is decorative unless a title prop is set."]
ignore = ["**/*.stories.tsx", "src/legacy/**"]   # skip these files
ext = "jsx,tsx"                              # optional; CLI --ext still wins
exclude = "node_modules,dist,coverage"      # optional; CLI --exclude still wins

primitives and rules are injected into the system prompt so the model stops adding redundant ARIA to your wrapped components. ignore patterns are matched (with fnmatch) against both the working-dir-relative path and the basename. Changing the config changes the prompt, which invalidates the cache.

Providers

The model call goes through a small LLMClient adapter, so any OpenAI-shaped API works:

# Groq (default)
a11y-fixer src/ --check

# OpenAI
a11y-fixer src/ --provider openai --model gpt-4o-mini --check      # needs OPENAI_API_KEY
pip install "a11y-fixer[openai]"                                    # installs the openai SDK

# Any OpenAI-compatible server (Ollama, vLLM, LiteLLM, LM Studio, ...)
a11y-fixer src/ --provider openai-compatible \
  --base-url http://localhost:11434/v1 --model qwen2.5-coder:7b --check

--base-url also lets the openai-compatible provider run without an API key (for local servers that don't need one). Changing provider, model, or base URL invalidates the cache.

Caching

Every run records the content hash of each file it verified as clean (unchanged) or already fixed (fixed) into --cache (default .a11y-cache.json, keyed relative to the working directory). A later run whose file content still matches skips the model call entirely and reports it as unchanged (from cache) with zero tokens.

Only known-good outcomes are cached, so the cache can never suppress a pending fix, a validation failure, or a diff you need to see — would-fix, invalid, and error files are re-processed every run. The whole cache is discarded when the model (--model) or the system prompt changes, since a stale "clean" verdict would no longer be trustworthy.

Add .a11y-cache.json to .gitignore (already done here). Use --no-cache in environments where you want every file re-checked unconditionally.

How validation works

For each file the agent builds a structural fingerprint (via tree-sitter) of the original and the model output:

1. Parse gate — if the original parsed cleanly and the fix does not, the fix is rejected (invalid) and never written.
2. Handler / hook preservation — any on* JSX attribute or use* hook call present in the original but gone from the fix rejects it.
3. --strict — additionally rejects if any other identifier drops to zero references (noisier: legitimate refactors like window.location.href = … → <a href> trip this).

If the original file does not parse, all checks are skipped (no trustworthy baseline) and the fix is written as-is under --write.

Repair pass — when the structural gate rejects a fix, it is sent back once (configurable with --repair-attempts N, default 1, 0 disables) with the exact reason (event handlers removed: onClick, does not parse, …) and re-validated. A recovered fix proceeds normally; the summary reports repair K/N recovered. This costs one extra model call per rejected file.

A truncated or empty model response is always an error, never a partial write.

File content is treated strictly as code to remediate; the system prompt tells the model to ignore any instructions embedded in it (prompt-injection guard), and the structural gate above is the real backstop regardless. The eval corpus carries prompt-injection fixtures (override text in a JS comment and in rendered JSX) that check this holds against the live model.

Files are read as UTF-8; a byte-order mark and CRLF line endings are detected and preserved on write-back. A non-UTF-8 file is reported as an error rather than guessed at.

--since REF narrows the run to files that differ from a git ref — tracked modifications plus new untracked files, deletions excluded. A run with nothing changed exits 0 ("No changed files to process"); an unknown ref or a non-git directory exits 2.

CI

See .github/workflows/a11y.yml: runs --check on pull requests touching .jsx/.tsx, and a manual workflow_dispatch produces a fix patch as a build artifact. Requires a GROQ_API_KEY repository secret (or OPENAI_API_KEY if you switch --provider).

To stop the build only on a fix that failed the safety gate — while letting merely-unremediated files through — use --fail-on:

a11y-fixer src/ --since origin/main --lint --fail-on invalid

Opening a PR with the fixes

a11y_pr.py (console script a11y-fixer-pr) runs the agent and turns the result into a pull request. Run it from the repo root with a clean tracked tree:

a11y-fixer-pr src/ --lint --label accessibility

It runs a11y-fixer src/ --lint --patch … --json-summary … (any extra args pass through), and if there are validated fixes: branches from HEAD (--branch, default a11y-fixes), applies the patch, commits (message carries the lint delta), force-pushes, and creates or refreshes a PR through the gh CLI. The PR body is a per-file table (status, violations_before → after, repair passes) with token/cost totals and a note that rejected fixes were left out. --dry-run stops after the local commit; without gh it pushes and prints the compare instructions. .github/workflows/a11y-pr.yml wires it to workflow_dispatch.

Limitations

• Without --lint, semantic correctness of a fix is not verified — only that it parses and keeps handlers/hooks. --lint verifies against eslint-plugin-jsx-a11y; axe-core on rendered output would be stronger still.
• The bundled lint/ directory ships in source checkouts and editable installs (pip install -e .); a plain wheel install does not carry it, so run --lint from a checkout.
• Each file is processed in isolation; the agent has no knowledge of shared design-system components (an a11y.toml mitigates this).
• Cost figures in the summary are estimates from MODEL_PRICING in a11y_fixer.py; update the rates for your model.

Tests

pytest

.github/workflows/ci.yml runs the suite on Python 3.10 / 3.11 / 3.12 on every push and PR, plus a smoke check that the bundled eslint-plugin-jsx-a11y config still flags a known violation and that the package builds with its runtime data files (lint/, a11y.toml.example).

Releasing

a11y_fixer.__version__ is the single source of truth; pyproject.toml reads it dynamically, so a release is: bump that string, move the CHANGELOG.md [Unreleased] items under a dated version heading, tag vX.Y.Z. a11y-fixer --version reports the installed version.

Evals

evals/ holds golden fixtures and a scorer for measuring a prompt / model / validation change rather than eyeballing it:

GROQ_API_KEY=... python evals/run.py --lint --json after.json
GROQ_API_KEY=... python evals/run.py --lint --axe   # also render + run axe-core

Reports fix_rate, false_positive_rate, regression_rate, and (with --axe) axe_clean_rate; exits non-zero below --threshold (default 1.0). --axe renders each opted-in fixture's fixed component and runs axe-core against it in jsdom, checking the fix actually helps assistive tech rather than just passing the linter. See evals/README.md.