agent-regression-lens-py

agent-regression-lens-py

Detect regressions between baseline and current AI agent runs. Pure Python, zero deps. Python port of @mukundakatta/agent-regression-lens.

pip install agent-regression-lens-py

from agent_regression_lens import analyze_agent_regression

baseline = {"events": [{"type": "tool_call", "tool": "search"}, {"type": "final", "content": "ok"}], "success": True}
current  = {"events": [{"type": "tool_call", "tool": "search"}, {"type": "error", "message": "boom"}], "success": False}

res = analyze_agent_regression(baseline, current)
# RegressionResult(passed=False, score=..., regressions=[Issue(code='lost_success', severity='critical', ...), ...])

API

• analyze_agent_regression(baseline, current, *, thresholds=None) -> RegressionResult
• compare_run_set(baseline_runs, current_runs, *, thresholds=None) -> ComparisonSummary

Detects: lost success, new errors, failed tool calls, output drift (Jaccard), step / latency / cost bloat above configurable ratios.

License

MIT