Catalog PyPI
PACKAGE EXPLORERView in registry

agentcontrolroom

PyPI

The Reliability Layer for Autonomous AI Agents — full tracing, cost intelligence, guardrails, and quality evaluation.

agentsaianthropicautogencrewaigeminigroqguardrailslangchainlanggraphllamaindexllmmistralmonitoringobservabilityollama
All keywords (20)
openaiopentelemetrypydantic-aitracing
Versions collected1
Direct dependencies22From release 0.5.0
Last observed20 Oct, 21:42 UTC

README

Version 0.5.0 documentation

🧭 Why Agent Control Room

Autonomous agents fail in ways traditional APM tools were never designed to catch — runaway loops, silent cost overruns, degraded reasoning quality, and multi-step tool chains that are impossible to debug from logs alone.

Agent Control Room (ACR) is an open-source reliability layer purpose-built for agents. Drop it into an existing stack in seconds and immediately get full-fidelity tracing, live cost tracking, and safety guardrails — no code changes required.

# Before ACR — flying blind, must add callbacks everywhere manually
llm = ChatMistralAI(model="mistral-small-2506", callbacks=[acr_cb])
chain.with_config(callbacks=[acr_cb])   # ← tedious, error-prone

# After ACR v0.3.0 — TRUE zero-effort, zero code changes
acr-run python agents.py
# ↑ That's it. Every LLM call, tool, loop, cost → ACR dashboard instantly.

Every LLM call, tool invocation, and agent decision is captured, costed, and monitored in real time. When an agent starts looping, blows through budget, or degrades in quality, ACR flags it before it becomes a production incident.


✨ Features

🔍 Real-Time Tracing

Every LLM call, tool use, chain, retrieval, and agent step captured as OpenTelemetry-aligned spans — visible in the Trace Inspector with exact latency and payloads.

💰 Cost Intelligence

Token usage and spend broken down per run, per agent, and per model. Supports OpenAI, Anthropic, Google, Mistral, Meta Llama, and more.

🛡️ Guardrails & Safety

Cost limits, latency budgets, loop detection, quality gates, and PII redaction — enforced client-side and server-side.

🔐 Enterprise & Security

SAML/OIDC SSO, immutable audit logging, field-level encryption, SOC 2 readiness, and PII redaction out of the box.

📊 Live Dashboard

A real-time control room for runs, spans, costs, guardrails, evaluations, and audit logs — all in one place.

🤖 Universal Auto-Instrumentation

acr-run CLI auto-detects and patches every installed AI framework at startup — no callbacks, no decorators, no code changes.

🧪 LLM-as-a-Judge Evaluations

Auto-score agent outputs for hallucination, faithfulness, correctness, and relevance using Groq (free, fast).

🌐 Multi-Agent Support

Hierarchical run trees link orchestrator runs to sub-agent runs via parent_run_id, visible in the dashboard.


🔌 Supported Frameworks (v0.4.0) — 13 Frameworks

FrameworkIntegrationHow
LangChainACRCallbackHandlerAttach to any LLM or chain
LangGraphinstrument_langgraph()One call, patches all graphs
CrewAIinstrument_crewai()Patches Crew.kickoff, Agent.execute_task, BaseTool.run
AutoGen / AG2instrument_autogen()Patches initiate_chat, reply hooks, function execution
LlamaIndexinstrument_llamaindex()Patches LLM, Retriever, QueryEngine, Embeddings
OpenAIwrap_openai()Wraps openai.OpenAI() / AsyncOpenAI() — sync + async + streaming
Groq ⭐wrap_groq() / instrument_groq()Llama3, Mixtral, Gemma via Groq cloud
Anthropic ⭐wrap_anthropic() / instrument_anthropic()Claude 3.5 / Opus 4.x — tool use spans included
Google Gemini ⭐wrap_gemini() / instrument_gemini()google-generativeai and new google-genai SDK (2025)
Mistral AI ⭐wrap_mistral() / instrument_mistral()Native Mistral API — large, small, codestral
Ollama ⭐wrap_ollama() / instrument_ollama()Local inference — tokens tracked, cost always $0.00
Google ADK ⭐instrument_google_adk()Monkey-patch + OTel exporter for Google Agent Dev Kit
Pydantic AI ⭐instrument_pydantic_ai()Wraps Agent.run / Agent.run_sync with full token attribution
Any Python@trace.agent / @trace.toolDecorator-based, framework-agnostic
Any languageREST APIPOST /api/v1/ingest/traces — works from JS, Go, Java, etc.

⭐ New in v0.4.0

Install all integrations at once:

pip install "agentcontrolroom[all]"

Or install only what you need:

pip install "agentcontrolroom[groq]"
pip install "agentcontrolroom[anthropic]"
pip install "agentcontrolroom[gemini]"
pip install "agentcontrolroom[mistral]"

🏗️ Architecture

flowchart TB
    A["🤖 Your AI Agents\nLangChain · CrewAI · AutoGen · LlamaIndex · Custom"] -->|"acr-run python app.py\n(zero code changes)"| B

    subgraph B["🎛️ Agent Control Room"]
        direction TB
        C["📥 Ingest API\n/api/v1/ingest/traces"]
        D["🧪 Groq LLM Judge\nAuto-evaluation"]
        E[("🗄️ PostgreSQL + Redis")]
        F["📊 React Dashboard\nRuns · Spans · Costs · Guardrails"]
        G["🔐 Enterprise Layer\nSSO · Audit Logs · PII Redaction"]

        C --> E
        D --> E
        E --> F
        G --> E
    end

    B --> H["🚨 Guardrail Alerts\nCost · Latency · Loops · Quality"]

    style A fill:#0ea5e9,color:#fff,stroke:#0369a1,stroke-width:2px
    style B fill:#f8fafc,stroke:#6366f1,stroke-width:2px
    style E fill:#6366f1,color:#fff,stroke:#4338ca,stroke-width:2px
    style F fill:#22c55e,color:#fff,stroke:#15803d,stroke-width:2px
    style H fill:#ef4444,color:#fff,stroke:#b91c1c,stroke-width:2px

Stack: FastAPI · SQLAlchemy · PostgreSQL · Redis · Dramatiq · React · Vite · OpenTelemetry


⚡ Quick Start

Get from zero to a fully traced agent in under 60 seconds.

Option 1 — acr-run (TRUE zero-effort, no code changes)

# 1. Install
pip install agentcontrolroom

# 2. Set ONE environment variable — that's all
export ACR_API_KEY=your-api-key-here
# (endpoint defaults to ACR cloud automatically)

# 3. Run your EXISTING, UNMODIFIED project
acr-run python agents.py

What you'll see:

──────────────────────────────────────────────────────
  ACR Auto-Instrumentation Runner  v0.3.0
──────────────────────────────────────────────────────
  Scanning for AI frameworks...

  ✅ LangChain  — global callback injected (zero-effort)
  ✅ LangGraph  — monkey-patched (zero-effort)
  ✅ CrewAI     — monkey-patched (zero-effort)
  ✅ AutoGen    — monkey-patched (zero-effort)
  ✅ LlamaIndex — monkey-patched (zero-effort)
  ✅ OpenAI     — client auto-wrapped (zero-effort)

  6 frameworks instrumented. All traces → ACR dashboard.
──────────────────────────────────────────────────────

Option 2 — Manual (per-framework, one line)

# LangChain / LangGraph
from agentcontrolroom.instruments.langchain import ACRCallbackHandler
llm = ChatMistralAI(model="mistral-small-2506", callbacks=[ACRCallbackHandler()])

# CrewAI
from agentcontrolroom import instrument_crewai
instrument_crewai()   # call before Crew/Agent creation

# AutoGen
from agentcontrolroom import instrument_autogen
instrument_autogen()  # call before creating agents

# LlamaIndex
from agentcontrolroom import instrument_llamaindex
instrument_llamaindex()  # call before using any index/query engine

# OpenAI direct
from agentcontrolroom import wrap_openai
import openai
client = wrap_openai(openai.OpenAI())

Option 3 — Decorator-based (any Python framework)

from agentcontrolroom import trace, configure

configure(api_key="acr-dev-xxxx")

@trace.agent(name="research-agent")
def research(query: str):
    result = my_llm.invoke(query)
    return result

@trace.tool(name="web-search")
def search(query: str):
    return requests.get(f"https://api.search.com?q={query}").json()

Verify Setup

acr-check
==========================================================
  Agent Control Room — System Health Check
==========================================================
  ✅ Backend reachable   (version: 0.3.0)
  ✅ API key valid
  ✅ Test trace ingested (1 span stored)
  ✅ Dashboard reachable
  🎉 All checks passed! (4/4)
==========================================================

🔌 Model Context Protocol (MCP) Server

Connect your AI coding assistants (Cursor, Windsurf, Claude Desktop, Antigravity, VS Code) directly to Agent Control Room. Ask your IDE's assistant to debug failed runs, audit token spend, and inspect traces in chat without leaving your code editor!

1. Cursor Setup (Settings -> MCP -> Add New MCP Server)

  • Name: acr
  • Type: command
  • Command: uvx agentcontrolroom mcp
  • Environment Variables:
    • ACR_API_KEY: acr-dev-xxxx
    • ACR_ENDPOINT: https://acr-backend-xlj5.onrender.com

2. Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "acr": {
      "command": "uvx",
      "args": ["agentcontrolroom", "mcp"],
      "env": {
        "ACR_API_KEY": "acr-dev-xxxx",
        "ACR_ENDPOINT": "https://acr-backend-xlj5.onrender.com"
      }
    }
  }
}

3. Available MCP Tools:

  • acr_list_runs: List recent agent runs with latency, cost, and status.
  • acr_get_run_trace: Fetch full execution tree, child spans, prompts, and completions.
  • acr_diagnose_failure: Pinpoint the exact failing span, triggering input, and stack trace so the AI can fix your code.
  • acr_get_cost_summary: Analyze token spend across LLM models and detect anomalies.
  • acr_list_guardrails: View active budget, latency, and loop detection rules.
  • acr_create_guardrail: Create or adjust safety rules via natural language.

Deployment Options

OptionBest forSetup
Cloud SaaSFastest way to get started, no infra to manage
pip install agentcontrolroom
export ACR_API_KEY=<your-key>
# ACR_ENDPOINT defaults to cloud automatically — nothing else needed
🖱️ One-click RenderFree tier, fully managed, self-hosted

Deploy to Render

Deploys PostgreSQL, Redis, the FastAPI backend, and the React dashboard automatically.

💻 Local devDevelopment & debugging
git clone https://github.com/Darshan-1812/Helm-AI
cd Helm-AI
cp .env.example .env  # fill in your values
docker compose up -d

Then open http://localhost:3000.

Full guide including Railway, VPS, and enterprise VPC setups: docs/deployment.md


📚 Examples

ExampleDescription
MSME Hallucination Demo(Recommended) Visual demo triggering LLM-as-a-Judge and Cost Guardrails
LangChain QuickstartZero-config LangChain monitoring
CrewAI IntegrationMulti-agent crew with full trace hierarchy
AutoGen IntegrationConversational agent tracing
LlamaIndex RAGRAG pipeline with retrieval + reranking spans
Manual TracingInstrument any framework by hand
Cost GuardrailsSet budget limits with alerts
Health CheckVerify your ACR setup

🗂️ Repository Structure

Helm-AI/
├── backend/                    FastAPI + PostgreSQL + Redis
│   ├── app/
│   │   ├── api/                REST endpoints (runs, ingest, guardrails, auth, ...)
│   │   ├── models/             SQLAlchemy ORM models
│   │   ├── services/           Business logic (tracing, eval, cost, PII, ...)
│   │   └── workers/            Dramatiq background workers
│   ├── alembic/                Database migrations
│   └── Dockerfile.prod
├── sdk/                        agentcontrolroom Python package (PyPI)
│   └── src/agentcontrolroom/
│       ├── instruments/
│       │   ├── langchain.py    LangChain + LangGraph handler
│       │   ├── crewai.py       CrewAI monkey-patcher
│       │   ├── autogen.py      AutoGen / AG2 monkey-patcher
│       │   ├── llamaindex.py   LlamaIndex monkey-patcher  ← NEW v0.3.0
│       │   └── openai.py       OpenAI client wrapper
│       ├── tracer.py           @trace.agent / @trace.tool decorators
│       ├── guardrails.py       Client-side guardrail engine
│       ├── cost.py             Token cost calculator (OpenAI/Anthropic/Mistral/...)
│       └── _cli.py             acr-check + acr-run CLI  ← NEW v0.3.0
├── frontend/                   React + Vite dashboard
│   └── src/components/
│       └── DashboardView.tsx   Main dashboard (Traces, Cost, Guardrails, Agents, ...)
├── sdk-node/                   @agentcontrolroom/node TypeScript SDK
├── docs/                       Documentation
├── examples/                   Working code examples
├── render.yaml                 One-click Render deploy
└── docker-compose.yml          Local development

🕒 Changelog

🟢 v0.4.0
  • 7 new framework instruments — Groq, Anthropic, Google Gemini, Mistral AI, Ollama, Google ADK, Pydantic AI
  • 13 total supported frameworks — most universal AI agent observability SDK in the Python ecosystem
  • Prompt Playground — built-in UI to prototype prompts directly from the dashboard with real inference and automatic trace capture
  • Comprehensive pricing tables — updated cost calculator with Claude 4.x, Gemini 2.5, Groq-hosted models
  • pip install agentcontrolroom[groq/anthropic/gemini/mistral/ollama] — granular provider installs
  • 5 new unit test files — test coverage for all new framework wrappers
🔵 v0.3.0
  • acr-run CLI — TRUE zero-effort auto-instrumentation. Run any project with no code changes: acr-run python agents.py
  • Full LlamaIndex support — instrument_llamaindex() patches LLM, Retriever, QueryEngine, Embeddings and Reranker
  • pip install agentcontrolroom[all] — installs all framework integrations at once
  • Dynamic model extraction — dashboard correctly shows Mistral, Claude, Llama, etc. from live telemetry instead of hardcoded values
  • Live Trace Inspector — clicking a run fetches and renders all child spans with prompts and tool outputs
🔵 v0.2.3
  • CrewAI auto-instrumentation — instrument_crewai() with full crew/agent/tool hierarchy
  • AutoGen / AG2 support — instrument_autogen() for both v0.2 and v0.4
  • LangGraph support — instrument_langgraph() for graph-based agents
  • OpenAI direct wrapper — wrap_openai() for the raw OpenAI Python SDK
🔵 v0.2.0
  • LangChain ACRCallbackHandler with zero-config integration
  • Groq LLM-as-a-Judge auto-evaluation (hallucination, faithfulness, correctness, relevance)
  • acr-check CLI health checker
  • SOC 2 Phase 4: data retention policies, encryption at rest
⚫ v0.1.0Core SDK: trace ingestion, cost tracking, run management, guardrails engine

📖 Documentation


📄 Citation

If you use Agent Control Room in research or production, please cite:

@software{agentcontrolroom2026,
  author  = {Darshan Girase},
  title   = {Agent Control Room: Reliability Layer for Autonomous AI Agents},
  year    = {2026},
  url     = {https://github.com/Darshan-1812/Helm-AI},
  version = {0.3.0}
}

📜 License

MIT License — see LICENSE for details.


Built with ❤️ by Darshan Girase

Downloads & popularity

Awaiting observations

No metric observations yet

Download counts and popularity will appear here as the collector receives them.

Versions 1

Collected releases
VersionPublishedDirect dependenciesOpen version
0.5.017 Oct 202622

Package advisories

0
No matching advisories collected

This describes our collected data; it is not a security assessment.

Repository

PENDING
darshan-1812/helm-ai

Shared repository data · GitHub

License detected in repository
Not collected

Metadata history

Meaningful changes
Current collected state Current

Observed since 20 Oct, 21:42 UTC

No superseded metadata states collected. Unchanged observations do not create new revisions.

Record identity

package:12dc3018a31ea8212770812daafea53d8f967fabb605daceba7313f0243146e8