agenteval-sdk

agenteval_sdk

Python SDK for the AgentEval observability platform. Instrument your AI agents in minutes and get Trace Tree MRI — a full visual breakdown of every agent decision, latency, PII exposure, and cost.

Install

pip install agenteval_sdk

Quick start

import os
from agenteval_sdk import AgentEvalClient, trace

client = AgentEvalClient(
    api_key=os.environ["AGENTEVAL_API_KEY"],
    project_id="aml-system",
)

# Decorate each agent function — span hierarchy is wired automatically
@trace(name="Orchestrator", latency_threshold=5000)
def run_case(case_id: str) -> str:
    result = kyc_agent(case_id)
    return result

@trace(name="KYC_Agent", latency_threshold=3000)
def kyc_agent(case_id: str) -> str:
    output = "KYC: PASS"
    client.log_trace(
        agent_name="KYC_Agent",
        input_text=f"Check case {case_id}",
        output_text=output,
        latency_ms=340,
        tokens_used=180,
        status="pass",
        pii_scan=True,          # lightweight regex pre-flight before Presidio
    )
    return output

run_case("TXN-9921")

This produces a Trace Tree visible in the AgentEval dashboard — parent/child spans, Gantt swimlane, PII audit trail, and cost estimate — with zero manual wiring.

@trace decorator

@trace                                           # bare
@trace(name="MyAgent")                          # custom display name
@trace(name="MyAgent", latency_threshold=2000)  # red UI alert if > 2 s

latency_threshold (ms) controls when the node appears red in the Trace Tree UI.
Nested agents inherit the parent's threshold unless they define their own.

AgentEvalClient.log_trace()

Standalone log_trace()

For lighter usage without AgentEvalClient:

from agenteval_sdk import log_trace

log_trace(
    trace_id="abc-123",
    input_text="Hello",
    output_text="World",
    latency_ms=200,
    tokens_used=50,
    status="pass",
    api_key=os.environ["AGENTEVAL_API_KEY"],
    pii_scan=True,
)

Environment variables

Offline / local-first

AgentEvalClient writes every trace to a local SQLite database before attempting the network POST. No traces are lost on network failure. Call client.flush() to retry unsynced traces.

What you get in the dashboard

• Trace Tree MRI — interactive D3 tree of agent spans, coloured red for slow nodes
• Gantt swimlane — execution timeline per span
• PII Audit Trail — which agents saw PII, what types, redacted by Presidio
• Cost estimate — token usage → USD at GPT-4o-mini rates
• Kill switch — freeze all agent traffic instantly from the operator panel