PkgFoundry
Catalog PyPI
Back to packageRELEASE DETAILS

0.3.1

PyPI

aieng-eval-agents

Agentic AI Evaluation reference implementation from Vector Institute AI Engineering

Published17 Apr 2026
Direct dependencies22
Transitive dependenciesNot calculated

README

Version 0.3.1 documentation

aieng-eval-agents

PyPI Python License

Shared library for Vector Institute's Agentic AI Evaluation Bootcamp. Provides reusable components for building, running, and evaluating LLM agents with Google ADK and Langfuse.

What's included

Agent implementations

ModuleDescription
aieng.agent_evals.knowledge_qaReAct agent that answers questions using live web search. Includes evaluation against the DeepSearchQA benchmark with LLM-as-a-judge metrics (precision/recall/F1).
aieng.agent_evals.aml_investigationAgent that investigates Anti-Money Laundering cases by querying a SQLite database of financial transactions via a read-only SQL tool.
aieng.agent_evals.report_generationAgent that generates structured Excel reports from a relational database based on natural language queries.

Reusable tools (aieng.agent_evals.tools)

  • search — Google Search with response grounding and citations
  • web — HTML and PDF content fetching
  • file — Download and search data files (CSV, XLSX, JSON)
  • sql_database — Read-only SQL database access via ReadOnlySqlDatabase

Evaluation harness (aieng.agent_evals.evaluation)

Wrappers around Langfuse for running agent experiments:

  • run_experiment — Run a dataset through an agent and score outputs
  • run_experiment_with_trace_evals — Run experiments with trace-level evaluation
  • run_trace_evaluations — Score existing Langfuse traces with LLM-based or heuristic graders

Utilities

  • display — Rich-based terminal and Jupyter display helpers for metrics and agent responses
  • progress — Progress tracking for batch evaluation runs
  • configs — Pydantic-based configuration loading from .env
  • langfuse — Langfuse client and trace utilities
  • db_manager — Database connection management

Installation

pip install aieng-eval-agents

Requires Python 3.12+.

Source

Full reference implementations and documentation are in the eval-agents repository.

Downloads & popularity

Awaiting observations

No metric observations yet

Download counts and popularity will appear here as the collector receives them.

Declared dependencies 22

Native requirements
22 direct targetsNot calculated transitive count
Package / targetRequirementRelationship
google-adk
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
google-adk>=1.23.0runtime
google-genai
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
google-genai>=1.52.0runtime
gradio
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
gradio>=6.7.0runtime
html-to-markdown
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
html-to-markdown>=2.24.0runtime
httpx
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
httpx>=0.27.0runtime
kagglehub
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
kagglehub>=0.4.1runtime
langfuse
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
langfuse>=3.10.3runtime
openai-agents
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
openai-agents>=0.6.1runtime
openai
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
openai>=2.8.1runtime
openinference-instrumentation-google-adk
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
openinference-instrumentation-google-adk>=0.1.0runtime
opentelemetry-exporter-otlp-proto-http
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
opentelemetry-exporter-otlp-proto-http>=1.20.0runtime
opentelemetry-sdk
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
opentelemetry-sdk>=1.20.0runtime
pandas
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
pandas>=2.3.0runtime
pydantic-settings
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
pydantic-settings>=2.12.0runtime
pydantic
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
pydantic>=2.12.0runtime
pypdf
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
pypdf>=6.7.5runtime
python-dotenv
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
python-dotenv>=1.0.0runtime
rich
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
rich>=13.9.0runtime
sqlalchemy
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
sqlalchemy>=2.0.46runtime
sqlglot
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
sqlglot>=28.6.0runtime
tenacity
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
tenacity>=8.2.0runtime
weaviate-client
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<4.0,>=3.12",
  "evaluation": "NOT_EVALUATED"
}
weaviate-client>=4.18.3runtime

Distribution files

2 collected
aieng_eval_agents-0.3.1-py3-none-any.whl

bdist_wheel · 204.16 KB · Python py3

Source file
aieng_eval_agents-0.3.1.tar.gz

sdist · 158.7 KB · Python source

Source file

Metadata history

Meaningful changes
Current collected state Current

Observed since 21 Oct, 10:38 UTC

No superseded metadata states collected. Unchanged observations do not create new revisions.

Record identity

package_version:c8d73458e7d645e75038a4316107e028cd6bcdae80da5298cf0d7a42300a49a1

Other collected versions

View all collected versions →