PkgFoundry
Catalog PyPI
PACKAGE EXPLORERView in registry

aieng-eval-agents

PyPI

Agentic AI Evaluation reference implementation from Vector Institute AI Engineering

Versions collected1
Direct dependencies22From release 0.3.1
Last observed21 Oct, 10:38 UTC

README

Version 0.3.1 documentation

aieng-eval-agents

PyPI Python License

Shared library for Vector Institute's Agentic AI Evaluation Bootcamp. Provides reusable components for building, running, and evaluating LLM agents with Google ADK and Langfuse.

What's included

Agent implementations

ModuleDescription
aieng.agent_evals.knowledge_qaReAct agent that answers questions using live web search. Includes evaluation against the DeepSearchQA benchmark with LLM-as-a-judge metrics (precision/recall/F1).
aieng.agent_evals.aml_investigationAgent that investigates Anti-Money Laundering cases by querying a SQLite database of financial transactions via a read-only SQL tool.
aieng.agent_evals.report_generationAgent that generates structured Excel reports from a relational database based on natural language queries.

Reusable tools (aieng.agent_evals.tools)

  • search — Google Search with response grounding and citations
  • web — HTML and PDF content fetching
  • file — Download and search data files (CSV, XLSX, JSON)
  • sql_database — Read-only SQL database access via ReadOnlySqlDatabase

Evaluation harness (aieng.agent_evals.evaluation)

Wrappers around Langfuse for running agent experiments:

  • run_experiment — Run a dataset through an agent and score outputs
  • run_experiment_with_trace_evals — Run experiments with trace-level evaluation
  • run_trace_evaluations — Score existing Langfuse traces with LLM-based or heuristic graders

Utilities

  • display — Rich-based terminal and Jupyter display helpers for metrics and agent responses
  • progress — Progress tracking for batch evaluation runs
  • configs — Pydantic-based configuration loading from .env
  • langfuse — Langfuse client and trace utilities
  • db_manager — Database connection management

Installation

pip install aieng-eval-agents

Requires Python 3.12+.

Source

Full reference implementations and documentation are in the eval-agents repository.

Downloads & popularity

Awaiting observations

No metric observations yet

Download counts and popularity will appear here as the collector receives them.

Versions 1

Collected releases
VersionPublishedDirect dependenciesOpen version
0.3.117 Apr 202622
Sorted by available source dates · UTC

Package advisories

0
No matching advisories collected

This describes our collected data; it is not a security assessment.

Metadata history

Meaningful changes
Current collected state Current

Observed since 21 Oct, 10:38 UTC

No superseded metadata states collected. Unchanged observations do not create new revisions.

Record identity

package:cb298f96b8e283afafc1f186030a5a582ea103486bb8390f837ae552c4749a81