agentsiq

AgentsIQ — Intelligent Multi-Model Router 🚀

🌟 What Makes AgentsIQ Intelligent

• 🧠 Intelligent Selection: Automatically picks the best model based on cost, latency, and quality
• 📊 Comprehensive Benchmarking: Beautiful graphs showing model performance across different tasks
• 💰 Cost Optimization: Save up to 90% on API costs while maintaining quality
• 🏠 Local Models: Full Ollama support for privacy and zero-cost inference
• ⚡ Real-time Analytics: Live dashboard with performance metrics and decision explanations
• 🎯 Task-Aware Routing: Different models excel at different tasks (coding, summarization, creative writing)
• 🔧 Zero Configuration: Works out of the box with smart defaults

Why it’s unique

• Decision intelligence: objective function balances cost, latency, quality with task-aware tweaks (code/summarize/math).
• Explainability: logs a full rationale (normalized scores, est. tokens, est. cost, savings vs next‑best & vs GPT‑4o).
• Live control: adjust strategy/weights on the fly via /control without redeploying.
• Shared router: RouterManager ensures both the runner and dashboard operate on the same model router instance in a single process (examples/serve_and_demo.py).
• Observability: dark dashboard with sparklines, p50/p90/p99 bars, raw logs and CSV metrics; AgentOps auto‑init if key present.
• Config‑first: override profiles/weights in config.yaml — no code edits required.

🚀 Quick Start

Installation

Option 1: Install from PyPI (Recommended)

pip install agentsiq

Option 2: Install from Source

git clone https://github.com/yourusername/AgentsIQ.git
cd AgentsIQ
python -m venv venv && source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -e .

Option 3: Development Installation

git clone https://github.com/yourusername/AgentsIQ.git
cd AgentsIQ
python -m venv venv && source venv/bin/activate
pip install -e ".[dev,docs]"

Environment Setup

Create a .env file with your API keys:

# Required for cloud models
OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key
GOOGLE_API_KEY=your_google_key
GROK_API_KEY=your_grok_key

# Optional: Ollama configuration (defaults to localhost:11434)
OLLAMA_URL=http://localhost:11434

# Optional: AgentOps for advanced analytics
AGENTOPS_API_KEY=your_agentops_key

Run the Enhanced Benchmark

python examples/benchmark.py

Choose option 1 for the comprehensive model comparison with beautiful visualizations!

📓 Jupyter Notebook Examples

# Start Jupyter Lab
jupyter lab examples/notebooks/

# Or start Jupyter Notebook
jupyter notebook examples/notebooks/

Available Notebooks:

• 🔍 Basic Search Agent - Learn fundamentals with simple search agents
• 🚀 Advanced Search Agent - Multi-agent collaboration and web search
• 🔬 Research Agent - Deep research and analysis capabilities
• 🎯 Complete Agent System - Enterprise-grade multi-agent systems

See Notebook Examples for detailed learning paths and tutorials.

📊 Supported Models

Provider	Models	Cost/1K tokens	Best For	Logo
	GPT-4o, GPT-4o-mini	$5.00, $0.15	Code generation, complex reasoning	🤖
	Claude-3-Haiku	$0.25	Summarization, analysis	🧠
	Gemini-Pro	$0.125	General purpose, fast responses	🔍
	Llama3.1 (8B/70B), Qwen2.5 (7B/72B)	$0.00	Local inference, privacy	🏠
	Grok-2, Grok-2-Vision	$0.20-0.25	Creative writing, humor	😄

🎯 Usage Examples

Basic Usage

from agentsiq.router import ModelRouter

router = ModelRouter()
model = router.select_model("Write a Python function to sort a list")
response, quality = router.call_model(model, "Write a Python function to sort a list")
print(f"Selected: {model}, Quality: {quality}")

📖 Want to understand how the intelligent selection works?

Check out our Architecture Documentation for a detailed explanation of the multi-objective optimization algorithm and decision-making process.

Multi-Agent Collaboration

from agentsiq.agent import Agent
from agentsiq.collab import Collab
from agentsiq.router import ModelRouter

researcher = Agent("Researcher", "Finds information", "openai:gpt-4o-mini", ["retrieval"])
analyst = Agent("Analyst", "Summarizes info", "anthropic:claude-3-haiku", ["summarize"])
router = ModelRouter()
collab = Collab([researcher, analyst], {"retrieval": lambda _: "[retrieval] ok", "summarize": lambda _: "[summary] ok"})

result = collab.run("Analyze the latest trends in AI")

🖥️ Dashboard & Analytics

Run the Dashboard

python -m examples.serve_and_demo
# open http://127.0.0.1:8000  → Summary, Decisions, Control, Health

Features

• Real-time Performance Metrics: Live charts showing model selection patterns
• Cost Analysis: Track savings vs baseline models
• Decision Explanations: See exactly why each model was chosen
• Control Panel: Adjust routing strategy and weights on the fly

📈 Advanced Benchmarking

The enhanced benchmark system provides comprehensive analysis across multiple dimensions:

Benchmark Features

• Multi-Task Testing: Tests models across coding, summarization, creative writing, technical analysis, and problem-solving
• Performance Visualizations: Beautiful charts showing response times, costs, and quality scores
• Trade-off Analysis: Scatter plots revealing cost vs quality and speed vs quality relationships
• Task-Specific Insights: Heatmaps showing which models excel at specific task categories
• Detailed Reporting: JSON exports with complete performance data

Sample Benchmark Output

🏆 MODEL PERFORMANCE SUMMARY
------------------------------
Model                    Avg Response Time (s)  Avg Cost ($)  Avg Quality Score  Tasks Completed
ollama:qwen2.5:7b        0.2500                 0.0000       0.8000             5
ollama:llama3.1:8b       0.3000                 0.0000       0.8200             5
google:gemini-pro        0.6000                 0.1250       0.7800             5
anthropic:claude-3-haiku 0.7000                 0.2500       0.8000             5
openai:gpt-4o-mini       0.8000                 0.1500       0.7500             5
grok:grok-2              0.9000                 0.2000       0.9000             5
openai:gpt-4o            1.0000                 5.0000       0.9500             5

Generated Visualizations

Chart Breakdown:

• Response Time Comparison: Bar chart showing average response times
• Cost Analysis: Cost per request across all models
• Quality Scores: Performance comparison with quality metrics
• Model Usage Count: Which models were selected during benchmark
• Trade-off Scatter Plots: Cost vs Quality and Speed vs Quality analysis
• Model Configuration Summary: Complete specs table for all models
• Task Category Heatmap: Performance matrix across different task types
• Cost Savings Analysis: Savings vs GPT-4o baseline

🏠 Local Model Setup (Ollama)

Install Ollama

# macOS/Linux
curl -fsSL https://ollama.ai/install.sh | sh

# Windows
# Download from https://ollama.ai/download

Pull Models

# Fast models (recommended for testing)
ollama pull llama3.1:8b
ollama pull qwen2.5:7b

# High-quality models (requires more RAM)
ollama pull llama3.1:70b
ollama pull qwen2.5:72b

Verify Installation

ollama list
# Should show your downloaded models

Automated Setup (Recommended)

python setup_ollama.py

This script will:

• Install Ollama if not present
• Download recommended models (llama3.1:8b, qwen2.5:7b)
• Test the connection
• Create .env.example with proper configuration

🏗️ Architecture Overview

graph TB
    subgraph "User Input"
        A[Task Request] --> B[Task Analysis]
    end
    
    subgraph "Intelligent Router"
        B --> C[Traits Detection]
        C --> D[Token Estimation]
        D --> E[Model Scoring]
        E --> F[Multi-Objective Optimization]
        F --> G[Model Selection]
    end
    
    subgraph "Model Providers"
        H[OpenAI<br/>GPT-4o, GPT-4o-mini]
        I[Anthropic<br/>Claude-3-Haiku]
        J[Google<br/>Gemini-Pro]
        K[Ollama<br/>Llama3.1, Qwen2.5]
        L[Grok<br/>Grok-2, Grok-2-Vision]
    end
    
    subgraph "Scoring Engine"
        M[Cost Analysis<br/>Weight: 60%]
        N[Latency Analysis<br/>Weight: 25%]
        O[Quality Analysis<br/>Weight: 15%]
        P[Task-Specific Boosts]
    end
    
    subgraph "Output & Analytics"
        Q[Response Generation]
        R[Quality Metrics]
        S[Decision Logging]
        T[AgentOps Analytics]
    end
    
    G --> H
    G --> I
    G --> J
    G --> K
    G --> L
    
    E --> M
    E --> N
    E --> O
    E --> P
    
    H --> Q
    I --> Q
    J --> Q
    K --> Q
    L --> Q
    
    Q --> R
    Q --> S
    Q --> T
    
    style A fill:#e1f5fe
    style G fill:#c8e6c9
    style Q fill:#fff3e0
    style T fill:#f3e5f5

📖 Detailed Architecture: See Architecture Documentation for complete technical details.

🔍 AgentOps Integration

AgentsIQ seamlessly integrates with AgentOps for advanced analytics and observability:

• 📈 Real-time Metrics: Track model performance, costs, and decision patterns
• 🎯 Decision Tracing: See exactly why each model was chosen
• 📊 Performance Analytics: Detailed insights into routing efficiency
• 💰 Cost Optimization: Monitor savings and identify optimization opportunities
• 🔍 Debugging: Comprehensive logs for troubleshooting and optimization

Setup AgentOps

# Add to your .env file
AGENTOPS_API_KEY=your_agentops_key_here

🎯 Why AgentsIQ ?

1. 💰 Massive Cost Savings: Automatically route to cheaper models without sacrificing quality
2. 🏠 Privacy-First: Full local model support with Ollama integration
3. 📊 Data-Driven: Comprehensive benchmarking with beautiful visualizations
4. ⚡ Performance: Smart routing reduces latency by choosing optimal models
5. 🔧 Developer-Friendly: Simple API, extensive documentation, and zero configuration
6. 🎨 Beautiful UI: Dark dashboard with real-time analytics and control panels
7. 🚀 Future-Proof: Easy to add new models and providers
8. 📈 Enterprise-Ready: AgentOps integration for production monitoring

Control Panel (the “tray” UI in your browser)

• Route strategy: smart | cheapest | fastest | hybrid
• Weights sliders: Cost / Latency / Quality
• Changes take effect immediately in this process.

Key files

• agentsiq/router.py — SMART routing + cost/latency/quality scoring
• agentsiq/router_manager.py — singleton router (shared in‑process)
• agentsiq/dashboard.py — dark HTML UI, /control, /summary, /decisions
• agentsiq/agentops_metrics.py — metrics CSV + percentiles
• agentsiq/decision_store.py — decision rationale JSON
• examples/serve_and_demo.py — single‑process server + demo runner

Sample HTML chart (from real metrics)

Open docs/metrics_demo.html locally in your browser — it renders this data with inline SVG sparklines and a dark table:

{
  "Researcher": {
    "calls": 7,
    "avg_latency": 7.43354994910104,
    "avg_confidence": 0.781428571428572,
    "models": {"openai:gpt-4o-mini": 3, "google:gemini-pro": 4},
    "series": [14.05, 21.05, 10.26, 1.74, 2.06, 1.18, 1.69],
    "p50": 2.0557, "p90": 16.8537, "p99": 20.6342
  },
  "Analyst": {
    "calls": 7,
    "avg_latency": 0.2281,
    "avg_confidence": 0.78,
    "models": {"tool:summarize": 2, "anthropic:claude-3-haiku": 1, "google:gemini-pro": 4},
    "series": [0, 0, 1.2955, 0.0757, 0.0778, 0.0690, 0.0789],
    "p50": 0.0757, "p90": 0.5655, "p99": 1.2225
  }
}

Windows console note

If you see UnicodeEncodeError from AgentOps (emoji in logs), this project applies a SafeConsoleFilter automatically when AGENTOPS_API_KEY is set. You can also set PYTHONUTF8=1 or PYTHONIOENCODING=utf-8 to allow emojis.

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.