agent-safeguards

Safeguards

A comprehensive framework for implementing safety measures in multi-agent systems, focusing on budget coordination, monitoring, and guardrails.

Overview

The Safeguards framework provides tools and infrastructure for ensuring safe and controlled operation of AI agent systems. It addresses key challenges in multi-agent environments:

• Resource management and budget enforcement
• Agent coordination and priority-based allocation
• Safety monitoring and violation detection
• Health assessment and alerting
• Dynamic resource adjustment based on operational needs

This framework is ideal for organizations deploying multiple AI agents that need to:

• Ensure predictable resource usage
• Prioritize critical operations
• Prevent runaway resource consumption
• Implement safe failure modes
• Monitor agent health and behavior

Features

• Budget Coordination System

  • Direct transfer functionality between agents
  • Dynamic pool selection and priority-based allocation
  • Automatic pool scaling and load balancing
  • Emergency allocation handling
  • Priority levels (1-10) for agents and operations

• Advanced Metrics Analysis

  • Resource trend analysis
  • Usage pattern detection
  • Budget efficiency tracking
  • Anomaly detection
  • Health monitoring and recommendations

• Safety Rules System

  • Customizable rule definitions
  • Priority-based execution
  • Rule chain dependencies
  • Context-aware evaluation
  • Violation detection and reporting

• API Contracts

  • Versioned API interfaces
  • Budget management
  • Metrics tracking
  • Agent coordination
  • Configuration management

Quick Start

Installation

pip install agent-safeguards

Basic Setup

from decimal import Decimal
from safeguards.core.budget_coordination import BudgetCoordinator
from safeguards.core.notification_manager import NotificationManager
from safeguards.api import APIFactory, APIVersion
from safeguards.types.agent import Agent

# Create core components
notification_manager = NotificationManager()
budget_coordinator = BudgetCoordinator(notification_manager)
api_factory = APIFactory()

# Create APIs
budget_api = api_factory.create_budget_api(APIVersion.V1, budget_coordinator)
agent_api = api_factory.create_agent_api(APIVersion.V1, budget_coordinator)

# Create a budget pool
pool = budget_api.create_budget_pool(
    name="main_pool",
    initial_budget=Decimal("100.0"),
    priority=5
)

# Create an agent
agent = agent_api.create_agent(
    name="example_agent",
    initial_budget=Decimal("10.0"),
    priority=3
)

# Check agent budget
budget = budget_api.get_budget(agent.id)
print(f"Agent {agent.name} has budget: {budget}")

Creating a Custom Agent

from decimal import Decimal
from typing import Dict, Any
from safeguards.types.agent import Agent

class MyAgent(Agent):
    def __init__(self, name: str, cost_per_action: Decimal = Decimal("0.1")):
        super().__init__(name)
        self.cost_per_action = cost_per_action
        self.action_count = 0

    def run(self, **kwargs: Any) -> Dict[str, Any]:
        """Execute agent logic with cost tracking."""
        self.action_count += 1
        # Your agent implementation here
        return {
            "result": "Task completed",
            "action_count": self.action_count,
            "cost": self.cost_per_action,
        }

# Create and register agent
my_agent = MyAgent("custom_agent")
registered_agent = agent_api.create_agent(
    name=my_agent.name,
    initial_budget=Decimal("20.0"),
    priority=5
)

# Run agent and update budget
for _ in range(3):
    result = my_agent.run(input="Example task")
    current_budget = budget_api.get_budget(registered_agent.id)
    budget_api.update_budget(
        registered_agent.id,
        current_budget - result["cost"]
    )

For more detailed examples, see the Quick Start Guide.

Documentation

• Core Concepts - Essential concepts and terminology
• Installation Guide - Detailed installation instructions
• Quick Start Guide - Get started with the framework
• Component Status - Current status of framework components
• Budget Management - How to manage agent budgets
• Safety Policies - Implementing and enforcing safety policies
• Safeguards - Safety features and guardrails
• Monitoring - Metrics, visualization, and alerts
• Agent Coordination - Multi-agent coordination
• API Reference - Detailed API documentation
• Architecture Overview - System design

For a complete documentation index, see the Documentation README.

Use Cases

The Safeguards framework is designed for a variety of use cases:

• Enterprise AI Systems: Manage resource allocation across multiple AI services
• Autonomous Systems: Ensure safety constraints in autonomous operations
• Research Environments: Control experiment resource usage and monitor behavior
• Agent Orchestration: Coordinate multiple specialized agents working together
• LLM Application Deployment: Manage token budgets and processing resources

Development

Prerequisites

• Python 3.10 or higher
• pip package manager

Setting Up Development Environment

# Clone the repository
git clone https://github.com/cirbuk/agent-safeguards.git
cd agent-safeguards

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install development dependencies
pip install -r requirements-dev.txt

# Install package in development mode
pip install -e .

Running Tests

pytest tests/

Code Style

The project uses:

• Black for code formatting
• isort for import sorting
• mypy for type checking
• flake8 and pylint for linting

Run formatters:

black .
isort .

Run type checking:

mypy src/

Run linters:

flake8 src/
pylint src/

Contributing

Contributions are welcome! Please see our Contributing Guide for details on how to contribute to the project.

Security

This framework implements several security measures:

• Pre-commit hooks for security scanning
• Automated security checks in CI/CD
• Regular dependency updates
• Code analysis tools

If you discover a security vulnerability, please report it to dev@getmason.io.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For support, please open an issue on the GitHub repository or contact the Mason team at dev@getmason.io

Acknowledgments

• Contributors and maintainers
• Security research community
• Open source projects that inspired this framework