agentharnessengine

---

📑 Table of Contents

• Overview & Motivation · Determinism vs. Industry · How It's Useful
• Deployment & Quick Start · Architecture · System Behavior
• Core Components · Governance Checklist · Failure Semantics
• Integrations · Performance · Security & Threat Model
• Testing & Validation

---

❓ What is Agent-Harness?

Agent-Harness is a deterministic governance runtime that acts as a non-bypassable Runtime Execution Firewall between an agent's reasoning loop and real-world execution surfaces. It translates environmental feedback (risk, effort, stagnation) into strict mathematical execution budgets. When an agent exceeds its bounds, the Harness forcibly terminates execution.

---

🛑 Motivation / Problem Statement

Autonomous agents rely on "soft" boundaries (prompt engineering, RLHF). Under sustained failure or adversarial conditions, agents ignore these and enter unbounded loops. Agent-Harness replaces soft boundaries with a runtime execution firewall that dynamically tracks Effort, Exploration, and Risk. Once budgets deplete, permission to execute always collapses.

---

💎 Why Agent-Harness?

Most AI safety solutions are probabilistic (LLM validation) or stateless (regex). Agent-Harness introduces Stateful Determinism—enforcing physical runtime bounds that an agent cannot reason its way out of.

⚖️ With vs. Without Agent-Harness

🏢 How Agent-Harness Compares

Key difference: Other systems ask "is this output safe?" — Agent-Harness asks "should this agent still be allowed to act?"

---

🔧 How Agent-Harness Is Useful

1. Prevent Runaway Loops — Agents frequently retry failing tools indefinitely. Agent-Harness detects stagnation (zero reward across steps) and halts execution deterministically via FailureType.STAGNATION.

2. Bound LLM API Costs — Effort budgets drain faster when the agent fails to produce real environment progress, preventing token burn from "reasoning theater" (verbose monologue with zero action).

3. Secure Tool Execution — Every tool call passes through GuardrailStack before execution, blocking dangerous code patterns (os.system(), exec(), PII leakage). The dangerous call never reaches the execution surface.

4. Multi-Agent Stability — SharedBudgetPool and CascadeDetector enforce global budget limits across swarms, preventing cascading agent spawn failures.

5. Compliance Audit Trails — SHA256 hash-chained JSONL logs provide tamper-evident, non-repudiable decision history for regulated environments (finance, healthcare, enterprise).

Agent-Harness uses deterministic budget math to guarantee bounded execution without LLM inference.

---

🚀 Deployment & Installation

Local Engine Setup

Install the lightweight engine via pip to integrate governance into your Python logic:

# Install via pip
pip install agentharnessengine

# Or install from source
git clone https://github.com/Sarthaksahu777/Agent-Harness
cd Agent-Harness
pip install -e .

Docker Proxy (Production Ready)

Deploy a non-root, multi-stage optimized governance proxy in seconds. This is the recommended method for production environments to ensure non-bypassable enforcement:

# Start the Governance Proxy + Metrics Stack
docker-compose -f deployment/docker-compose.yml up -d

# Check health
curl http://localhost:8080/health

The proxy exposes port 8080 (Production) and 8081 (Dev/Hot-reload).

Quick Start (Engine Only)

A minimal, framework-free implementation of the engine checking a basic step progression:

from governance.kernel import GovernanceKernel
from governance.profiles import BALANCED

# 1. Initialize the Kernel with a standard behavioral profile
kernel = GovernanceKernel(profile=BALANCED)

# 2. Simulate environmental signals on an agent step
result = kernel.step(reward=0.6, novelty=0.1, urgency=0.0)

if result.halted:
    print(f"TERMINATED: {result.failure}")
else:
    print(f"ALLOWED. Remaining Effort: {result.budget.effort:.2f}")

---

🏗️ Architecture

Agent-Harness intercepts execution across three distinct, modular layers. It strictly separates the "Mind" (Agent Reasoning) from the "Body" (Tool Execution).

LEVEL 1: High-Level System Model

• Agent (Mind): The LLM controller generating tool inputs and plans. It never touches APIs directly.
• Agent-Harness (Governance Runtime): The non-bypassable firewall. It translates abstract environmental signals into concrete behavioral budgets and blocks non-compliant behavior.
• Execution Surface (Tools/APIs): The actual code functions, network requests, or database queries the agent wishes to execute.

LEVEL 2: Governance Pipeline

Every tool call goes through a deterministic processing flow before execution:

1. Agent outputs a tool request.
2. Proxy Enforcer intercepts the request and normalizes it.
3. Guardrail Stack scans the payload for malicious logic or PII.
4. Signal Evaluator continuously processes external feedback (reward, difficulty).
5. Governance Kernel translates signals into deterministic behavioral budgets.
6. Budget Decision fuses the state and issues a hard ALLOW or HALT.
7. Execute / Halt: The tool is executed, or a 403-Forbidden block terminates the session.

LEVEL 3: Budget Decision Loop

The internal kernel state update algorithm ensures that no action is taken without a fresh budget calculation.

---

Example System Behavior

Here is how the Agent-Harness reacts to standard scenarios:

1. Successful Step (Allowed)

[KERNEL] Action: query_database (args: search="Q3 revenue")
[KERNEL] Signals: Reward=0.8, Novelty=0.1, Urgency=0.0
[KERNEL] Status: ALLOWED. 
[KERNEL] Remaining Effort: 0.98. Executing tool...

2. Blocked Tool Call (Guardrail Triggered)

[KERNEL] Action: execute_python (args: code="os.system('cat /etc/passwd')")
[GUARDRAIL] Triggered: code_execution 
[GUARDRAIL] Reason: Potentially dangerous code pattern detected (matched: \bos\.system\s*\()
[KERNEL] Status: HALTED. 
[KERNEL] Result: FailureType.SAFETY (Guardrail violation). Block 403.

3. Halted Agent (Budget Exhausted / Stagnation)

[KERNEL] Action: search_web (args: query="retry 104")
[KERNEL] Signals: Reward=0.0, Difficulty=1.0, Urgency=0.4
[KERNEL] ⚠️ Effort depleted to 0.15.
[KERNEL] Status: HALTED. 
[KERNEL] Result: FailureType.EXHAUSTION (effort_exhausted). Session terminated.

---

📦 Core Components

Based on the src/governance/ module structure, the repository is composed of several critical files:

• kernel.py (GovernanceKernel): The central state machine. Deterministic orchestrator that receives signals, evaluates budgets, and enforces terminal halt conditions based on progress accumulation.
• guardrails.py (GuardrailStack): Pluggable security detectors that intercept and block dangerous payloads.
• audit.py (HashChainedAuditLogger): Records a cryptographically immutable JSONL log of every system decision.
• evaluation.py (SignalEvaluator): Normalizes external semantic signals before integrating them into the core control state.
• profiles.py & behavior.py: Defines agent temperament patterns (e.g., BALANCED, CONSERVATIVE).

---

✅ The 15-Point AI Governance Checklist

Agent-Harness natively implements the 15-Point AI Governance Checklist via deterministic runtime execution mechanisms:

📉 Budget Dynamics

Behavioral budgets (effort, persistence) are strictly bounded and monotonically depleting. Under sustained failure, the budget inevitably crosses the exhaustion threshold, forcing a terminal halt.

---

🛑 Failure Semantics

🛑 Failure Mode Progression

State transitions: Healthy → Warning (low ROI) → Critical (near boundary) → Terminal Halt (EXHAUSTION, STAGNATION, OVERRISK). A 403 permanently ends the session.

---

🔌 Integrations

The Harness detects native abstractions across common LLM agent frameworks and enforces boundaries transparently. Working wrapper examples are located in integrations/:

• LangChain: Intercepts AgentExecutor loops (langchain_ollama.py, langchain_openai.py).
• CrewAI: Limits task iterations and monitors role boundaries (crewai_ollama.py, crewai_openai.py).
• AutoGen: Hooks into UserProxyAgent conversations (autogen_ollama.py, autogen_openai.py).
• OpenAI SDK: Provides direct function-calling guardrails (openai_sdk.py, openai_sdk_ollama.py).

---

📂 Project Structure

Agent-Harness/
├── src/
│   └── governance/         # Core kernel, budgets, evaluators, mechanisms
├── problems/               # S-class problem sets and benchmarks
├── integrations/           # Connectors for LangChain, AutoGen, CrewAI, OpenAI
├── examples/               # Demonstrations of layered governance and edge cases
├── tests/                  # PyTest suites verifying mathematical limits
├── docs/                   # Deep dives and visual assets
├── deployment/             # Docker, Compose, and Dashboards
├── scripts/                # Utility and maintenance scripts
└── config/                 # YAML configuration bindings

---

⚡ Performance & Efficiency

Agent-Harness is designed for high-frequency runtime interception with near-zero overhead. Values below are from a real benchmark of 10,000 kernel.step() calls on commodity hardware.

---

🛡️ Anti-Gaming & Robustness

Native anti-gaming logic in extractor.py:

• Yapping Detection: Decays trust when agent produces high internal activity with zero environment change.
• Fake Success Anchoring: Rejects agent-claimed success if env_state_delta hasn't moved.
• Novelty Debt: Prevents budget resets via "new but useless" actions.
• State Cycling Detection: Environment hashing penalizes oscillating behaviors.

---

📊 Observability and Logging

# Live terminal dashboard
python -m governance.visualizer

# Prometheus metrics export
curl http://localhost:8080/metrics

# Verify audit chain integrity
python scripts/replay_audit.py verify audit_chain.jsonl

---

🔐 Security Model

• Fail-Closed: Exceeding any budget translates into FailureType flags and physical session termination. No auto-recovery—humans must explicitly reset().
• Pattern Isolation: GuardrailStack catches malicious patterns via regex without LLM inference.
• Read-Only Access: External systems cannot mutate ControlState except through deterministic evaluator signals.

---

🛡️ Threat Model

Agent-Harness enforces bounded execution. It is designed with a specific threat landscape in mind.

What it PROTECTS against:

• Runaway Loops: Agents getting stuck hallucinating the same failing tool calls indefinitely.
• Prompt Isolation Breakdown: Adversarial inputs (Prompt Injections) overriding the system prompt and hijacking the agent's goals.
• Unauthorized OS Probing: LLMs attempting to execute arbitrary code (e.g., os.system(), exec()) when they shouldn't.
• Unbounded Costs: Infinite loops draining LLM API token budgets.
• Data Exfiltration (Basic): Pattern-based PII leakage in tool outputs or inputs.

What it DOES NOT protect against:

• Semantic Hallucinations: If the underlying evaluator feeds "fake" rewards to the Harness, the Harness cannot know the progress is hallucinated.
• Sophisticated Obfuscation: The current guardrails use regex/heuristics and may miss highly encoded or obfuscated attacks.
• Host System Compromise: Agent-Harness bounds the agent's logic, but sandbox isolation (like Docker) is still required to secure the host machine.

---

🧪 Testing & Validation Rigor

Agent-Harness is validated against extreme operational conditions to ensure zero-bypass governance:

• Event Horizon Benchmarks: 15 internal problem sets mapping complex semantic failures (from the Black-Hole problem set) to deterministic kernel signal patterns (RL Generalization, Distributed Consensus, etc.).
• Monte Carlo Stress Tests: A high-intensity suite (monte_carlo_stress.py) that executes 12,000 randomized trajectories per run to verify that governance invariants hold under chaotic conditions.
• Adversarial Pattern Suite: Unit tests verifying the GuardrailStack against known prompt injection, PII leakage, and dangerous code patterns (regex-based detection).
• Multi-Agent Swarm Simulations: Validated coordination of swarm-level constraints via SharedBudgetPool and CascadeDetector tests.
• Policy Enforcement Scaling: Continuous stability validation across 1,000+ sequential policy evaluations to ensure zero memory leaks or state drift.

---

⚠️ Limitations

• Heuristic Reliance: The built-in PromptInjectionDetector, PIIDetector, and CodeExecutionGuard use hardcoded regex strategies. They may raise false positives on highly contextual payloads or miss deeply obfuscated attacks.
• Signal Definition: The engine trusts the semantic signals (reward, novelty) provided by the broader orchestrator. If the orchestrator feeds "fake" rewards, the Engine cannot know it is hallucinating.
• Coarse Reset Handling: Because halting is permanent for safety, workflows hitting false-positive stagnation must manage their own session checkpoints (kernel.reset()) conservatively.

---

🗺️ Future Projections & Research

The Agent-Harness roadmap focuses on broadening the scope of autonomous safety. Key research directions include:

• OS-Level Isolation: Exploring eBPF and Kernel-level enforcement to move security boundaries outside the application runtime.
• Mechanical Signal Extraction: Researching reasoning-free telemetry to eliminate evaluation bias.
• Policy & Swarms: Enhancing multi-agent coordination and decentralized budget pooling.
• LLM-assisted Semantic Guardrails: Moving beyond regex into fast, quantized models evaluating signal context natively.
• Dynamic Threshold Selection: Automatically pivoting between CONSERVATIVE and AGGRESSIVE profiles based on historical session hashes.

For a full list of documentation and research papers, see the Master Index.

---