aegisloop

AegisLoop - Agent Trust Fabric

A model-agnostic harness that lets any LLM (GPT, Claude, Gemini, local) operate autonomously with grounded, verifiable output and production-grade guardrails. It also governs work from any agent, IDE, GitHub event, or remote worker through canonical change envelopes, evidence graphs, signed proofs, hidden oracles, policy, and transactional effects. Ships with an interactive coding agent (harness chat) - a Claude-Code/Copilot-style REPL with real file & shell tools - and a hosted HTTP API + web dashboard (harness serve) so the same engine deploys as a service. Built on the harness's safety layers.

Honest framing: hallucination can't be eliminated with today's models. This harness minimizes and detects it via layered, independent verification - correctness over cost - and abstains ("I don't know") instead of guessing.

---

What it is

The model is one component. This harness is everything around it:

• Provider gateway - one interface, swappable models (+ retries, fallback, cost).
• Context & memory - prompt assembly, conversation, vector retrieval, compaction.
• Tools & sandbox - MCP-based tools; untrusted code in hardened, network-denied containers.
• Agent loop - typed state machine: plan -> act -> observe -> reflect -> verify -> stop.
• Verification - layered, always-on, independent: deterministic tools, claim/NLI checks, cross-model judges, semantic-entropy uncertainty -> verify-or-abstain.
• Zero-trust security - assume the model is compromised: dual-LLM split, taint tracking, egress lockdown (breaks the "lethal trifecta"), capability policy + human approval.
• Evaluation - datasets + scorers + a CI regression gate.
• Observability - OTel GenAI tracing -> Langfuse; durable execution (crash-resumable), replay.

---

Documentation

• docs/ARCHITECTURE.md - the 8 layers, interfaces, anti-hallucination strategy, tech stack, repo structure. -docs/ROADMAP.md - phased build plan with quality gates.

Prior art: design influenced by an analysis of safe-agentic-workflow. Adopted patterns (progressive disclosure, independence gates, skill packs, manifest-sync) are documented in ARCHITECTURE §15.

Frontier (2024-2026): MCP, dual-LLM/CaMeL zero-trust, semantic-entropy verification, durable execution, context engineering - with sources in ARCHITECTURE §16.

---

Design principles

1. Provider-agnostic core - 2. Everything typed (pydantic) - 3. Ground, then generate -
2. Verify before commit - 5. Fail closed - 6. Everything observable -
3. Deterministic where possible - 8. Budget-bounded - 9. Process as service -
4. Independent verification (the checker is never the author) -
5. Assume compromise (zero-trust).

Correctness over cost - verification is layered and never skipped to save tokens.

---

Status

Working trust fabric (src/harness/) - a real, live-model-integrated, tested system (847 tests plus offline, live-model, Docker, Postgres, hermetic, security, packaging, and deployment gates), hardened through repeated multi-model adversarial review (GPT-5.5 + Gemini-3.1-Pro) of the entire codebase (rounds of findings fixed, then re-challenged to convergence - provider fault-handling, exactly-once keys, verifier quorum, zero-trust egress, workspace path-confinement, bounded process control, circuit-breaker concurrency, attributed-verify soundness, injection ReDoS, plus the trust-gate rounds below):

The trust moat (what makes it differentiated - answers with verifiable evidence, resists manipulation, and provably abstains): claim-level attributed verification, injection ASR driven to 0%, and selective prediction with risk control.

Honest remaining: repository rules must mark attest-gate, offline-proof, and infrastructure-proof as required. The current private-repository plan returns GitHub API 403 ("Upgrade to GitHub Pro or make this repository public"), so remote prevention cannot be enabled without changing repository visibility/account plan; signed CI and local hooks still detect violations. Deeper polyglot coverage remains for Rust/Java native lines; mutation and protected oracle commands are now language-neutral, while codeintel remains regex-best-effort outside Python. The shared Postgres control plane is proven across independent server/store connections and concurrent claims; a geographically distributed, long-duration production soak is still an operations task. Fresh-snapshot test-file isolation now removes filesystem/process/env leakage; fully virtualized wall-clock time remains a limit. Also environment-gated: microVM/gVisor isolation beyond the hardened Docker boundary, a dedicated NLI model (torch), LATS tree-search, and large external benchmarks (AgentDojo/SWE-bench). Shipped since earlier drafts: DLP secret-scrubbing, durable plan resume, the async approval gate, per-branch provenance, and signed content-bound verdicts.

Quickstart

python -m venv .venv && .venv\Scripts\pip install -e ".[dev]"
# WSL/Linux: python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
# point at an authenticated OpenAI-compatible proxy when required:
# export HARNESS_BASE_URL=http://127.0.0.1:5001/v1
# export HARNESS_PROVIDER_API_KEY=<local proxy key>
python -m harness chat                                            # interactive coding agent (Claude-Code-style)
python -m harness models                                          # live catalog + route per model
python -m harness embed "semantic search text"                    # provider embedding
python -m harness plan "Design and verify a release"              # verified adaptive DAG
python -m harness refine "Write a precise API contract"           # independent critic/refinement
python -m harness ask "When?" --sources docs/                     # citation-required grounded answer
python -m harness run --code "Compute the 20th Fibonacci number"  # one-shot agent (sandboxed code)
python -m harness verify-full "Capital of Australia?"             # ensemble + NLI + CoVe + best-of-N
python -m harness verify-claims "Tell me about the Eiffel Tower"  # claim-level attributed verification
python -m harness multi "Capitals of France, Japan, Egypt?"       # multi-agent orchestrator
python -m harness redteam --fuzz 2                                # 96-case live adversarial campaign
python -m harness factory "6*7" "capital of Italy" "100-1"        # headless durable batch
python -m harness eval                                            # trap-set scorecard (7/7)
python -m harness tools                                           # the tool catalog (what the agent can touch)
python -m harness index --root src                                # codebase intelligence (symbols/deps/test-impact)
python -m harness vertical research "capital of France?"          # ready-made profile (coding/research/sre)
python -m harness swe "fix the off-by-one in paginate()" --test-cmd "pytest -q"  # autonomous SWE worker
python -m harness memory ingest-git --repo .                     # personal engineering memory (commits/decisions)
python -m harness bench                                          # offline SWE resolved-rate scorecard (3/3)
python -m harness flywheel --cycles 2                             # self-improvement loop (learn from verified runs)
python -m harness serve                                          # HTTP API + web dashboard (http://127.0.0.1:8080)
python -m harness init --repo .                                  # create aegis.yml trust policy
python -m harness doctor --repo .                                # diagnose policy, CI, keys, Docker and gateway
python -m harness trust evaluate --envelope change.json          # signed canonical decision + evidence graph
python -m harness oracle install-builtins                        # out-of-repo hidden domain packs
python -m harness fleet list --store aegis-workers.db            # workers, capabilities and reputation
python -m harness metrics report --store aegis-quality.db        # safe automation + uncertainty bounds
python -m harness compliance --tenant default --out evidence.json # audit/proof/policy compliance bundle
python -m harness gate-external --repo . --files change.json --nonce req-1  # diff-coverage is on by default
python -m harness improve --repo . --propose-invariant test_authz.py \
  --invariant-file proposed_authz.py                           # quarantined until `harness approve`
python -m pytest -q                                               # 847 tests in the complete environment
python scripts/proof_gate.py --infra --live --hermetic            # offline + fleet + live + fresh-snapshot gate

In the chat REPL: /help, /model <name>, /tools, /auto, /save, /resume, /cwd, /clear, /exit. Mutating/exec actions ask [y/N/a] (use --auto-approve or /auto on to skip).

SDK-only primitives

Some exported APIs are intentionally composable SDK building blocks rather than separate commands: LearningAgent (custom acceptance/experience loops), selective_metrics and risk curves (offline analysis), metamorphic relation helpers (check_idempotent, check_round_trip, check_commutative), and load_mcp_registry (embedding an already-open MCP client). Product equivalents are wired where a safe default exists (refine, ask, plan, flywheel, run --mcp); the SDK forms remain for custom applications instead of duplicating near-identical CLI modes.

Deploy: install (PyPI), Docker, API & dashboard

Published on PyPI - install anywhere (adds the aegisloop / harness commands to your PATH):

pip install "aegisloop[api]"          # extras: api, postgres, embeddings, otel, all
aegisloop serve --host 0.0.0.0 --port 8080     # or: aegisloop-serve / harness serve
# shared Postgres run + trust control plane (approvals and oracle secrets remain isolated):
aegisloop serve --store postgresql://db.internal/aegis --approvals /data/approvals.db \
  --policy /etc/aegisloop/policy.json --multitenant \
  --oracle-root /var/lib/aegisloop/oracles
# OIDC mode: configure providers/role bindings first, then restart with --oidc-auth.
# External audit anchors require HARNESS_AUDIT_ANCHOR_KEYS='{"notary":"<pinned-ed25519-public-key>"}'.
# or run the container:
docker build -t aegisloop . && docker run -p 8080:8080 aegisloop

serve exposes the whole harness over HTTP plus a single-page web console at /:

Multi-user auth is off by default; set HARNESS_API_KEY to require Authorization: Bearer <key> on every /api/* call (the dashboard has a key field), or run serve --multitenant for real per-tenant accounts - hashed API keys with admin/user roles and strict per-tenant run isolation (a bootstrap admin key is minted on first start).

Honest limitations (what this is not)

Reviewed by GPT-5.5 and Gemini; we agree and state it plainly - this is powerful infrastructure, not magic autonomy:

• "0% injection ASR" is suite-specific, not universal security. It holds across the 96-case live base+held-out-variant campaign (harness redteam --fuzz 2 reports per-category ASR), but no system is provably unbreakable against novel attacks. Treat it as strong evidence, not a guarantee.
• The self-improvement flywheel is targeted gap-filling, not rising "model IQ." It recalls verified past solutions; it does not make the base model smarter, and left unguarded it can overfit or cement a wrong "verified" answer. We ship a drift guard (GuardedImprovementLoop) that quarantines any learning which regresses an independent held-out set - but it only catches what the holdout covers.
• Exactly-once still depends on careful external side-effect handling. Our guarantee comes from the idempotent effect-commit + fail-closed on unconfirmed non-idempotent tools; a tool that lies about completion can still double-effect. Give external tools real idempotency keys.
• A multi-repository release is a saga, not a global database transaction. A failed downstream repository is compensated in reverse order; a failed compensation is surfaced for operator recovery.
• Hidden oracles prove the installed probes, not the entire specification. Container separation keeps expected values away from the subject and signs the transcript, but black-box coverage is only as strong as the protected pack. Novel behaviors still need new requirements/probes.
• OIDC, audit anchoring, and escaped-defect metrics have external trust roots. The IdP/JWKS, pinned notary key, and incident/revert ingestion service must themselves be operated securely. Wilson bounds expose small samples; they do not turn a small sample into broad certainty.
• Sandboxed JS/TS V8 coverage is advisory for adversarial changes. Node writes coverage in-process, so hostile tests can forge the JSON artifact. The external trust gate therefore forces mandatory review for JS/TS changes even when V8 reports full coverage; the signal can guide review but cannot authorize. The same trust boundary applies to other in-process proof artifacts: every sandbox-required external change is review-only. A clean automatic accept is reserved for explicitly trusted, non-sandbox callers.
• It won't invent novel architectures or do large multi-repo design. The SWE worker resolves scoped tickets with tests; SWE-bench-Lite capability does not imply big-system design.
• Autonomous SWE/SRE stop at an approval gate by design. The workers propose and prove (diff + passing tests / diagnosis + evidence); a human or policy owns the merge/apply. Run the SWE worker's tests inside a sandbox/CI (running a repo's tests is arbitrary code execution). The SRE worker's read-only tools give a reduced, not zero, blast radius - a heavy query can still load a DB and a GET can exfiltrate; pair with egress allowlists and query caps in production.

Hardened after the GPT-5.5 / Gemini review (we asked them to red-team these very designs): per-tenant API keys now use a salted HMAC (not bare SHA-256, so a leaked DB can't be rainbow-tabled); the audit log exposes head() to anchor externally (so a full-table rewrite is detectable, not just in-place edits); and the SRE "zero blast radius" claim is corrected to "reduced." Verification isn't skipped to look good - the reviewers' valid critiques were implemented.

Target stack

Python 3.11+ - pydantic v2 - httpx/asyncio - MCP - instructor/Outlines - chromadb/qdrant - DBOS - Langfuse/OTel - pytest. See ARCHITECTURE §9 for the full table.