agentic-llm-router

llm-router

Ecosystem role and current integration status: component roadmap. The public cross-repository plan is owned by the Agentic Security Harness ecosystem roadmap.

A tiny, dependency-light async LLM router with role tiers and per-call cost logging. One call() interface for the tested OpenAI-compatible request shape and a separate Yandex AI Studio path. Provider compatibility depends on each endpoint's current API contract and must be verified before use.

The public library demonstrates a cheap-to-chief routing pattern with offline tests. It does not publish or verify a production deployment claim. No SDKs or models are hardcoded in the routing logic.

The repository now publishes a source-owned, offline invocation-receipt contract and is therefore contract_only in the ecosystem. Its source tree builds the unique distribution agentic-llm-router==0.2.1, imported as llm_router. Exact publication is established by PyPI release history, not source metadata alone. It is not automatically activated or invoked by Harness.

Supply-chain boundary: the generic PyPI name llm-router belongs to another project. Do not install or declare that coordinate for this repository. The only planned public distribution coordinate is agentic-llm-router.

---

Why

In agentic systems most LLM calls are cheap bulk work (extract, classify, filter) and a few are high-stakes (the final decision). Paying flagship prices for everything is wasteful; juggling provider SDKs is annoying. llm-router gives you:

• Role tiers — cheap / mid / chief / audit, each mapped to a model via env. Route volume to cheap, escalate only candidates to chief.
• Provider flexibility — a custom OPENAI_BASE_URL can target endpoints that implement the tested request/response contract; Yandex AI Studio has a separate path. Provider identity, terms, availability, and exact compatibility are external gates.
• Per-call cost — every call returns token counts and cost in USD + a configurable local currency (set LLM_FX / LLM_CCY). Aggregate the dicts to a budget log.
• Budget helpers — aggregate usage records, check a daily cap, and estimate savings versus sending the same tokens to the chief model.
• Resilience — retries on 429 / 5xx with exponential backoff.
• Canonical receipt contract — strict, digest-bound attempt, usage, pricing, and FX evidence without credentials, endpoints, prompts, output text, response bodies, or exception messages.

---

Features

• Single async call(role, system, user) -> (text | None, usage) interface.
• Four configurable role tiers, models set per provider via env.
• OpenAI-compatible and Yandex AI Studio providers.
• json_mode=True → adds response_format={"type":"json_object"} (OpenAI-compatible).
• Cost estimation from an override-able price table (LLM_PRICE_<MODEL>_IN/OUT_USD_PER_1M).
• Budget helpers for logs you own: summarize_usage, budget_status, and build_savings_report.
• Zero secrets cached at import — all config read live from env.
• Importing llm_router, including from llm_router import call, performs no provider I/O and starts no child process. The aiohttp network dependency is loaded only when the caller explicitly invokes call() with provider configuration present.
• ~150 LOC, one runtime dependency (aiohttp).

---

Install

python -m pip install agentic-llm-router==0.2.1

For editable development use python -m pip install -e .[dev]. Requires Python 3.9+. CI builds and audits the exact installed wheel on Linux and Windows. Historical Harness v1.4.0 and v1.5.0 use Router 0.2.0 for their passive router extra; they do not include the passive-import repair. A new Harness release must update that pin after Router 0.2.1 is published. Installation does not configure credentials, choose a provider, or make a provider call.

Version 0.2.1 packages the passive-import repair from PR #12: ordinary package import does not load aiohttp or start a child process. Source and installed-wheel tests install process/network audit denial before import. Provider calls and production compatibility are not exercised or certified by those tests. Published 0.2.0 bytes are unchanged.

---

Quickstart

import asyncio
from llm_router import call

async def main():
    text, usage = await call("cheap", "You are concise.", "Name 3 primary colors.")
    print(text)
    print(usage)   # {provider, model, role, input_tokens, output_tokens,
                   #  total_tokens, cost_usd, cost_local, currency}

asyncio.run(main())

Set at least a provider + key first (see Configuration). For OpenAI:

export OPENAI_API_KEY=sk-...

---

Providers

The base URL must NOT include /chat/completions — the router appends it. Yandex requires YANDEX_FOLDER_ID (or an explicit YANDEX_<ROLE>_MODEL); otherwise model_for raises a clear configuration error (fail-fast) instead of sending an empty model.

---

Roles

from llm_router import call, model_for

model_for("cheap")   # -> e.g. "gpt-4o-mini" (or your LLM_CHEAP_MODEL)
model_for("chief")   # -> e.g. "gpt-4o"

# pattern: cheap for volume, chief only when it matters
facts, u1 = await call("cheap", EXTRACT_PROMPT, raw_text)
if looks_important(facts):
    verdict, u2 = await call("chief", DECIDE_PROMPT, facts, json_mode=True)

---

Cost logging

text, usage = await call("cheap", sys, user)
# usage["cost_usd"]   -> e.g. 0.0001
# usage["cost_local"] -> cost_usd * LLM_FX
# usage["currency"]   -> LLM_CCY (e.g. "RUB")

Append each usage to a JSONL file and you have a per-call budget log. Prices come from a small built-in table and are illustrative — override per model:

export LLM_PRICE_GPT_4O_MINI_IN_USD_PER_1M=0.15
export LLM_PRICE_GPT_4O_MINI_OUT_USD_PER_1M=0.60

Summarize a batch of usage records:

from llm_router import summarize_usage, budget_status, build_savings_report

usages = [u1, u2]  # dicts returned by call()
print(summarize_usage(usages).as_dict())
print(budget_status(usages, limit_usd=1.00).as_dict())
print(build_savings_report(usages, counterfactual_role="chief").as_dict())

LLM_BUDGET_USD_DAY can be used as a default budget cap for budget_status(...). The router stays stateless; you decide where the JSONL budget log lives.

Canonical invocation receipts

router-invocation-receipt-v1.0 is a separate offline interchange surface for already-observed sanitized values. It uses canonical UTF-8 JSON, domain-separated content identities, contiguous attempt accounting, strict token totals, and integer nano-unit cost arithmetic. Pricing and FX inputs bind caller-supplied source artifact digests; those digests are evidence references, not authenticity proofs.

The receipt builder never calls a provider and the existing call() return value is unchanged. A receipt contains digests of request, response, output, model, and producer identity—not their raw bytes. It always declares invoice_authoritative=false and operational_authority=none.

Deterministic hashes are content-minimizing, not anonymizing: they remain linkable and can be guessed when the source space is small. A receipt is not automatically safe to publish.

from llm_router import InvocationAttemptV1

attempt = InvocationAttemptV1(
    attempt_index=1,
    outcome="network_error",
    http_status=None,
    reason_code="provider.network_error",
    response_payload_sha256=None,
)
# Supply only already-observed sanitized values; see docs/invocation-receipt.md.

---

Configuration (env)

See .env.example.

---

Examples & tests

python examples/basic.py          # one call
python examples/role_tiers.py     # cheap vs chief + cost
python -m pytest -q               # offline unit tests (no network)

What each example shows and what it does not prove: examples/README.md.

---

Docs

• Component roadmap — source-owned ecosystem role, platform evidence, historical projections, and integration gates.
• Project map — modules, what exists today vs not included, reviewer checklist.
• Use cases — who this is for, practical workflows, limitations.
• Operating model — role budgets, usage records, escalation gates, and residual risk.
• Invocation receipt V1 — canonical codec, attempt state machine, fixed-point arithmetic, privacy boundary, and non-claims.
• Harness ecosystem roadmap — the canonical public ordering for cross-repository integration work.

---

Limitations / non-goals

• Chat completions only (no streaming, embeddings, tools/function-calling, vision — kept intentionally small).
• One system + one user message per call (no multi-turn history helper).
• The price table is illustrative; confirm real prices with your provider.
• Not a full framework — it's a focused routing + cost-logging utility you drop into your own agent loop.
• Not the portfolio flagship, policy authority, or security boundary. Larger systems own their own validation, authorization, storage, and safety rules.

---

License

MIT — see LICENSE.