agent-rewind

rewind

Record, replay, and time-travel-debug LLM agent runs.

Agents fail at step 37 of 40 and all you have is logs. rewind records every LLM and tool call of a run to a portable trace file, replays the run deterministically offline (no API key, $0), lets you fork a run at any step to test an intervention, and turns recorded traces into free CI tests.

• Zero dependencies. The core is pure stdlib — provider SDKs are optional extras.
• Zero code changes. auto_patch() hooks OpenAI, Anthropic, litellm, Mistral, Cohere, Gemini, Groq, Together AI, Fireworks, and MCP directly — works unmodified with LangChain, LangGraph, the OpenAI Agents SDK, pydantic-ai, smolagents, and anything else built on them.
• Crash-safe. Events are flushed per call; a crashed run still leaves a usable trace.
• Divergence as a feature. When a replayed run escapes its trace, the error tells you the exact step, with a diff.

Install

pip install agent-rewind        # import name: rewind

Zero runtime dependencies. Python 3.10+.

Quickstart

import rewind

@rewind.llm
def call_model(**request):
    return client.chat.completions.create(**request)

@rewind.tool
def search(query: str) -> dict:
    return search_api(query)

# 1. Record a live run
with rewind.record("run.rewind"):
    agent.run("find me a flight")

# 2. Replay it -- fully offline, deterministic, free
with rewind.replay("run.rewind"):
    agent.run("find me a flight")

# 3. Time-travel: history up to step 23, live (with your fix) after
with rewind.fork("run.rewind", at=23, save_as="fixed.rewind") as session:
    agent.run("find me a flight")
print(session.new_events)  # what happened after the intervention

Don't want decorators? Patch the SDK itself instead — no code changes at all:

import rewind
rewind.auto_patch()              # patches every installed provider SDK

with rewind.record("run.rewind"):
    agent.run("task")            # unmodified framework agent -- any provider

with rewind.replay("run.rewind"):
    agent.run("task")            # same run, offline; responses come back as
                                  # real SDK types, so resp.choices[0] works

Supported providers

Embeddings (.embeddings.create, litellm.embedding, etc.) are recorded too, everywhere they exist. Also covered for free, no matter which SDK constructs it: anything reached by pointing the openai SDK's base_url at it — Azure OpenAI, DeepSeek, OpenRouter, Ollama's /v1, vLLM. One sharp exception worth knowing: Groq/Together AI/Fireworks/xAI each also ship their own dedicated Python package, so agent code built directly on those needs the matching native patcher above, not patch_openai(). Full detail, per-provider caveats, and tested SDK version ranges: docs/providers.md.

Use cases

🔁 Regression-test your agent in CI, for free

@pytest.mark.rewind_trace("traces/flight.rewind")
def test_flight(rewind_session):
    result = run_agent("find me a flight")
    assert "SFO" in result["final"]

Plain pytest replays the trace offline — no API key needed in CI, ever. pytest --rewind-record runs the marked tests live and (re-)records their golden traces, when your agent's behavior is supposed to change: the cassette workflow in one flag.

🐛 Reproduce a bug without the API

with rewind.record("bug-42.rewind"):
    agent.run(customer_input)     # capture it live, once

with rewind.replay("bug-42.rewind"):
    agent.run(customer_input)     # debug it offline, as many times as you want, $0

⏱️ Time-travel debug a failing multi-step run

with rewind.replay("run.rewind", break_at=5):
    agent.run(task)   # pauses inside breakpoint() at step 5 -- real stack, real state

Ordinary pdb/ipdb work here since replay is deterministic and in-process. Fork from any step to test a fix without re-running the whole thing:

with rewind.fork("run.rewind", at=5, save_as="fixed.rewind") as session:
    agent.run(task)    # steps 0-4 replayed from the trace, step 5+ goes live

🔍 See exactly where two runs diverged

$ rewind diff before.rewind after.rewind

Aligns two runs step-by-step and points at the first divergence — the step where the model answered differently and everything downstream followed. Exit code 1 on divergence makes it a one-line CI regression gate.

💰 Know what a run cost

$ rewind stats run.rewind
tokens & cost (estimated):
  model                 calls      input     output       cost
  gpt-4o-2024-08-06         2       2400        680    $0.0128

Tokens are exact (read from the recorded usage); the dollar figure is a best-effort estimate you can override with --price MODEL=IN/OUT or hide with --no-cost.

🧪 Prove your tests never hit a live API

backend.impl = rewind.NeverCalled()   # raises if anything reaches it live
with rewind.replay("traces/flight.rewind"):
    result = agent.run("find me a flight")

ScriptedLLM (a scriptable fake model) and NeverCalled are exported for deterministic agent tests without an API key at all. For CI, replay(path, require_full_consumption=True) turns leftover recorded events into a hard failure — every recorded event must actually be served for the run to pass.

CLI reference

$ rewind show run.rewind -v          # dump every step
$ rewind show run.rewind -i          # step through it: full-screen TUI on a
                                     # terminal, line stepper otherwise
$ rewind stats run.rewind            # counts by kind, errors, time, tokens + est. cost
$ rewind diff a.rewind b.rewind      # where did two runs diverge? exit 1 if they did
$ rewind compact run.rewind          # gzip a finished trace for archival

rewind show --tui opens a two-pane curses browser: event list on the left, selected step's request/response on the right (j/k move, / filters, enter expands JSON, q quits). It degrades gracefully with no terminal — a pipe, CI, rewind show | less all fall back to a line stepper automatically. Full command and flag reference: docs/api-reference.md.

Replay matching strategies

with rewind.replay("run.rewind", match="fuzzy"):
    ...

Custom matchers, the embedding-similarity backend, and the full mechanics behind each strategy: docs/internals.md.

Big traces & multimodal runs

Trace size is driven by payload bytes, not step count. What actually blows traces up is multimodal content re-sent every turn — solved with one kwarg:

with rewind.record("run.rewind", externalize=64_000):   # bytes threshold
    agent.run("describe these screenshots")

externalize= stores any payload leaf at or above the threshold as a content-addressed file under run.rewind.blobs/, so a vision agent re-sending one 1 MB image for 200 turns writes a 72 KB trace plus one blob instead of ~200 MB — replay identity is unchanged. Lazy loading is automatic (replay memory is O(events), not O(bytes)), and rewind compact run.rewind gzips a finished trace for archival (10–50× on repeated-history runs). Details and measurements: docs/internals.md.

Redacting secrets

def scrub(payload):                    # any callable: payload -> payload
    if isinstance(payload, dict):
        payload.pop("api_key", None)
    return payload

with rewind.record("run.rewind", redact=scrub):
    ...

Redaction runs before anything touches disk, and never affects replay identity — a call's fingerprint is taken over the un-redacted request, so scrubbing a payload can't make a trace fail to replay.

What gets captured

rewind records the nondeterminism boundary, not your agent:

• LLMs — @rewind.llm wraps any provider callable, or skip the decorator entirely with auto_patch() / the provider-specific patches above.
• Tools — @rewind.tool wraps any side-effecting callable: HTTP, DBs, shell. patch_mcp() records Model Context Protocol tool calls directly. Exceptions are recorded and replayed too.
• Memory — in-process state (message history, scratchpads) needs no capture: it's deterministic given the same llm/tool responses. External memory (vector stores, Redis) is a nondeterminism source — wrap reads with @rewind.memory so stats/diff can separate "the model changed its answer" from "retrieval returned different context."

All three decorators, and everything under Use cases above, work on async def functions unchanged, and streaming (stream=True) is recorded lazily — chunks are written as your agent consumes them, so aborting a stream early stops paying for tokens without losing the trace.

Runnable examples

All demos run offline against local stub servers — no API key needed unless you point them at a real endpoint.

Learn more

• docs/providers.md — full provider coverage detail, the base_url trap, compatibility & tested SDK versions
• docs/api-reference.md — every export, one line each
• docs/known-limitations.md — honest edges, none silent
• docs/internals.md — trace format, determinism, concurrency model, matching internals
• CHANGELOG.md — release history

Contributing

Issues and PRs welcome — see CHANGELOG.md for what's already shipped, docs/providers.md for provider work that's designed but not yet built (xAI, AWS Bedrock), and per-run cost budgets / HTML trace export as the two open feature ideas beyond that.

License

MIT