agent-handshake

agent-handshake
Contract tests for multi-agent systems

agent-handshake finds the fields that one agent reads from another agent. It tests these fields in CI. The tests use recorded outputs, so they make no LLM calls and need no API keys.

agent-handshake was called a2a-spec before version 0.7.

The problem

Agent A sends data to agent B. You change the prompt of agent A, and its output changes. For example, agent A now sends urgency instead of priority.

Agent B reads priority with a default value. It does not stop with an error. It gives a wrong result, and no test fails.

What agent-handshake does

1. It reads the code of agent B and finds the fields that agent B uses.
2. It records the output of agent A one time and saves it as JSON files in git.
3. It tests the recorded output against the fields that agent B needs. You run these tests in pytest or in CI.

Terms

Get started

You need Python 3.11 or later and a git repository.

This example uses two agents. The producer triage_agent classifies a message. The consumer resolution_agent uses that result.

# agents/triage.py
async def triage_agent(data: dict) -> dict:
    text = data["message"].lower()
    category = "billing" if "charge" in text else "general"
    return {"category": category, "priority": "high" if "urgent" in text else "low"}

# agents/resolution.py
async def resolution_agent(ticket: dict) -> dict:
    escalate = ticket.get("priority", "low") == "high"
    return {"reply": f"We are checking your {ticket['category']} issue.", "escalate": escalate}

1. Install agent-handshake.

pip install agent-handshake

2. Create the project files.

handshake init

This command creates handshake.yaml and the handshake/ directory, and adds __pycache__/ to .gitignore.

3. Remove the example spec and the example scenario.

rm handshake/specs/example-spec.yaml handshake/scenarios/example-scenarios.yaml

4. Add the producer to handshake/adapters/__init__.py.

from agent_handshake import FunctionAdapter
from agents.triage import triage_agent

ADAPTERS = {
    "triage-agent": FunctionAdapter(fn=triage_agent, agent_id="triage-agent", version="1.0"),
}

5. Add inputs for the producer to handshake/scenarios/support.yaml.

scenarios:
  - name: double_charge
    input:
      message: "I was charged twice. This is urgent."
  - name: question
    input:
      message: "How do I change my address?"

6. Record the output of the producer.

handshake record

The command calls the producer one time for each scenario:

    PASS triage-agent / double_charge recorded
    PASS triage-agent / question recorded
[done] 2 snapshots saved to ./handshake/snapshots

If ADAPTERS has more than one agent, the command calls each agent with each scenario. To record one agent only, use handshake record --agent triage-agent. To give a scenario to some agents only, add agents: [triage-agent] to the scenario.

7. Make the spec from the code of the consumer.

handshake infer --producer triage-agent --consumer resolution-agent \
  --source agents/resolution.py:resolution_agent

The command writes handshake/specs/triage-agent-to-resolution-agent.yaml:

# Generated by 'handshake infer' from agents/resolution.py:resolution_agent and 2 triage-agent snapshot(s). Review before committing.
#   priority: required, read at line 2
#   category: required, read at line 3
spec:
  name: triage-agent-to-resolution-agent
  producer: triage-agent
  consumer: resolution-agent
  structural:
    type: object
    required: [priority, category]
    properties:
      priority: {type: string}
      category: {type: string}

Read the spec before you commit it. If the command shows a WARN line, read Limits.

8. Run the tests.

handshake test

PASS triage-agent-to-resolution-agent / double_charge
PASS triage-agent-to-resolution-agent / question
  2 passed  (total: 2)

9. Commit the files.

git add handshake.yaml handshake agents
git commit -m "test: add agent contracts"

The committed snapshots are the baseline for the next steps.

When you change an agent

1. Change the producer. For example, change its prompt, its model or its code. In this example, the producer now sends urgency instead of priority.

2. Record the output again.

handshake record

3. Compare the new output with the committed snapshots.

handshake diff

  triage-agent / double_charge:
    FAIL priority: Field 'priority' was removed; resolution-agent reads it at agents/resolution.py:2
    INFO urgency: Field 'urgency' was added; no consumer spec reads it

With a spec for the producer, diff shows which consumer reads a removed field and on which line. Changes to fields that no spec names are INFO and do not fail.

4. Run the tests.

handshake test

FAIL triage-agent-to-resolution-agent / double_charge
    [<root>] 'priority' is a required property

5. Correct the problem.

• If the change in the producer is an error, correct the producer. Then do steps 2 to 4 again.
• If the change is necessary, change the consumer. Then run handshake infer with --force and commit the new spec and snapshots.

Do not record the snapshots again only to make a test pass. Do not delete snapshots or remove fields from a spec for this purpose.

Keep the specs up to date

A consumer can start to read a new field when you change its code. Then its spec is not complete. To find this problem, run:

handshake infer --check

FAIL triage-agent-to-resolution-agent (agents/resolution.py:resolution_agent)
    resolution_agent reads 'language' (line 3), which the spec does not cover

The command compares each spec that handshake infer made with the current code of its consumer. It reads only code, so it needs no snapshots and makes no LLM calls. The output also shows the command that makes the spec again.

Add handshake infer --check to CI. The pytest plugin also runs this check as the consumer_code test of each inferred spec.

Run the tests with pytest

agent-handshake has a pytest plugin. It turns on by itself when it finds handshake.yaml in the directory where pytest starts, or in a parent directory up to the pytest rootdir; the nearest file wins, so each project in a monorepo can have its own. It adds the specs directory to the run. pytest then runs one test for each snapshot:

$ pytest -v
handshake/specs/triage-agent-to-resolution-agent.yaml::double_charge PASSED   [ 50%]
handshake/specs/triage-agent-to-resolution-agent.yaml::question PASSED        [100%]

A spec whose producer has no snapshots is a failed test, not a skipped one. The usual pytest options work, for example -k, -x and --junitxml.

• To turn the plugin off for one run, use pytest -p no:handshake. To keep it off, add handshake_enabled = false to the pytest configuration.
• If handshake.yaml is somewhere else, use pytest --handshake-config path/to/handshake.yaml, or set handshake_config in the pytest configuration.
• If you name test paths on the command line, the plugin does not add the specs directory.

To test a live output in your own test, use the handshake_check fixture. This example is an async test, so it needs an async plugin such as pytest-asyncio.

async def test_triage_live(handshake_check):
    output = await triage_agent({"message": "I was charged twice"})
    handshake_check("triage-agent-to-resolution-agent", output)

Run the tests in CI

The tests use the committed snapshots. CI needs no API keys.

On GitHub, use the agent-handshake action. It runs handshake infer --check and handshake test, and it shows each failure as an annotation in the pull request:

# .github/workflows/contracts.yml
name: Agent contracts
on: [push, pull_request]

jobs:
  contracts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: padobrik/agent-handshake@v0.10.0

The action has these inputs: package (for example agent-handshake[typescript]), working-directory, config, check-inferred and args. Its output test-exit-code is the exit code of handshake test (0 when every contract passed), and it is empty when that step did not run.

You can also run the commands yourself:

# .github/workflows/contracts.yml
name: Agent contracts
on: [push, pull_request]

jobs:
  contracts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-python@v6
        with:
          python-version: "3.12"
      - run: pip install agent-handshake
      - run: handshake infer --check
      - run: handshake test --format github

--format github shows each failure as an annotation in the pull request. For JUnit XML, use --format junit > results.xml.

Write a spec by hand

A spec can have two types of rules:

spec:
  name: triage-to-resolution
  version: "1.0"
  producer: triage-agent
  consumer: resolution-agent
  structural:
    type: object
    required: [category]
    properties:
      category: {type: string, enum: [billing, general]}
  policy:
    - rule: no_internal_ticket_ids
      method: regex
      patterns: ['\bINT-\d{6}\b']

See Writing specs for all options.

Check exact values

A scenario can also give exact values for the output of an agent. handshake test and pytest compare these values with the snapshot or with the live output.

scenarios:
  - name: double_charge
    input:
      message: "I was charged twice. This is urgent."
    expectations:
      triage-agent:
        category: billing

FAIL triage-agent-to-resolution-agent / double_charge
  expectations FAIL
    expected 'category' = 'billing', got 'general'

Connect your agents

For an A2A agent, give the base URL of the agent. The adapter reads the Agent Card, finds the JSON-RPC interface and sends the scenario input. It records the agent version from the Agent Card in each snapshot.

from agent_handshake import A2AAdapter

ADAPTERS = {
    "triage-agent": A2AAdapter(url="https://triage.example.com", agent_id="triage-agent"),
}

See Writing adapters for all options.

Use with coding agents

agent-handshake contains an Agent Skill. The skill tells a coding agent how to use agent-handshake. It also tells the agent not to make a failed test pass by recording the snapshots again.

handshake skill --dir path/to/skills

This command writes path/to/skills/agent-handshake/SKILL.md. Export the skill again after you update agent-handshake.

Exit codes

Every handshake command exits with one of three codes:

When a run has both, the code is 2: the run was not complete, so its passes cannot be trusted.

Limits

• handshake infer reads Python consumers. It finds data["key"], data.get("key") and "key" in data, also in nested objects (data["customer"]["tier"], data.get("customer", {}).get("tier")) and through a local name (customer = data["customer"]). It also follows list items (for item in data["searches"], comprehensions, data["searches"][0]; the field is searches[].query), a value taken with or or a conditional (headers = data.get("headers") or {}), and helper functions in the same file that get the data (self.review(state), up to three levels).
• handshake infer also reads TypeScript and JavaScript consumers (.ts, .mts, .cts, .tsx, .js, .jsx, .mjs, .cjs) after pip install "agent-handshake[typescript]". It finds data.key, data["key"], data?.key, "key" in data and destructuring (const { key } = data). The code is parsed, not run, so Node.js is not needed.
• handshake infer picks the parameter that holds the producer's output by its type: a Pydantic model, a TypedDict or a dataclass in Python, an interface, a type or a Zod schema in TypeScript, defined in the same file or imported from the project. With several typed parameters, it takes the one whose type declares most of the producer's output keys. If no parameter fits, it says so and writes no spec: the data may come from shared state. Without types, it takes the first parameter after self that is not a string or a number. Name a method by its own name (manager.py:_search), not with its class (Manager._search).
• On a typed parameter, attribute reads (report.summary, getattr(report, "summary", "")) count as field reads, and a read of a field the type does not declare gets a WARN line. A list of outputs (Sequence[Report], Report[]) is read per item, and the final_output of an OpenAI Agents SDK RunResult is the agent's output.
• handshake infer does not follow attribute reads of an untyped input. It shows a WARN line with the line number, and sets reads_unknown_keys: true.
• If the consumer gives its input to a function in another file, or more than three levels down, handshake infer cannot follow it. The command then shows a WARN line and sets reads_unknown_keys: true in the spec, so handshake diff does not treat other fields as unread. Add the missing fields to the spec by hand.
• handshake infer --check checks only specs that contain inferred_from. Specs from agent-handshake 0.3 and earlier do not have it. Run handshake infer with --force again to add it.
• handshake diff compares with the snapshots in git. Commit the snapshots before you use it.
• A2AAdapter supports A2A 1.0 with JSON-RPC, with or without streaming. It does not support gRPC or HTTP+JSON.

Commands

See CLI reference for all options.

Documentation

• Writing specs
• Writing adapters
• CLI reference
• Python API
• CI integration
• Concepts
• Architecture
• Example project

Contributing

See CONTRIBUTING.md.

License

Apache 2.0. See LICENSE.