agentcore-rl-toolkit

🚀 AgentCore RL Toolkit (ART)

Scalable, Composable Reinforcement Learning for Real-World Agents.

Website · Documentation · Getting Started · Blogs · Examples

AgentCore RL Toolkit (ART) helps developers improve custom agents with reinforcement learning on AWS Bedrock AgentCore Runtime. It brings together scalable, sandboxed rollouts, token-level trajectory capture, training integrations, and examples spanning coding, office automation, and customer service.

Bring your agent framework, tools, and environments. Define your reward function and choose either a self-hosted or a managed training backend while keeping most of your production code intact.

🎯 Agents We've Trained

Our experiments include Strands and OpenHands agents working with real repositories, office applications, and stateful customer-service environments.

🧩 What ART Provides

• Train your own agent. Reuse your framework, tools, and application logic. Examples include Strands, OpenHands, and mini-swe-agent; agents use standard OpenAI- or Anthropic-compatible model clients.
• Scale complex rollouts. Run long-running agents and their environments on Bedrock AgentCore Runtime, with isolated sessions and on-demand scaling. Builds, tests, and tool execution run separately from the GPU training infrastructure.
• Capture the data RL needs. The rollout gateway records the exact sampled token IDs, logprobs, and loss masks across model turns, including branching conversations, without custom model wrappers in your agent.
• Choose your training backend. Train with verl, slime, Tinker, or SkyRL. Agent execution stays separate from model training.
• Evaluate and verify. Collect rollout results with concurrency control, timeouts, and session cleanup. Use the Sandbox SDK to execute commands and verifiers in task-specific container environments on AgentCore.

How It Works

%%{init: {"flowchart": {"rankSpacing": 20}}}%%
flowchart LR
    subgraph acr["AgentCore Runtime"]
        direction TB
        agent["Agent session<br/>Your agent + tools"]
        sandbox["Sandbox session<br/>(optional)"]
        agent <-.->|Tool I/O| sandbox
    end

    gateway["ART rollout gateway<br/>Token capture"]
    backend["Training backend<br/>Sampling + updates"]
    acr <-->|Model calls| gateway
    gateway <-->|Tokens +<br/>logprobs| backend
    acr -->|Task rewards| backend

Tools can run within the agent session or execute commands in a separate sandbox session for task isolation.

ART's training integrations combine captured trajectories with task rewards so the backend can update the model. Your agent keeps its application-level model interface throughout the loop. See the architecture overview for the runtime, gateway, and result-delivery details.

🔬 Research & Collaborations

We share the training methods and systems work behind these agents, including joint experiments with the rLLM team and collaboration with BMW on CAR-bench.

• Training a Frontier Java Code Migration Agent with AWS AgentCore Runtime — with rLLM: long-horizon RL on MigrationBench, from agent deployment to training and evaluation.
• Continual Learning via Real-Time RL for Agents — with rLLM: learning from one rollout per task in a simulated real-time setting, improving MigrationBench success from 43% to 59.2%.
• Growing verifiable training tasks from the environment alone — synthesize training tasks from tools, environment state, and policy documents, with experiments across τ²-Bench and CAR-bench.
• Stable Updates, Fewer Sequences — stable optimizer-step batching and token-history preservation; linear mode reduced training sequences by 62.76% and actor-update time by 18.24%.
• Async RL Performance Optimization — asynchronous training on SWE-Gym achieved approximately 22% higher reward improvement per GPU-hour.

⚡ Getting Started

The core SDK supports Python 3.11+. Training environments such as Megatron and the SkyRL service require Python 3.12; follow their setup guides.

Install the core SDK:

pip install 'agentcore-rl-toolkit[core]'

Or use uv add 'agentcore-rl-toolkit[core]'. The core SDK includes agent apps and rollout clients. See the installation guide for component-specific dependencies and the deployment CLI.

1. Start with an agent. Try the math agent for a small end-to-end example, or choose a task from Example Agents.
2. Choose a training backend. Follow its guide for the required agent entrypoint, dependencies, AWS deployment, and networking setup.
3. Define success and train. Supply task inputs and a reward function, evaluate your baseline, then launch the backend's training recipe.

Already have an agent? The AgentCoreRuntimeApp example shows how to keep @app.entrypoint while running rollouts and returning task rewards. Follow your training backend's guide for its supported agent setup and deployment steps.

🔌 Training Backends

ART integrates with the following training backends through its rollout gateway for token-level trajectory capture.

SkyRL and Tinker share the toolkit's Tinker-compatible API integration.

rLLM

rLLM natively integrates with AWS Bedrock AgentCore Runtime. Follow its AgentCore integration guide for installation, configuration, and training.

🛠️ Example Agents

Terminal agents with separate sandboxes. The terminal agent example uses the Sandbox SDK to run mini-swe-agent and its task environment in separate AgentCore sessions. Verifier tests are withheld until the agent finishes, keeping tests and grading feedback out of the model conversation.

Contributing

Contributions to agent examples, training integrations, and the core toolkit are welcome. See CONTRIBUTING.md for development setup and contribution guidelines, and the design document process for significant features or architectural changes.

Security

See security issue notifications.

License

This project is licensed under the Apache-2.0 License.