AWS Details Multi-Turn Reinforcement Learning Tool on SageMaker AI
AWS published a technical guide on Multi-Turn Reinforcement Learning on SageMaker AI, a method for training models on multi-step agentic tasks using outcome-based rewards.
Quick answer
What did AWS announce about Multi-Turn Reinforcement Learning on SageMaker AI?
AWS published a technical guide describing Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI, a serverless pipeline that trains models on multi-step agentic tasks using Amazon Bedrock AgentCore to host the agent and reward function, scoring entire action sequences rather than single prompts.
Key takeaways
- AWS describes Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI as a method for training models on agentic tasks that require a sequence of actions, where only the final outcome of the sequence can be scored.
- According to AWS, SageMaker AI sends prompts to an agent hosted on Amazon Bedrock AgentCore (or on customer infrastructure such as Amazon EKS, EC2, or Fargate via an AWS Lambda forwarder), and the agent reports one reward when an episode ends.
- AWS says the pipeline uses Group Relative Policy Optimization (GRPO) to update a Low-Rank Adaptation (LoRA) adapter on the policy model based on reward differences across multiple rollouts.
- AWS states MTRL currently supports four base models with LoRA adapters: Amazon Nova 2 Lite, GPT-OSS-20B, Gemma 4 31B (instruction-tuned), and Qwen 3.6 27B, available in specific AWS Regions depending on the model.
- In a Wordle-based example run described by AWS, MTRL raised GPT-OSS-20B's solve rate from 61 to 84 percent at default reasoning effort and from 15 to 69 percent at low reasoning effort, at a reported training cost of about $339 in tokens.
What AWS announced
AWS published a technical blog post detailing Multi-Turn Reinforcement Learning (MTRL), a training capability on Amazon SageMaker AI designed for agentic tasks that require a sequence of actions rather than a single response. According to AWS, the method addresses a gap in existing training approaches: supervised fine-tuning requires labeled correct answers, which don't exist for multi-step tasks with many possible successful paths, and single-turn reinforcement learning cannot attribute credit to a decision made several steps before an outcome is known.
AWS says MTRL trains models using only a final reward reported at the end of a full task episode, without specifying which exact sequence of actions should have been taken.
How the system works
AWS describes a two-part architecture. SageMaker AI manages the training job and the policy model being trained. Amazon Bedrock AgentCore hosts the agent, the task environment, and the reward function. For each training rollout, AWS says the MTRL service invokes the AgentCore runtime with a single training example, and a Strands agent inside that runtime calls the policy model through an OpenAI-compatible endpoint provided by the SageMaker Job Runtime.
AWS states that agents can alternatively be hosted on a customer's own infrastructure, including Amazon Elastic Kubernetes Service, Amazon EC2, or AWS Fargate, connected through an AWS Lambda forwarder, though the example in the post uses AgentCore.
Once a task episode ends, AWS says the agent returns a single reward, and the system applies Group Relative Policy Optimization (GRPO) to compare rewards across multiple rollouts of the same task and update a Low-Rank Adaptation (LoRA) adapter on the policy model.
AWS highlights three design points: the agent and reward function live together in one deployed file, the training dataset carries only the task-specific input needed for scoring, and reward structures are weighted toward final outcomes with smaller terms for intermediate strategy.
Worked example and results
AWS illustrates the pipeline using Wordle, a game where a player has six attempts to guess a five-letter word, selected because its rules are simple, feedback accumulates turn to turn, and outcomes can be scored objectively. AWS notes that frontier models already play Wordle well and frames the example as a stand-in for internal, business-specific multi-step workflows that public models haven't been trained on.
In a reported training run, AWS says MTRL raised GPT-OSS-20B's solve rate on held-out words from 61 to 84 percent at the model's default reasoning effort, and from 15 to 69 percent at a lower reasoning effort used during training, for a reported training cost of about $339 in tokens. AWS says the 100-step run consumed 491.6 million prefill tokens, 159.7 million sample tokens, and 643.3 million train tokens across 105,486 rollouts over 12 hours and 40 minutes.
Supported models and pricing
AWS states MTRL currently supports four base models, each trained with a LoRA adapter: Amazon Nova 2 Lite, GPT-OSS-20B, Gemma 4 31B (instruction-tuned), and Qwen 3.6 27B. Availability varies by AWS Region, with Nova 2 Lite and GPT-OSS-20B available in US East (N. Virginia) and US West (Oregon), and Gemma 4 31B and Qwen 3.6 27B available only in US West (Oregon). AWS says it chose GPT-OSS-20B for its example because it has the lowest per-token price among the four and allows reasoning effort to be set per request.
AWS describes MTRL as serverless, meaning customers select a base model while the service provisions training and sampling infrastructure automatically. Pricing is per million tokens across three categories: prefill tokens processed as input, sample tokens generated by the policy model including reasoning tokens, and train tokens processed during the backward pass that updates the LoRA adapter. AWS also states that customer datasets, agents, and trained models remain within the customer's AWS account and are not used to train AWS's base models.
Source: AWS Machine Learning Blog, "Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide," published October 23, 2026.
Frequently asked questions
- What is Multi-Turn Reinforcement Learning (MTRL) on SageMaker AI?
- AWS describes it as a training approach for agentic tasks that involve multiple sequential actions, where the model is trained using a single reward reported at the end of an episode rather than labeled examples of correct steps.
- Which models does MTRL support?
- According to AWS, MTRL supports Amazon Nova 2 Lite, GPT-OSS-20B, Gemma 4 31B (instruction-tuned), and Qwen 3.6 27B, each trained with a LoRA adapter, with availability varying by AWS Region.
- How is MTRL priced?
- AWS says customers pay per million tokens across three categories — prefill (input tokens), sample (generated tokens), and train (tokens in the backward pass updating the LoRA adapter) — with pricing varying by base model.
- What example did AWS use to illustrate MTRL?
- AWS used Wordle, a word-guessing game, as a teaching example, reporting that MTRL improved GPT-OSS-20B's solve rate on held-out words from 61 to 84 percent at default reasoning effort.
Sources
- Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide | Artificial Intelligence — Amazon Web Services (AWS)