Last week Microsoft open-sourced Orchard in full under the MIT license — a framework built specifically for training and evaluating agents. The result that turns heads: Qwen3.5-35B-A3B, a model with roughly 3 billion active parameters, scored 69.7 percent on SWE-bench Verified — 73.0 percent after value-model reranking — closing in on frontier systems with more than ten times its parameter count. Ships alongside: 107,000 SWE training trajectories and the complete training recipes that produced them.
For developers, the practical meaning is bigger than one benchmark. Training your own coding agent, GUI agent, or personal assistant no longer requires building sandbox infrastructure from scratch — the environment layer, the data, and the training scripts all exist, ready to run.
The Core Idea: One Foundation, Three Recipes
Orchard’s design philosophy is easy to state: treat the environment as a stable service layer, not a disposable part of a training stack. Every recipe shares the same foundation, and distillation, on-policy RL, and evaluation reuse the entire chain.
The foundation: Orchard Env, a Kubernetes-native sandbox service
- A clean REST API covers the full sandbox lifecycle — create, exec, files, patch, delete — in a protocol any language can talk to.
- A Python SDK offers sync and async modes with retries, automatic cleanup, and context managers built in.
- Any base image works. The agent is injected through an init container carrying its own Python interpreter — your image does not even need Python installed.
- Harnesses ship preinstalled: codex, claude, pi, opencode, and hermes sit on every sandbox’s PATH. Switching harnesses means switching a command, not rebuilding an image.
- Isolation and cost control are defaults: Calico NetworkPolicy blocks egress out of the box, every sandbox gets CPU, memory, and timeout limits, TTLs clean up leftovers, and API keys gate access.
Official performance numbers make the case: mean command execution latency of 0.28 seconds, against E2B’s 0.747s and Modal’s 2.046s. Spinning up 1,000 sandboxes in parallel succeeded 100 percent of the time in 26 seconds end to end. Running 128 sandboxes for 240 hours costs $3,362 on-demand or $673 on spot instances — roughly a tenfold gap — while managed sandbox services quote $7,078 to $10,305 for the same workload.
Three recipes, three agents
| Recipe | Base model | Training data | Key techniques | Headline result |
|---|---|---|---|---|
| Orchard-SWE | Qwen3.5-35B-A3B | 107K distilled trajectories | Credit-assignment SFT, balanced adaptive rollout, on-policy distillation, rubric-based process rewards, value-model reranking | 73.0% SWE-bench Verified |
| Orchard-GUI | Qwen3-VL-4B-Thinking | 0.4K SFT + 2.2K RL tasks | Distill first, then online RL on real websites | 68.4% average (74.1 / 67.0 / 64.0) |
| Orchard-Claw | Qwen3-30B-A3B-Thinking | 0.2K synthetic tasks | Opus-synthesized tasks, trained across two harnesses | 59.6% pass@3 (73.9% under ZeroClaw) |
The generalization numbers argue louder than the headline scores. Orchard-SWE holds 51.0 on SWE-bench Multilingual, where OpenSWE-32B manages 28.7. Moved to the Kimi-CLI harness — one it never saw during training — it still scores 45.0, while OpenSWE-32B collapses to 3.6. And Orchard-Claw gains +14.3 pass@3 when upgraded to the stronger ZeroClaw harness. Training against harness-agnostic environments is what makes capability transfer.
The hardest problem: credit assignment
The current environment model is linear: a sandbox is created, driven for N turns, destroyed. One trajectory yields one final reward that must be spread across an average of 47.5 turns. The paper handles this indirectly with retrospective value estimation; the roadmap aims at something better — snapshots (pause and resume), branching (fork k continuations from an identical state), and prefix sharing, turning each turn’s value from an inference into a measurement. Those primitives go straight into the environment layer, and they are the core of the next research cycle.
Ten Minutes to a Running Sandbox
# 1. Install (pure Python SDK, in the orchard_env directory)
pip install -e "orchard_env[dev]"
# 2. Point at your orchestrator (none yet? The docs stand one up
# on AKS in about 20 minutes)
export SANDBOX_BASE_URL="http://your-orchestrator-host"
export SANDBOX_API_KEY="your-api-key"
# 3. Create a sandbox, run a command, read the output
from orchard_env import SandboxClient
with SandboxClient() as client: # reads the env vars above
with client.create_sandbox("python:3.11-slim") as sandbox:
result = sandbox.exec("echo 'Hello, Orchard!'")
print(result.stdout) # Hello, Orchard!No cluster? Four scripts stand one up on Azure AKS — provision, build and push images, deploy, smoke-test — in about twenty minutes. Non-Azure clusters and cost estimation live in orchard_env/docs/deployment.md.
Want to train your own SWE agent? The dataset and training stack ship too:
# Dataset: huggingface.co/datasets/microsoft/Orchard
# swe subset: 107,185 multi-turn SWE trajectories
# (19,287 task instances / 2,788 repositories, with verified
# solve labels: 74,649 solved / 32,536 unsolved)
# gui subset: 3,070 screenshot-annotated web navigation
# trajectories (409 WebVoyager tasks)
#
# Trainer: trainer/slime/ (a fork of THUDM's slime)
# RL scripts in trainer/slime/examples/orchard/
# every local change documented in ORCHARD_CHANGES.mdPractical Advice
- To reproduce the results: start from the Qwen3.5-35B-A3B base with the microsoft/Orchard dataset, run the slime trainer’s orchard example, and follow the paper’s order — credit-assignment SFT, then on-policy distillation, then RL.
- To keep costs down: run evaluation and replay on spot instances. 128 sandboxes for 240 hours at $673 undercuts managed sandbox services by an order of magnitude.
- To kill train-deploy mismatch: look at the companion OpenForge RL work (arXiv:2607.21557), which trains inside real harnesses — ZeroClaw, OpenClaw, Codex — instead of simplified reimplementations.
- To validate quickly: skip Kubernetes for now. Read the SDK and REST API docs (docs/sdk.md, docs/api.md) and wire the SDK into your own project first.
Why It Matters
Orchard’s real contribution is not a leaderboard number. It is the claim — now demonstrated — that the environment layer is the right place to invest when agents get serious: a stable, measurable, versionable service that every downstream recipe reuses. Model checkpoints expire; a good environment compounds. With the foundation, three proof-of-credit recipes, a 107K-trajectory dataset, and the training stack all public, the bottleneck for training a credible coding agent just moved from infrastructure back to ideas.
The project lives at github.com/microsoft/Orchard, the paper at arXiv:2605.15040, the data on Hugging Face, and Microsoft’s own write-up on the Microsoft Research blog.
