By Microsoft Research just open-sourced Orchard — and the headline number is hard to ignore: 73% on SWE-bench Verified using a model with roughly 3 billion active parameters. For comparison, the frontier systems that score similarly run on architectures 10 times that size. The full package is live today: environment service, three domain-specific training recipes, 107,000 labeled software-engineering trajectories, and a companion paper. If you are building or training AI agents, this is worth your attention.
What Orchard Actually Is (And What It Is Not)
Orchard is not a coding assistant you add to your IDE. It is training infrastructure — the kind of scaffolding that every team building AI agents currently rebuilds from scratch. The core is Orchard Env, a Kubernetes-native service that abstracts away sandbox lifecycle management, command execution, file I/O, and network policy behind a clean REST API. Researchers plug in their own agent harness, trainer, and inference backend; Orchard handles the rest.
Architecturally: a Python SDK issues REST calls to a FastAPI orchestrator, which manages pod creation in a Kubernetes cluster. Execution requests bypass the API server and go directly to each sandbox pod — a deliberate choice to keep latency low at scale. It is lightweight, harness-agnostic, and designed to run the same infrastructure whether you are doing supervised fine-tuning data collection, reinforcement learning rollouts, or final evaluation.
The 73% Number — How They Got There
The Orchard-SWE recipe is the headliner. It improves from a 61.4% baseline on SWE-bench Verified through a sequence of techniques:
- Balanced Adaptive Rollout (BAR): Pushes to 69.1%
- Dense reward signals (on-policy distillation + rubric-based process reward): 69.7%
- Value-model reranking: 73.0%
The genuinely interesting technical move is what they do with failed trajectories. Standard supervised fine-tuning throws out runs where the agent did not solve the problem. Orchard uses retrospective value estimation to extract productive segments from failed attempts — the sections where the agent was making progress, even if it ultimately missed. The dataset includes 32,536 unresolved runs alongside 74,649 resolved ones, and those failures are treated as signal rather than noise. That is not a trivial insight.
Three Recipes, Three Domains
The release ships three ready-made training recipes built on the same infrastructure:
- Orchard-SWE — software engineering. 73% SWE-bench Verified with ~3B active parameters. Training data spans 2,788 GitHub repositories, generated with MiniMax-M2.5 and Qwen3.5-397B using two harnesses (OpenHands and mini-swe-agent) for behavioral diversity.
- Orchard-GUI — browser navigation. A 4B vision-language model trained on 400 distilled demonstrations and 2,200 open-ended tasks hits 74.1% on WebVoyager, 67% on Online-Mind2Web, and 64% on DeepShop.
- Orchard-Claw — personal assistant. Trained on 200 synthetic tasks (email, calendar, search), it reaches 59.6% on Claw-Eval solo and 73.9% paired with ZeroClaw.
GUI and Claw are thinner — 400 and 200 examples versus 107,000 for SWE. Treat them as proofs of concept rather than production-ready recipes.
What Is Actually Released
This is a full open-source release, not a teaser. The GitHub repo has the environment service and training code. The Hugging Face dataset has all 107,000 SWE trajectories and 3,070 GUI rollouts — including the failed runs. The companion paper covers the full methodology. There is also a companion project, OpenForge RL, that extends Orchard to train agents inside their real deployment harnesses — ZeroClaw, OpenClaw, Codex — instead of simplified reimplementations. This removes the train-deploy mismatch that plagues most research-to-production pipelines.
Who Should Care Right Now
If you are a researcher or work at an organization actively training AI agents, this is infrastructure you would otherwise build yourself. The SWE dataset alone — particularly the 32,000 failed trajectories with credit-assignment labels — is worth pulling regardless of whether you use Orchard Env.
If you are a developer who uses AI coding tools rather than builds them, the relevance is more indirect: Orchard is part of the trend making smaller, cheaper models competitive with frontier systems. The Microsoft Research release post goes deeper on the architecture for those who want the full picture.
The framework is research-grade — Kubernetes overhead and all. Most teams will not deploy this tomorrow. But open-sourcing the infrastructure, the data, and the training recipes means the research community no longer has to rebuild from zero. That compression of effort matters, and the SWE-bench trajectory of 3B parameters approaching 30B-class agents is the trend to watch.
