NVIDIA open-sourced Cosmos Framework this week — the training and inference toolkit that sits underneath its physical AI model family. Combined with TAO 7’s new agent skills, the pitch is specific: post-train a world model on your own robot or vehicle data in a single day, using nothing but natural-language prompts to your coding agent. The numbers hold up. Here’s what actually changed for developers.
Cosmos Framework Is Not Cosmos 3
Coverage of the Cosmos announcement has blurred two distinct things. Cosmos 3 is the model — a 4B/16B/64B omnimodal world model that understands video, images, text, and physical actions. Cosmos Framework is the toolbox: the inference scripts, training pipelines, and agent skills that let you actually work with those models.
The framework lives at github.com/NVIDIA/cosmos-framework. The model weights live on Hugging Face. You need both, but they serve different roles. If you want to run Cosmos 3 and adapt it to your data, you are working with the framework. That is the story here.
Coding Agents Are Now First-Class Citizens
The most interesting thing about cosmos-framework is not the training scripts — it is the AGENTS.md integration. NVIDIA shipped the repo with five bundled agent skills that make it natively readable by Claude Code, Codex CLI, Cursor, Gemini CLI, and any other AGENTS.md-aware tool.
The skills cover the entire workflow:
- cosmos3-setup — environment install, checkpoint downloads, Docker
- cosmos3-inference — offline and online inference, parallelism, sampling
- cosmos3-post-training — SFT end-to-end: data prep, DCP conversion, training launch, export
- cosmos3-codebase-nav — finding functions and parameters across the codebase
- cosmos3-env-troubleshoot — CUDA errors, Docker issues, checkpoint failures
When you open cosmos-framework in Claude Code and ask “set up Nano and run inference on this video,” the agent reads the cosmos3-setup skill and executes the right steps. You do not read the docs. The agent reads the docs. This is NVIDIA betting that in 2026, that is how developers actually work — and they are right to bet that way.
This is also the first major ML framework repo to ship a full AGENTS.md skill suite. Other repos will follow. Watch for this pattern.
One-Day Post-Training: The Numbers
TAO 7’s agent skills automate the full post-training loop: data patching, baseline evaluation, training container execution, and hyperparameter sweeping via TAO AutoML. The developer interaction is natural language; the agent handles the pipeline.
NVIDIA published real numbers using the Woven Traffic Safety dataset:
- Baseline (no fine-tuning): 54.41% exact-match video Q&A accuracy
- After LoRA post-training: 87.14%
- After LoRA + TAO AutoML (43 Bayesian trials): 93.35%
A single LoRA epoch takes roughly 30 minutes on a node of A100 (80 GB) GPUs. LoRA also uses about 7x fewer GPU hours than full-parameter SFT — which is what makes the one-day timeline realistic for teams without a cluster. The accuracy jump from 54% to 87% is straightforward LoRA. The jump to 93% is AutoML sweeping learning rate, LoRA rank, dropout, batch size, and scheduler settings across 43 parallel trials. That second step is where the meaningful gains are.
The official tutorial is at the NVIDIA developer blog post-training guide. The TAO skill bank is at github.com/NVIDIA-TAO/tao-skill-bank.
Picking Your Model Size
Cosmos 3 ships in three sizes, each targeting a different hardware tier:
- Edge (4B): Jetson AGX Orin and Thor, RTX PRO 6000 — on-device robot deployment
- Nano (16B): RTX PRO 6000, H100, workstation-class — the practical post-training target for most teams
- Super (64B): H200, B200, GB200 — data center scale synthetic data generation
For post-training experiments, Nano is the right starting point. It fits on a single H100 (80 GB) with FP8 quantization, and the TAO agent skills default to Nano. Edge is the deployment target once you have a trained checkpoint — it runs on Jetson Thor at the robot’s edge.
The License Is Not Apache 2.0 (and That Matters)
Cosmos Framework and the model weights ship under OpenMDW-1.1, a license developed by NVIDIA and the Linux Foundation specifically to cover AI model weights — an artifact that standard open-source licenses were not designed for.
What OpenMDW-1.1 permits: commercial use, modification, redistribution, and building commercial products on top. What it does not do: give you the same derivative-work freedom as Apache 2.0. It is not yet OSI-approved (submitted August 2026) and there are restrictions on relicensing derivatives that differ from Apache. Before shipping anything to production, read the license. For enterprise use cases with more complex requirements, NVIDIA offers custom terms at cosmos-license@nvidia.com.
The practical implication: you need to accept the OpenMDW-1.1 terms on each model’s Hugging Face page before the download link unlocks. It is a click, not a legal barrier — but do not skip it.
Who Should Actually Care
The honest answer is: not every developer. If you are building a standard LLM-powered application, Cosmos Framework is not relevant to your stack. This matters to teams building robots, autonomous vehicles, vision AI pipelines, or surgical simulation systems — and to ML engineers who want to generate synthetic training data for physical environments at scale.
The AGENTS.md pattern, though, matters broadly. NVIDIA just demonstrated that a large production ML repo can ship with agent skills that make it accessible to developers who would not otherwise have the depth to use it. That is the template other frameworks will copy. The question is which repo ships the second full AGENTS.md skill suite.
