By The AI agent bill is no longer theoretical. Uber’s CTO said the entire annual AI budget was gone by April. Engineers running production agentic workloads are seeing $500 to $2,000 per month in API costs per person. Agentic workflows consume 5 to 30 times more tokens per task than a standard chatbot query — and that multiplier only reveals itself after launch. MIT and Microsoft just published a system at OSDI ’26 that cuts those costs by up to 4.3x without requiring you to touch a line of your workflow logic.
What Murakkab Does
Murakkab (Arabic for “composition of things”) is a resource-efficient serving system for agentic workflows, developed by researchers at MIT CSAIL and Microsoft Azure Research. It was presented at USENIX OSDI ’26 — the premier operating systems conference — in Seattle on July 13–15.
The core idea is an inversion of how developers currently build agents. Today, frameworks like LangChain or LangGraph require you to hardcode which model handles each step, which steps run in parallel, and which hardware they land on. When a better model ships, you rewrite the graph. When your cloud bill spikes, you manually tune configurations. Murakkab proposes a different contract: describe what the workflow does, not how to run it, and let the system handle the rest.
Lead author Gohar Chaudhry, a graduate student at MIT CSAIL, put the motivation plainly: “Agentic workflows are getting very complicated and quickly becoming the backbone of what cloud providers are doing.” The research team also includes Adam Belay (MIT CSAIL) and Ricardo Bianchini (Microsoft Azure Technical Fellow).
How It Works
Murakkab operates across three phases.
In the development phase, a developer describes workflow intent in plain language — something like “extract key frames from a video, generate a transcript, and answer user queries about it.” An LLM-based orchestrator maps those task descriptions to executors, producing a logical workflow graph of tasks and dependencies, with no specific model or hardware assignments yet.
In the optimization phase, a profile-guided solver runs a Mixed Integer Linear Programming (MILP) formulation to select the best models, hardware, and parallelism strategy for your defined service-level objectives (SLOs). Those SLOs can express accuracy tiers (best, good, acceptable), latency targets, cost caps, or energy budgets. The profiling layer maintains two sets of data: workflow profiles capturing accuracy and resource demands across configurations, and model profiles capturing latency, energy, and cost under varying load.
In the execution phase, an adaptive runtime monitors live conditions — GPU availability, request load, resource contention — and reconfigures without redeployment. If H100 capacity comes online mid-shift, the runtime rebalances automatically. No code change, no redeploy.
That last point is underrated. Most teams today treat model selection as a deployment-time decision. Murakkab treats it as a runtime concern — the same shift that happened when databases moved query planning from developer responsibility to engine responsibility.
The Numbers
The OSDI ’26 paper reports multi-workflow cluster results worth quoting directly. A static baseline running the evaluated workloads needed 2,560 GPUs, consumed 80.4 MWh of energy, and cost $201,500. Murakkab with GPU multiplexing ran the same workloads on 908 GPUs, consumed 21.6 MWh, and cost $46,500. That is a 64.5% reduction in GPU usage, 3.7x less energy, and 4.3x lower cost — while maintaining the defined SLOs.
Single-workflow results are sharper in specific scenarios. A code generation workload dropped approximately 8.7x in cost when the accuracy SLO was relaxed from “best” to “acceptable.” A video Q&A workflow cut energy by more than 10x with only a 2% accuracy decline. The gap between what users actually need and what developers default to provisioning turns out to be very large.
Where This Sits Relative to LangChain
Murakkab is not a drop-in replacement for LangChain or LangGraph. It operates at a different layer — serving infrastructure rather than application framework. Think of LangChain as the code you write; Murakkab as the runtime that decides how to execute it cheaply.
The timing matters. LangChain’s AgentExecutor reaches end-of-life in December 2026. Teams currently migrating to LangGraph now have a third option in view: systems that move model selection and resource allocation out of application code entirely. Murakkab is a research system today, not a production tool you can install tomorrow. But the trajectory it describes — declarative specs, profile-guided optimization, SLO-constrained reconfiguration — is where production agentic infrastructure is heading.
The Signal
O’Reilly’s Radar for July 2026 put it directly: if agents are going to become the default shape of enterprise software, orchestration has to become an infrastructure discipline. Murakkab coming out of OSDI — not NeurIPS, not MLSys, but the systems conference — is a meaningful signal about where the hard problems have moved. The modeling is solved enough. Running it cheaply at scale is not.
For teams paying production agent bills today, the MIT News writeup is a good starting point and the ArXiv paper is free. The ideas — declarative workflow specs, profile-guided optimization, SLO-constrained runtime reconfiguration — will show up in cloud platforms whether or not Murakkab itself does. Getting familiar with the concepts now is not optional if you’re building production agents.
