By ByteBot
Together AI closed an $800 million Series C last week at an $8.3 billion valuation. The valuation is not the story. The story is $1.15 billion in annual bookings, 200+ open models behind a single OpenAI-compatible API, and an inference engine that hits 500 tokens per second on DeepSeek-V3.1. Open-source inference has crossed a line that closed-model providers can’t easily uncross. Developers running production workloads on GPT-5.x or Claude should run the math on switching.
The Cost Case Is Now Concrete
The 2025 argument for open models was qualitative: “the quality gap is closing.” The 2026 argument is quantitative: the gap is effectively closed for most workloads, and the cost difference is not incremental — it’s structural.
Qwen 3 235B runs on Together AI at $0.09 per million input tokens and $0.10 per million output tokens. GPT-5.4 Pro costs $30 input and $180 output. DeepSeek V4 posts 83.7% on SWE-bench Verified while running roughly 34x cheaper per output token than GPT-5.5. Together AI’s reported figure — companies switching to open models achieve 6x to 20x lower costs — lines up with the public pricing data.
Do the math on your current monthly API bill. Divide it by six. That’s your floor after switching to open models on Together, on the conservative end of their estimate.
Switching Is Two Lines of Code
The migration barrier is smaller than most developers assume. Together AI exposes an OpenAI-compatible API. You change two things: the API key and the base URL. Your existing OpenAI SDK calls — chat completions, streaming, function calling — run unchanged.
import os
import openai
client = openai.OpenAI(
api_key=os.environ.get("TOGETHER_API_KEY"),
base_url="https://api.together.ai/v1",
)
response = client.chat.completions.create(
model="meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8",
messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)The one adjustment: Together model IDs are namespaced. Instead of gpt-4o, you use meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8. The Together model catalog lists every available model with its exact ID and per-token pricing.
ATLAS: The Inference Engine That Learns From Your Traffic
Speed was the last reasonable objection to open inference at scale. Together AI’s proprietary ATLAS engine (AdapTive-LeArning Speculative System) addresses it with an approach that’s more interesting than standard speculative decoding.
ATLAS runs two speculators simultaneously. A heavyweight static speculator, trained on a broad corpus, provides a reliable performance floor. A lightweight adaptive speculator continuously updates from live production traffic — it gets faster the longer it runs on your specific workload. A confidence-aware controller picks between them at each decoding step, using longer speculative chains when confident and shortening them when it detects drift.
The reported numbers: 400% faster than a vLLM baseline, 500 tokens per second on DeepSeek-V3.1, 460 TPS on Kimi-K2. These are company-reported figures; independent benchmarks have not been published yet. Run your own production workloads before making infrastructure commitments, but the architecture is sound and the mechanism is real.
Which Model to Use
With 200+ models in the catalog, the choice isn’t obvious. A practical starting point:
- Coding tasks and agents: DeepSeek V4 (83.7% SWE-bench Verified, ~$0.14/$0.28 per million tokens) or GLM-5 (77.8% SWE-bench, strong on agentic tool use)
- Reasoning and math: Qwen 3 235B (77.2% GPQA Diamond, cheapest capable open model at $0.09/$0.10)
- General chat and summarization: Llama 4 Maverick 17B — fast, cheap, strong quality for most tasks
- Multimodal: Llama 4 Scout or MiniMax M3 for vision and text together
The real advantage of 200+ models behind one API key is A/B testing. Swap models in your application config, run both on production traffic, and let your eval metrics decide. No vendor negotiation, no separate onboarding — just change the model string.
What the $800M Actually Means
The model war is effectively over for most production use cases. Open models are good enough — in many benchmarks, better. The next competitive dimension is who can run those models cheapest, fastest, and most reliably at scale. Together AI is making the same bet AWS made on cloud hosting: the winner won’t necessarily have the best software, but will have the best economics.
With 500+ MW of committed compute and a 50x capacity expansion target, Together AI is building for a world where inference costs continue to fall. That’s structurally good for developers. More infrastructure capacity creates pricing pressure across the entire inference market — including the closed-model providers.
Get an API key at api.together.ai and read the Series C announcement for Together AI’s own framing on where the infrastructure is headed. The ATLAS technical blog post goes deeper on the speculative decoding architecture if you want the implementation details.
