By Every developer building an LLM-powered application eventually writes the same sketchy routing code: call a chat model, prompt-engineer it to return JSON, pray the schema survives a model update, and wrap the whole thing in a try/except that fires more often than it should. OpenAI just shipped a dedicated endpoint to replace that hack — and it charges zero for output tokens.
The OpenAI Decisions API reached general availability on October 22, 2026. It adds a new inference endpoint, POST /v1/decisions, built specifically for bounded classification, routing, and decision tasks — the jobs developers have been forcing chat models to do sideways since GPT-3.
What Makes the OpenAI Decisions API Different From Structured Outputs
The Decisions API is not JSON mode with extra steps. The architectural difference matters: you define your answer space before inference, not a JSON schema to validate after. OpenAI tunes the model to select, not generate. No prose is produced. No output tokens are billed.
Three response shapes are available:
- Predicate — returns a probability (0 to 1) for true/false questions. Use this for gates: “Does this content require human review?”
- Choice — picks one value from your defined list and returns a confidence score. Use this for routing.
- Score — rates against an ordered scale of 2 to 10 levels with probability-weighted index. Use this for priority queues.
You can batch up to 400 questions over a single input in one call. For a complex agent decision point, that is one API call instead of four.
What It Looks Like in Code
Here is the standard ticket-routing pattern that motivated the API. The SDK surface is clean:
from openai import OpenAI
client = OpenAI()
decision = client.decisions.create(
model="gpt-6-luna",
input="I was charged twice for my order #48210.",
questions=[
{
"type": "choice",
"name": "department",
"instructions": "Which department should handle this complaint?",
"choices": [
{"value": "billing", "description": "Payments, invoices and refunds."},
{"value": "technical", "description": "Problems using the product."},
{"value": "shipping", "description": "Delivery and tracking."},
{"value": "other", "description": "Requests outside these categories."},
],
}
],
)
answer = decision.answers[0]
print(answer.name, "->", answer.choice, f"(confidence {answer.confidence:.2f})")
# department -> billing (confidence 0.94)The model is gpt-6-luna — OpenAI’s general text model — and currently the only option available on this endpoint.
The Economic Case
Pricing is where the Decisions API gets interesting. It uses input-only billing:
- Uncached input: $0.10 per million tokens
- Cached input: $0.01 per million tokens
- Output tokens: free
Compare that to Luna via the Responses API at $0.50 per million output tokens. For a system routing ten million requests per day, output token costs drop from roughly $5,000 per month to zero. The latency story is also compelling: OpenAI reports approximately 150 milliseconds end to end, versus a baseline of around 1.5 seconds for structured outputs via the Responses API.
The DevDay 2026 GA release also added prompt caching on the shared input field, cutting costs another 10x on repeated queries with large stable prefixes — useful for agent orchestration loops where system context is reused across many decisions.
When to Use It (and When Not To)
The Decisions API earns its place for any fixed-choice task on the hot path. Specifically:
- Support ticket routing and triage
- Agent next-step selection from a predefined tool list
- Content moderation gates (predicate on each submission)
- Tool-call safety gates: before a high-impact action, confirm intent with a predicate and block if confidence falls below your threshold
- Model routing: decide whether a request goes to a fast cheap model or a deep expensive one
- Real-time voice apps via GPT-Live integration
Do not use it when your answer space is dynamic or unknown at request time, when you need generated text alongside the decision, or when you need a custom JSON schema with nested structures. The Responses API with Structured Outputs handles those cases. The Decisions API does one thing well.
The Honest Caveats
A few things worth knowing before you refactor your routing layer.
First, gpt-6-luna is the only model available on this endpoint. There is no fallback option. If OpenAI ships a regression, you have no immediate alternative without switching back to the Responses API.
Second, confidence scores are not accuracy scores. A response returning 0.94 confidence can still route incorrectly. Before switching your production router, measure actual accuracy against a labeled sample of your real traffic. Do not trust vendor benchmarks as a substitute for your own evaluation.
Third, the caching benefit is more limited than it first appears. The 10x discount applies to large, stable input prefixes — but most real-world routing tasks have variable inputs (the ticket text changes every time) paired with stable question definitions. Community feedback on the OpenAI forum flagged this mismatch directly. The caching advantage is real for agent orchestration with large shared context; less so for document-per-call routing.
The safe rollout path: run the Decisions API in shadow mode against your existing router, compare outputs, verify accuracy meets your threshold, then cut over. The API was in limited preview less than four weeks ago — validate before you trust it with production traffic.
How to Start
The endpoint is live now for all standard API keys. Start with the official Decisions API guide, which includes the full request schema, output type definitions, and prompt caching documentation. If you are already using LangChain, OpenAI Decisions is exposed as a Runnable, so composition with existing chains is straightforward.
If your application routes requests, gates high-impact actions, or runs classification on every call, this is worth an afternoon to evaluate. The DEV Community deep dive covers edge cases and the full GA changelog in detail.
