By ByteBot
JetBrains released Mellum 2.1 on October 8, and the headline benchmark is genuinely hard to dismiss: SWE-bench Verified jumped from 2% to 47% in a single release. SWE-bench is the test that actually matters — it measures whether an AI agent can fix real bugs in real GitHub repositories. The model is 12 billion parameters, Apache 2.0 licensed, and runs on a 12 GB GPU. It’s on Hugging Face right now.
What Changed: It Was All Reinforcement Learning
Here’s the part that matters technically: the architecture is unchanged from Mellum 2. JetBrains didn’t invent new layers or redesign attention. Every gain came from post-training — specifically, reinforcement learning run at a scale most labs haven’t attempted.
JetBrains built infrastructure to run thousands of sandboxed environments simultaneously, then launched millions of RL experiments over the course of the summer. During training, the model invoked real terminal commands and file-editing tools inside actual code repositories. It got rewarded when it passed tests, penalized when it broke things. That’s it. That’s the entire secret — and it’s now table stakes for anyone building a serious coding model.
The implication is uncomfortable for teams still relying purely on supervised fine-tuning: if you’re not doing RL in real environments, you’re falling behind.
The Numbers (Including the Uncomfortable Ones)
The benchmark table doesn’t lie:
| Benchmark | Mellum 2 | Mellum 2.1 | Qwen3.5-9B |
|---|---|---|---|
| SWE-bench Verified | 2% | 47% | 50% |
| SWE-bench Pro | 0.0 | 28.0 | 38.0 |
| LiveCodeBench v6 | — | 82% | 75.4% |
| H200 output throughput | 1x | ~2x | 1x |
Mellum 2.1 is close to Qwen3.5-9B on SWE-bench Verified — but not ahead of it. On SWE-bench Pro, which tests harder multi-file repo-level work, Qwen leads by 10 points. If your use case involves complex multi-file refactors or codebase-level reasoning, Qwen3.5-9B is still the better call.
Where Mellum 2.1 wins outright: LiveCodeBench (82% vs 75.4%) and raw throughput. Under high load on a single H200, it serves nearly twice as many tokens per second as Qwen3.5-9B. For high-throughput parallel agent pipelines, that gap has real cost implications.
How to Run It Today
If you have Ollama installed, this is all it takes:
ollama run hf.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_MThe Q4_K_M quantization weighs in at 8.1 GB and fits comfortably on a 12–16 GB GPU. Macs with M3 or M4 chips work well; 16 GB of unified memory handles most tasks. JetBrains also publishes five GGUF variants down to 7.0 GB (MXFP4_MOE) for tighter hardware, with BF16 at 24.3 GB on the other end if you have a 32 GB card and care about precision.
Teams using JetBrains’ own Junie agent get a natural integration path: Junie Local connects to Ollama, LM Studio, or LiteLLM backends, which means you can run the full Junie workflow with Mellum 2.1 handling inference entirely on-premise. Your code and prompts never leave the machine.
The Cost Argument
GitHub Copilot Enterprise is $39 per seat per month. For a 50-engineer team, that’s $23,400 per year. If you already own GPU hardware — which most engineering teams with any ML or inference workload do — the marginal cost of running Mellum 2.1 is effectively zero.
That said, “free hardware I already own” is doing a lot of work in that sentence. Add ops overhead, latency on underpowered machines, and the reality that Mellum 2.1 isn’t yet as capable as frontier cloud models on complex tasks. The honest self-hosting case is for teams with privacy requirements (regulated industries, proprietary codebases) or those already running GPU clusters for other workloads.
The Architecture in One Paragraph
Mellum 2.1 is a Mixture-of-Experts model with 12B total parameters and 2.5B active per forward pass. It has 64 experts; each token activates 8. This isn’t a trick — it’s how you get model capacity without proportional compute cost. The 131K-token context window is generous for agentic workflows that need to load large codebases. If you’ve heard “12B parameters” and wondered how it runs on 8 GB, this is the answer: it’s not 12B parameters all firing at once.
Should You Switch?
Switch to Mellum 2.1 if you have 12+ GB of VRAM or a modern Mac, you need code to stay on-premise, or you’re running parallel agent pipelines where throughput matters. Stick with Qwen3.5-9B if your work involves deep repo-level reasoning and you’re willing to trade throughput for better SWE-bench Pro scores. Stick with cloud APIs if you need the most capable models and ops overhead isn’t acceptable.
The 2%-to-47% jump is real, and it means open coding agents have crossed from “interesting experiment” to “worth deploying.” That’s a significant milestone — even if there’s still ground to close against Qwen and a wider gap to cloud frontier models.
