By ByteBot
PrismML dropped Bonsai 2 27B on September 17 — a ternary-quantized version of Qwen3.8 27B that fits a 54GB frontier model into 5.9GB while retaining 98.2% of its benchmark performance. That is not a typo, and it is not marketing rounding. Math retention is 99.5%. Coding is 99.3%. Instruction following actually beats full precision by 1.7%. The question is not whether this compression works. It is which 1.8% you are actually giving up.
Why Ternary Beats 1-Bit
Ternary quantization maps every model weight to one of three values: -1, 0, or +1. Groups of 128 weights share a single FP16 scaling factor so the network can still represent a range of magnitudes. The zero is the key. It lets the network effectively turn a connection off — something pure 1-bit quantization (where every weight must be ±1) cannot do.
PrismML trained Bonsai 2 ternary from scratch, not as post-training quantization of an existing checkpoint. The original Bonsai (based on Qwen3.6) held at 95% retention using the same approach. Bonsai 2 builds on Qwen3.8 with a refined training recipe and hits 98.2%. The extra 3.2 percentage points cost nothing in file size — still 5.9GB, still 1.76 effective bits per weight.
Where the Numbers Land
| Task | Bonsai 2 | Full FP16 | Retention |
|---|---|---|---|
| Math | 96.57 | 97.06 | 99.5% |
| Coding | 81.58 | 82.17 | 99.3% |
| Instructions | 82.66 | 81.25 | +1.7% |
| Agentic / Tool use | 77.57 | 79.74 | 97.3% |
| Knowledge | 83.95 | 86.66 | 96.9% |
| Vision | 78.59 | 81.64 | 96.3% |
The pattern is clear: tasks that require crisp logical reasoning — math, code, following structured instructions — transfer almost perfectly through ternary compression. Tasks that rely on subtle pattern matching across large knowledge bases or perceptual nuance — reasoning depth, vision — see real but modest degradation. A 3% vision gap is not nothing, but it is not the collapse that naive compression theory would predict.
The Hardware You Actually Need
NVIDIA and Apple are the supported platforms right now. AMD GPU users are left out — no ROCm or SYCL support in the current release. That is a real gap PrismML will need to close.
- RTX 5090: 143 tokens/second — the best consumer result
- Apple M5 Max: 46.8 tok/s via MLX — excellent
- Apple M4 Pro (24GB): ~15–20 tok/s (community measured) — usable for dev work
- RTX 4090 (24GB): comfortable fit, 40% more energy-efficient than an 8B full-precision model at the same throughput tier
- iPhone 16 Pro / iPad: runs via Apple MLX — genuinely on-device
For comparison: a standard Q4_K_M GGUF of the same Qwen3.8 27B base runs roughly 16GB and still trails Bonsai 2 in several benchmark categories. The common assumption that more bits always means better quality does not hold here.
Should You Cancel Your API Subscription?
No. But the question is becoming more interesting.
Bonsai 2 makes compelling sense for high-volume, private workloads — CI/CD checks, local coding assistance, document analysis, math-heavy pipelines. Apache 2.0 means you can embed it commercially, fine-tune it, or ship it in a product without licensing friction.
Where it falls short: extended multi-step agentic work. Hacker News testers reported degradation on complex tasks — the model looping or losing coherence in ways that frontier cloud models handle cleanly. Vision pipelines sensitive to perceptual accuracy will also notice the 3.7% gap. For those cases, a subscription is still the right call.
The smarter framing is not binary. Use Bonsai 2 for the workloads it dominates — code review, unit test generation, math verification, local chat. Keep cloud access for the frontier-dependent cases. The combination cuts costs without sacrificing quality where quality is critical.
Access
Bonsai 2 27B is available now on Hugging Face in GGUF format for llama.cpp and Ollama, and via MLX weights for Apple devices. Apache 2.0 license. The full benchmark breakdown is available if you want to verify the methodology before committing.
