MoE inference | byteiota

Tag: MoE inference

Data visualization dashboard comparing Colibri inference speeds across CPU-only, M4 Pro, and M5 Max hardware with storage hierarchy diagram

AI & Development

Colibri: Run a 744B MoE Model Locally on 25GB RAM

The headline writes itself: run a 744-billion-parameter model on a laptop. The fine print is ...

September 18, 2026

llama.cpp 0.4.0 — Lazy Tensors, Video Input, and Server Deployment Fixes

AI & Development

llama.cpp 0.4.0: Lazy Tensors, Video, and Server Fixes

llama.cpp 0.4.0 ships lazy tensor reading, per-slot context limits, video input, and sparse flash attention. ...

September 5, 2026

Colibri inference engine diagram showing a hummingbird made of circuit traces streaming expert model weights from NVMe to RAM

AI & Development

Colibri Runs a 744B Model on 25GB RAM — Here’s the Architecture

Colibri is a pure-C, zero-dependency inference engine that streams 744B MoE expert weights from NVMe ...

August 13, 2026