Cloudflare Cut 100TB of RAM With 5 Rust Struct Fixes

Cloudflare’s DNS resolver stores over 250 billion cache entries at any given moment. Last week, its engineering team published how they freed roughly 100 terabytes of RAM from their global fleet — not by adding hardware, but by fixing how a Rust struct stores its fields. The five techniques they used apply to almost any Rust service that stores a lot of something.

Why 8 Bytes Becomes 2 Terabytes

The DNS cache lives inside a service called Big Pineapple, Cloudflare’s Rust-and-Wasm-based resolver behind 1.1.1.1, Gateway DNS, and DNS Firewall. At 250 billion entries, the arithmetic is unforgiving: one wasted byte per entry equals 250 GB of dead RAM across the fleet. Eight wasted bytes equals 2 TB. The engineers found dozens of wasted bytes per entry — all hiding in perfectly ordinary Rust types.

The culprit behind the single biggest savings was Vec<T>.

The Capacity Field You’re Never Using

In Rust, a Vec<T> is three pointers wide: a pointer to the heap allocation, a length, and a capacity. That capacity field exists so the vector can grow without reallocating. It’s the right default for mutable collections. But DNS cache entries are never mutated after they’re written — they’re set once, read many times, then evicted. The capacity field is permanently dead weight on every single entry.

The fix is replacing Vec<T> with Box<[T]> and String with Box<str>. A boxed slice drops the capacity field, cutting each field from 24 bytes to 16 bytes. Across eight fields per cache entry, that’s 64 bytes saved per entry — roughly 15 terabytes freed fleet-wide from this one change alone.

// Before: capacity field burns 8 bytes per field, unused
name: String,          // ptr + len + cap = 24 bytes
records: Vec<Record>,  // ptr + len + cap = 24 bytes

// After: boxed slices carry only what they need
name: Box<str>,           // ptr + len = 16 bytes
records: Box<[Record]>,   // ptr + len = 16 bytes

If you have any Rust struct storing immutable collections — caches, configuration, pre-built response objects — audit every Vec and String field. For write-once, read-many data, Box<[T]> is almost always the right choice.

The Enum That Was 36x Larger Than Necessary

DNS record data is naturally an enum: an A record holds an IPv4 address (4 bytes), an AAAA record holds an IPv6 address (16 bytes), and a NAPTR record holds a complex structure (136 bytes). Rust enums must be sized to accommodate their largest variant. That means every A record — 56% of Cloudflare’s DNS traffic — was padded to 144 bytes to make room for a variant it would never hold.

The fix is to box the outlier. When Naptr becomes Box<Naptr>, the heap handles the allocation and the inline enum variant shrinks to 8 bytes. The common A and AAAA variants stop paying the padding penalty.

// Before: NAPTR forces all variants to 144 bytes
pub enum RecordData {
    A(Ipv4Addr),    // 4 bytes of real data, 144 bytes of storage
    Aaaa(Ipv6Addr), // 16 bytes of real data, 144 bytes of storage
    Naptr(Naptr),   // 136 bytes — the whole fleet pays for this
}

// After: box the outlier, common paths stay compact
pub enum RecordData {
    A(Ipv4Addr),          // 4 bytes
    Aaaa(Ipv6Addr),       // 16 bytes
    Naptr(Box<Naptr>),   // 8 bytes (pointer to heap)
}

This pattern generalizes. If your enum has one large variant that appears rarely in production, box it. The extra allocation cost for the rare case is almost always cheaper than the padding penalty imposed on the common case across millions of instances. The Rust Performance Book’s type sizes section covers this pattern in detail.

Stop Parsing, Start Copying

The most counterintuitive optimization — and the one with a speed bonus — was to stop storing parsed record data entirely. Instead of structured Rust types, Cloudflare now stores each record as raw wire bytes with a 2-byte length prefix. When a DNS response needs to go out, the bytes copy directly from cache to wire without re-serialization.

This eliminated per-variant enum overhead, removed most heap allocations per record, and packed data contiguously — improving CPU cache locality. The result: 5% lower lookup latency and 13% higher insert throughput. The memory savings also made the system faster. These things went together, not against each other.

Worth internalizing: if your code always re-serializes a type before sending it downstream, you may be doing unnecessary work. Store the wire format, skip the parse-store-serialize cycle.

Two More Changes That Added Up

DNS responses have three sections: answer, authority, and additional. The original code stored each as a separate list — three pointer/length pairs at 8 bytes each, 48 bytes total. The fix: one flat list with two u16 offsets marking section boundaries. Three 8-byte pairs became 4 bytes total, saving 28 bytes per entry while also making data contiguous.

The fifth change applied Option to the record owner field. Most records’ owners match the queried domain — already in the cache key. Storing None for the common case and inferring the owner at lookup time eliminated the majority of heap allocations for owner data without changing external behavior.

The Results

Metric	Before	After	Change
Per-entry footprint	953 bytes	420 bytes	-56%
Per-entry allocations	1.1 KB	461 bytes	-58%
Insert throughput	625K/s	893K/s	+43%
Lookup latency	828 ns	670 ns	-19%
Fleet memory saved	—	~100 TB	—

Production memory per instance dropped 42–43% at both p90 and p99. The rollout ran from May 18 to July 6, 2026 — not because the code was slow to write, but because migrating a live data structure serving billions of queries requires careful, staged deployment. As several engineers noted in the Hacker News discussion (593 points), the code change is the fast part. The production migration is the real engineering.

What This Means for Your Code

Most Rust services will never serve 250 billion cache entries. But these techniques are not scale-specific. If you have a cache, an in-memory store, or any collection that is written once and read many times: audit your Vec and String fields, check whether your enum’s largest variant is padding the common ones, and ask whether you’re parsing data that you’ll immediately re-serialize anyway.

The full write-up, including intermediate optimization steps and benchmark methodology, is on the Cloudflare engineering blog.