By The same request. The same code. The same server. 80 nanoseconds one second, 250 nanoseconds the next. If your p99 tail latency looks fine on paper but something feels wrong at scale, there is a reasonable chance NUMA is the reason — and your cloud dashboard is not going to tell you.
NUMA stands for Non-Uniform Memory Access. It describes the fundamental architecture of every modern multi-socket server: each CPU socket has its own memory controller and RAM banks. Any CPU can access any memory, but accessing remote RAM — attached to a different socket — costs more. How much more? On idle hardware, roughly 1.5x to 3x. Under real load, when multiple cores are competing for the same cross-socket interconnect, reliably 4x to 5x. Edera published a detailed breakdown of NUMA topology this week that hit Hacker News front page with over 100 upvotes, signaling that developers are encountering NUMA in production and want actionable answers. This is not a bug. It is physics.
Why NUMA Exists (And Why It Is Not Going Away)
SMP systems with 2-4 cores could fake uniform memory access. At 64, 128, or 300+ cores across multiple sockets, a shared memory bus cannot scale — electrical trace lengths get too long, and signal propagation delays become measurable. AMD ships Infinity Fabric; Intel ships UPI. These are high-speed cross-socket interconnects. The interconnect adds latency. That latency is NUMA.
On a dual-socket AMD EPYC system set to NPS4 (four NUMA nodes per socket), you get eight NUMA nodes total with as little as 32GB per node. A workload that fits within one CPU’s core count can still require memory from three or four nodes. If your capacity planning assumed “one server, uniform access,” NPS4 breaks that assumption without a single warning log line. Intel offers an equivalent called Sub-NUMA Clustering (SNC), which is off by default on Xeon but available in BIOS.
The First-Touch Bug You Probably Already Have
Linux allocates physical memory on the first write: whichever CPU writes to a virtual address first determines which NUMA node gets the underlying page. This is the first-touch policy, and it creates performance bugs that are almost impossible to spot without NUMA-specific tooling.
The pattern looks like this: an initialization thread allocates and zeros a large buffer on Node 0. Worker threads distributed across all nodes then process it. Every page physically lives on Node 0. Every worker on Node 1 pays remote-access latency on every read. The application logs no error. The health check passes. The metrics look unremarkable except for that stubborn p99 spike nobody can explain.
The inference server version of this is common: a model loader thread initializes weights from a single goroutine, then a request-handling thread pool spread across both sockets serves traffic. Every forward pass crosses NUMA. The cost is invisible until someone runs numastat.
Cloud VMs and Kubernetes Make It Worse
On bare metal, NUMA topology is at least deterministic. In a cloud VM, three independent layers make placement decisions without coordinating: the hypervisor placing vCPUs, the guest OS allocating memory via first-touch, and the container scheduler placing pods based on CPU availability. None of these layers sees the others’ choices.
A Kubernetes pod can end up with its CPUs on NUMA node 2 and its memory on NUMA node 0 because the guest OS defaulted to first-touch from a different goroutine at startup. For GPU workloads, when the container’s CPUs are on a different NUMA node from the GPU, every CPU-to-GPU data transfer crosses a NUMA boundary on the critical path. Research into GPU workload placement on Kubernetes found that pods whose CPUs span both sockets see 30% or more higher p99 tail latency compared to pods pinned to a single socket. That is not a benchmark edge case. That is production.
What Not to Do (and What to Actually Do)
numactl --interleave=all is the fix most developers reach for first. It spreads memory uniformly across all nodes. The thinking is sound: if everything is equally distributed, no node is overloaded. The result is that everything becomes equally slow. You trade unpredictable performance spikes for consistent mediocrity. For workloads with genuinely random memory access patterns, interleaving is reasonable. For anything that can be made locality-aware, it is the wrong answer.
The audit to run right now takes under 30 seconds. On any Linux server, including EC2 instances, AWS provides guidance on reviewing NUMA statistics:
# See your NUMA topology
numactl --hardware
# Check if your process has NUMA misses
numastat -p $(pgrep yourapp)High numa_miss values mean pages are landing on the wrong node due to memory pressure — a direct signal that your process is paying the remote access tax. The targeted fix for a single-process workload:
# Bind process to NUMA node 0 entirely
numactl --cpunodebind=0 --membind=0 ./your-applicationFor Kubernetes, the Topology Manager can enforce NUMA alignment at the pod level, but requires explicit opt-in in the kubelet configuration:
cpuManagerPolicy: static
topologyManagerPolicy: single-numa-nodeMost managed Kubernetes offerings — EKS, GKE, AKS — do not enable these by default. If you are running latency-sensitive or GPU workloads on Kubernetes, this is worth checking today.
Why NUMA Matters More in 2026
NUMA is not new. What is new is the combination: larger multi-socket servers are now the default cloud instance size for serious workloads, AI inference pipelines are both memory-intensive and latency-sensitive, and Kubernetes scheduling does not know about physical memory topology unless explicitly configured.
The tooling to audit and fix NUMA alignment has existed for years. What has lagged is awareness that it applies to everyone running on modern cloud infrastructure, not just HPC engineers and database administrators tuning Oracle clusters. Run numactl --hardware on your next EC2 instance. If you see more than one node — and on anything larger than a small-tier instance, you will — NUMA is a factor in your latency profile. Whether it is a small factor or a large one is something only the data will tell you.
