By ByteBot
On August 14, 2026, Google released HEIR (Homomorphic Encryption Intermediate Representation) — an open-source compiler that converts pre-trained AI models to run on fully encrypted data, without the server ever decrypting it. Historically, building anything with fully homomorphic encryption (FHE) required a team of cryptographers. HEIR changes that: annotate your Python types to mark sensitive data, install heir_py from PyPI, and the compiler handles the rest. Four production applications are already live, including a credit card fraud detector and a voice hotword recognizer that never hears the actual audio.
What HEIR Actually Does
HEIR is built on MLIR, the same intermediate representation framework used by modern ML compilers including TensorFlow’s XLA. It operates as a multi-layer compiler toolchain: developers write Python, annotate which data is secret, and HEIR compiles the program to run under one of several FHE backends — OpenFHE, Lattigo, tfhe-rs, or Jaxite. The compiled model runs on the server, operating on ciphertext the entire time. The server never sees the plaintext input.
The developer-facing API is deliberately minimal. A Python type annotation marks which function arguments are secret:
from heir_py import secret
def detect_fraud(transaction: secret[float], threshold: float) -> bool:
# The server never sees 'transaction' — only encrypted ciphertext
return transaction > threshold
HEIR also targets hardware accelerators directly, generating code for GPU, TPU, FPGA, and custom ASICs. The GitHub repository has 805 stars and active weekly office hours. Full technical documentation is at heir.dev.
Four Production Demos That Are Not Toys
Google shipped HEIR alongside four working production applications, which matters. A new cryptographic toolchain with zero deployments is a research announcement. Four deployments across different verticals is something else entirely.
The four use cases: a deep learning recommendation model that serves personalized results without exposing what users are interested in; a credit card fraud detector built with Niobium and hardshell.ai that analyzes transactions without seeing the transaction details; the Kitsune network threat detection system, which identifies intrusions in encrypted traffic without packet inspection; and a hotword detector that recognizes voice commands without the server ever processing the audio. These span finance, security, audio, and recommendations — four genuinely different domains, not a single narrow niche.
Related: Claude Now Watermarks Its Text — What Developers Must Know
Homomorphic Encryption Performance: The Real Numbers
HEIR is a real milestone. The 1,000x overhead is also real. Fully homomorphic encryption currently runs 1,000x to 10,000x slower than plaintext computation. Community benchmarks from the Hacker News discussion (365 points, 215 comments) show sorting 32 integers takes roughly 34 seconds under FHE; equality checks run at ~80ms; a division operation takes ~8 seconds. Running LLM inference under FHE produces approximately 0.00008 tokens per second. That is not a rounding error — that is a fundamentally different category of usability.
The practical rule: if your plaintext operation takes microseconds or low milliseconds, FHE overhead is tolerable. If it takes seconds, FHE is not yet the right tool. The four production demos all fit the first category — bounded, small-input computations, not frontier model inference. Google notably did not publish specific latency numbers for the demos. The community noticed.
When to Use FHE vs. Trusted Execution Environments
For most developers evaluating private AI inference today, the real choice is between FHE and Trusted Execution Environments (TEEs). TEEs — Intel SGX, AMD SEV-SNP, ARM TrustZone, NVIDIA Confidential Compute — provide hardware-enforced isolation with only 7–8% overhead for LLM inference. That makes TEEs practical for real-time, high-throughput workloads right now, with no cryptographic heroics required.
FHE via HEIR fills a different niche: scenarios where you cannot trust any hardware vendor’s supply chain and need zero-trust cryptographic guarantees. Healthcare diagnostics on encrypted patient records, cross-organization data pipelines where no party can see raw data, and private lookup queries — these are cases where FHE’s guarantees matter enough to pay the performance cost. However, the correct approach is to choose based on your actual threat model, not the technology’s marketing narrative.
Key Takeaways
- Google released HEIR on August 14, 2026 — an open-source MLIR-based compiler that converts AI models to run on fully encrypted data; install via
pip install heir_py - Four production applications are already live across fraud detection, hotword recognition, network security, and recommendations — this is beyond research stage
- FHE overhead remains 1,000x–10,000x versus plaintext; LLM inference is not viable; bounded computations on small inputs work today
- Trusted Execution Environments (AMD SEV, Intel SGX) carry only 7–8% overhead for LLM inference — use TEEs for real-time AI, HEIR for zero-trust bounded computations
