Google Unveils HEIR: AI That Computes on Encrypted Data, Without Decrypting It

Google unveiled HEIR, an open-source compiler that converts pretrained AI models so they can run directly on encrypted data, without ever needing to decrypt it on the server along the way. It sounds like a contradiction: how does a model compute anything on data it can't see. The answer is homomorphic encryption, a technology known for years but only now genuinely closing in on practical use.

How this is even supposed to work

Fully homomorphic encryption (FHE) lets you run computations directly on encrypted data, and the result of those computations, once decrypted, matches exactly what you'd get running the same operations on plaintext data. The server processing the data never sees it in readable form, it only ever operates on a mathematical ciphertext. HEIR, short for Homomorphic Encryption Intermediate Representation, is a compiler layer built on top of the MLIR framework that automates turning an existing, already-trained AI model into a version capable of working on that kind of encrypted input.

Google demonstrated HEIR on four specific applications: deep-learning-based recommendations, credit-card fraud detection, network intrusion detection, and hotword detection, the mechanism behind "OK Google" and similar wake phrases. That's not a random selection: all four scenarios share the trait that the input data is inherently sensitive, purchase history, card numbers, network traffic, voice recordings, while the company processing that data wants, or is regulatorily required, to be able to prove it never had readable access to it.

Why this didn't work in practice before

Homomorphic encryption has existed as a mathematical concept for decades, and the first practical constructions appeared more than fifteen years ago. Performance was always the problem: operations on encrypted data run orders of magnitude slower than the same operations on plaintext, which for years confined FHE to niche academic use rather than real production systems serving millions of requests a day.

HEIR attacks that problem from two directions at once. First, as a compiler it automates a process that previously needed hands-on, expert work from cryptography engineers for every single model, meaningfully lowering the barrier for companies that want to deploy FHE without hiring their own specialist team. Second, Google is co-developing hardware acceleration with Belfort, Niobium, Cornami, and Optalysys, all building chips dedicated to speeding up homomorphic operations specifically, meant to close part of the performance gap at the silicon level, not just the software level.

What this actually changes

For companies handling financial, medical, or any other category of data under strict regulation, HEIR opens up a scenario where you can use an external AI compute provider without actually handing over data in a form that could be read or misused, even in the event of a breach or leak on the provider's side. That's a different level of security than standard transport encryption (HTTPS) or encryption at rest, both of which protect data in transit or on disk, but not during processing itself, exactly the moment where most real leaks and abuse actually happen.

HEIR is publicly available on GitHub under an open-source license, with documentation, weekly office hours, and monthly community meetings. That's a signal Google isn't treating this as a closed, internal tool, it's meant as a foundation for other companies and researchers to build their own applications on top of, similar to what's happened with several other Google projects built on MLIR before it.