By Google shipped LiteRT.js on July 9 — a JavaScript runtime that executes AI models entirely inside the browser with no server required. It runs up to 3x faster than TensorFlow.js and ONNX Runtime Web on CPU and GPU benchmarks, and between 5x and 60x faster when it taps the GPU via WebGPU. One npm install, and your per-query inference cost drops to zero.
What LiteRT.js Actually Is
LiteRT is Google’s successor to TensorFlow Lite — a runtime for on-device AI inference. LiteRT.js is the JavaScript binding of that same runtime, compiled to WebAssembly, so it runs in browsers. It loads .tflite models and executes them client-side. No API calls. No GPU server. No data leaving the device.
The key departure from TensorFlow.js is architectural. TF.js runs JavaScript kernels — fast by JS standards, slow by native code standards. LiteRT.js routes through WebAssembly, which means you get the same optimized runtime Google uses on Android, now running in Chrome or Firefox.
Three Backends: Pick Your Hardware
LiteRT.js exposes three execution backends, and the choice matters for performance:
- wasm — CPU inference via XNNPACK, Google’s highly optimized kernel library with multi-threading and relaxed SIMD. Works in every modern browser, no setup required.
- webgpu — GPU inference via ML Drift, Google’s current-generation GPU engine. Targets the WebGPU API. Stable in Chrome, Edge, and Firefox. This is the one you want for demanding workloads.
- webnn — NPU inference via the WebNN API. Experimental. Requires Chrome or Edge with flags enabled, plus Windows 11 24H2+. Do not ship this to production today.
For most real-time applications — object detection, image processing, audio transcription — reach for webgpu. WebNN will matter in 2027 when the spec stabilizes and NPU hardware is everywhere. For now, it is a preview.
Performance Numbers
Google’s benchmarks, run on an M4 MacBook Pro, show LiteRT.js delivering up to 3x speedups over TensorFlow.js and ONNX Runtime Web on both CPU and GPU. Move from wasm to webgpu and demanding workloads — real-time object tracking, audio processing, image manipulation — jump 5x to 60x.
Take those numbers seriously but not literally. Performance scales with your user’s GPU, thermal envelope, browser driver quality, and model size. The headline numbers represent an upper bound on modern Apple Silicon, not an average across the fleet.
Getting Started
Install from npm:
npm install @litertjs/coreCopy the Wasm files from node_modules/@litertjs/core/wasm/ to your public directory — LiteRT.js needs to load them at runtime. Then, a typical inference flow:
import { loadLiteRt, loadAndCompile } from '@litertjs/core';
// Initialize the Wasm runtime
await loadLiteRt('path/to/your/wasm/');
// Load and compile the model, selecting a backend
const model = await loadAndCompile(
'/models/mobilenet_v2.tflite',
{ accelerator: 'webgpu' }
);
// Run inference
const results = await model.run(inputTensor);
const output = await results[0].data();
// Always clean up
inputTensor.delete();
results[0].delete();Memory management is manual. Call .delete() on every tensor or you will leak. This is a WebAssembly reality, not a LiteRT.js quirk.
Model Conversion: The Old Pain Is Gone
The previous TensorFlow.js workflow involved four conversion steps: PyTorch → ONNX → TensorFlow → TensorFlow.js format. Each handoff introduced potential shape mismatches, quantization issues, and debugging sessions.
LiteRT.js collapses this to one step. PyTorch models convert to .tflite directly via LiteRT Torch. TensorFlow and JAX models follow the same path. If you have been avoiding browser AI inference because the conversion pipeline was painful, that excuse is now gone.
What You Can Build
Google shipped working demos: YOLO-based object detection running in the browser with no server, a webcam feed converted to a 3D point cloud using Depth-Anything-V2 in real time, and 4x image upscaling via Real-ESRGAN running entirely client-side.
The cost math is straightforward. On-device inference runs roughly 90% cheaper than cloud equivalents for high-volume apps. A $0.50-per-query cloud call becomes $0.05 on-device, and at thousands of queries per hour, that is not a marginal savings — it is a different business model.
The TensorFlow.js Verdict
TensorFlow.js is not going anywhere. It has years of ecosystem depth, community tutorials, and compatibility with the SavedModel format. But for new projects where you control the model format and care about performance, LiteRT.js is the better starting point. See the official GitHub repo for model examples and roadmap.
The browser is now a serious AI inference runtime. LiteRT.js is the clearest evidence of that yet. The only thing holding it back from production ubiquity is WebNN’s experimental status — and that clock is running.
