推論與開發工具
LiteRT.js Brings the Native LiteRT Inference Stack to the Browser, Connecting WebGPU, Wasm, and Experimental WebNN
Google has released LiteRT.js, enabling web applications to run `.tflite` models directly and use the CPU, GPU, or NPU depending on the device. It shifts browser-based AI away from JavaScript-centric runtimes toward a cross-platform native execution stack, although performance and compatibility still depend heavily on the browser and hardware.
Google is extending its on-device inference framework, LiteRT, to the browser with LiteRT.js, a JavaScript/TypeScript binding. Unlike earlier web inference solutions that relied primarily on JavaScript cores, the new runtime brings LiteRT’s native optimization stack to the browser through WebAssembly: the CPU path uses XNNPACK, the GPU path runs ML Drift through WebGPU, and the NPU path targets WebNN. Existing `.tflite` models can be loaded, compiled, and run entirely in the browser. PyTorch models can also be converted first using LiteRT Torch and paired with layer-by-layer quantization tools to reduce model size and computational requirements.
The significance of this change goes beyond “offline execution.” Keeping models and input data on the client can reduce server-side inference costs, network round-trip latency, and the risk of exposing sensitive data. It also brings object detection, audio processing, vector search, and image enhancement closer to real-time interaction. The LiteRT repository is licensed under Apache 2.0 and lists WebGPU and Wasm web inference as officially supported capabilities. Google has also demonstrated workloads including YOLO, Depth Anything V2, and Real-ESRGAN.
Google claims that, in a controlled browser environment on a 2024 M4 MacBook Pro, LiteRT.js was up to three times faster than other web runtimes for some traditional vision and audio models. WebGPU or WebNN paths using Apple Core ML delivered speedups of 5× to 60× over CPU execution. These figures are not universally applicable across devices: browser drivers, GPU capabilities, thermal conditions, and operator coverage can all affect the results, while WebNN remains experimental in Chrome and Edge.
Engineering teams should next measure cold-start time, model download time, peak memory usage, energy consumption, and tail latency across their target device matrix instead of comparing only single-inference performance. They should also track WebNN standardization and browser adoption, operator coverage for generative models, and quality degradation after quantization. These factors will determine whether LiteRT.js can move beyond demos and into products at scale.