推論系統與開發工具
Google Launches LiteRT.js, Bringing the Native LiteRT Inference Path to the Browser
LiteRT.js wraps Google’s cross-platform inference runtime in WebAssembly, using XNNPACK, WebGPU, and experimental WebNN for CPU, GPU, and NPU execution, respectively. It makes it easier to move existing `.tflite` models to client-side execution, but performance and compatibility still depend heavily on the browser and device.
Google has released LiteRT.js, adding a JavaScript/TypeScript interface to LiteRT that allows web pages to load, compile, and run `.tflite` models directly. The key change is not the creation of another JavaScript compute core, but the compilation of the native LiteRT runtime to WebAssembly, bringing its existing operator optimizations, quantization workflows, and hardware backends into the browser.
The CPU path uses XNNPACK with multithreading and SIMD support; the GPU path runs ML Drift through WebGPU; and NPU support is planned through WebNN, which remains experimental in Chrome and Edge. LiteRT Torch can convert PyTorch models, while AI Edge Quantizer supports per-layer quantization configuration. Google also provides a TensorFlow.js interoperability package, allowing existing applications to retain their pre- and post-processing code while replacing only the model execution layer, reducing migration costs.
Google says that in tests involving vision and audio models, LiteRT.js was up to roughly three times faster than other web runtimes. With a GPU or NPU, it delivered speedups ranging from five to sixty times over its CPU path. However, these figures come from a controlled environment using a 2024 M4 MacBook Pro and cannot be directly extrapolated to Android devices, Windows PCs, or different browsers. Drivers, thermal constraints, operator coverage, and data transfers can all affect the results.
For engineering teams, the value lies in moving small language models, object detection, speech processing, vector search, and image enhancement to the client, reducing server inference costs, round-trip latency, and the transmission of raw data. However, model weights must still be downloaded to the device, increasing initial load time and offering no protection for model intellectual property.
Key issues to watch include whether WebNN develops into a consistent cross-browser NPU interface, whether memory management and operator coverage for generative models mature, and whether Google’s benchmarks can be reproduced across a wider range of hardware. Before production adoption, developers still need to measure cold-start latency, weight caching, peak memory usage, the proportion of operators that fall back to WASM, and end-to-end power consumption.