推論系統

推論 runtime、量化、 serving、KV cache、成本與效能。

LiteRT.js Brings the Native LiteRT Inference Stack to the Browser, Connecting WebGPU, Wasm, and Experimental WebNN

Google has released LiteRT.js, enabling web applications to run `.tflite` models directly and use the CPU, GPU, or NPU depending on the device. It shifts browser-based AI away from JavaScript-centric runtimes toward a cross-platform native execution stack, although performance and compatibility still depend heavily on the browser and hardware.

2026-07-23

推論系統

Google Launches LiteRT.js, Bringing the Native LiteRT Inference Path to the Browser

LiteRT.js wraps Google’s cross-platform inference runtime in WebAssembly, using XNNPACK, WebGPU, and experimental WebNN for CPU, GPU, and NPU execution, respectively. It makes it easier to move existing `.tflite` models to client-side execution, but performance and compatibility still depend heavily on the browser and device.

2026-07-22