Ollama Preview Adds Qwen3.8 Flash Next Memory Optimization, Loading Embedding Tables on Demand

Ollama’s November 9 release listing includes preview version v0.40.16, which incorporates an MLX memory optimization for Qwen3.8 Flash Next. The code was merged on November 6 and has now entered the preview release process. Maintainers report that the model’s locked memory footprint fell from 97.6 to 70.8 GiB, with the goal of making local inference more practical on 128 GiB Macs. This figure describes a specific memory metric and should not be taken as the total memory use of the entire machine. Release listing, merged pull request

The optimization targets a PLE embedding table stored in NVFP4 format and occupying about 26.8 GiB. Each forward pass accesses only a few hash-selected rows, but previously the full table had to remain resident in memory. The new path maps safetensors shards into the host address space and moves only the required compressed rows and scale parameters into MLX for dequantization. Other layouts continue to use the existing resident-processing approach. The loading stage also excludes these tensors, avoiding allocating the entire table at startup only to release it afterward. Implementation and review

The quantization strategy has also changed. The author says applying NVFP4 across the board would hurt long-form generation, so sensitive QSA projections and the non-expert MTP path retain BF16. Other suitable dense and control weights use MXFP8, while routed experts and the embedding table remain in NVFP4. This makes memory allocation and numerical precision parts of the same design problem. Quantization design

This approach is directly tied to the model architecture. Ollama’s model page lists 125B total parameters, with 6B parameters activated per token, as well as a large N-gram embedding table. For engineers, sparse activation reduces computation, but the costs of storing and moving weights still need to be addressed. The practical value of this update is placing data according to how it is accessed. Model architecture

Before deployment, prompt prefill, per-token decoding, and peak memory use in long conversations should be measured separately. Maintainers say they observed no measurable drop in target decoding throughput, but discussion has also included reports of slower prefill for long prompts, without controlled comparison tests. The model page still lists a 105GB package size; download size and runtime locked memory should be interpreted separately. Next, watch for inclusion in a stable release and reproducible results on the same hardware and with the same prompt lengths. Performance discussion, package information