Bonsai 2 Compresses 27B Language Model Weights to 5.95 GB, but Deployment Still Requires Specialized Runtime Kernels

PrismML released Bonsai 2 27B on September 17, based on Qwen3.8-27B, with weights available under the Apache 2.0 license. The update raises overall benchmark performance retention from roughly 95% in the previous generation to a claimed 98.2%. For deployment, however, the central question remains whether the compression format, runtime kernels, and task quality can work together in practice. [Release announcement](https://prismml.com/news/bonsai-2-27b)

The model represents its main language weights as −1, 0, and +1, with each group of 128 weights sharing a half-precision scaling factor. It also applies blockwise Hadamard orthogonal rotations before quantization. These rotations are folded into the stored weights, while corresponding transformations must be applied to activations during inference. A small subset of recurrent states and normalization parameters remains at higher precision, so the entire model should not be understood as using purely ternary arithmetic. [Model card](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)

The downloadable files offer two tradeoffs: PTQ1_0 densely packs ternary values, bringing the language model to 5.95 GB; PQ2_0 uses two-bit slots and takes up 7.21 GB in exchange for lower unpacking overhead. Image input requires an additional vision module of approximately 0.63 GB. Deployment must also account for the context cache and temporary buffers; the weight file size cannot be treated as the system’s total memory requirement. [Format details](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)

Compatibility is a deployment hurdle. The official demo repository explicitly requires PrismML’s runtime fork. The previous generation’s format has been integrated into the llama.cpp mainline, but that does not mean Bonsai 2 can already run there directly. A runtime that understands the low-bit packing but omits the activation rotations may still produce incorrect outputs. The team therefore provides installation and benchmarking procedures with pinned versions. [Runtime project](https://github.com/PrismML-Eng/Bonsai-demo)

This means the effective deployment unit is the combination of weights and runtime. Teams seeking to provide reproducible services should pin both versions and retain test cases for output regression testing. When switching to mainline kernels later, they should also revalidate the image and tool-calling paths. For maintainers, progress on upstream integration will directly affect update costs and package compatibility.

Evaluation methodologies also need to be aligned: the release page lists 20 tests, with aggregate scores of 83.9 versus 85.4; the current model card lists 14, with averages of 84.78 versus 86.32. Both work out to roughly 98.2%, but they are different sets of measurements, and neither implies that every capability loses only 1.8%. [Official evaluation](https://prismml.com/news/prismml-launches-bonsai-2-27b), [Per-test results](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf)

For developers building local agents, the value lies in fitting a larger model into a smaller memory budget. The next developments to watch are mainline integration and independent retesting with fixed prompts, context lengths, and tool environments. Before production adoption, teams still need to verify success rates on extended workflows and tail latency; average benchmark scores cannot guarantee either.