Needle 3 supports multiple depths with a single set of weights, shrinking on-device tool models to 8–29 MB

Cactus published architecture and deployment documentation for Needle 3 on September 17–18, and the team showcased the on-device automation model on Hacker News. Its focus is packing tool selection, argument filling, and structured data extraction into roughly 8–29 MB of quantized weights, allowing phones, wearables, and small computers to process instructions locally. [Team announcement](https://news.ycombinator.com/item?id=49748553)

The new version supports subnetworks ranging from two to twenty layers using a single set of weights. Pruning preserves the first and last blocks, then progressively adds intermediate blocks, so shallower versions still include computation from both ends of the network. Training samples different depths and uses the full model’s predictions for self-distillation. Engineers can fine-tune first, then export models at different depths to match device resources, reducing the burden of maintaining multiple models separately. Depth is currently selected primarily at deployment time; the official documentation still lists dynamic depth switching for each request as a feature yet to ship. [Architecture documentation](https://cactuscompute.com/blog/intelligence-ladders)

The model card describes designs including Monarch Hadamard operations in place of conventional feed-forward layers, grouped-query attention, and lookup-table memory. For output, a byte-level grammar is compiled from tool schemas to constrain the structure of generated JSON. This ensures a parseable format, but whether the instruction is interpreted correctly and the argument values are correct still requires separate validation. The team has released weights, fine-tuning checkpoints, and runtime engines for multiple platforms. [Model card](https://huggingface.co/Cactus-Compute/needle3)

The performance claims have a narrow scope. Cactus says variants with four or more layers can outperform DeepSeek V4 Flash after fine-tuning on DroidCall, but the comparison uses a setting that forces the models to produce tool calls. For the two-layer version without fine-tuning, enabling a confidence threshold blocked nearly all calls. This indicates that the smallest version still needs task-specific training; these results cannot be taken as evidence of general-purpose agent capabilities. [Evaluation conditions](https://cactuscompute.com/blog/intelligence-ladders)

Two further distinctions matter for adoption: local fine-tuning exports 4-bit weights, while the official 2-bit post-processing depends on the Cactus platform. The GitHub repository also notes that the binary engines enable telemetry by default, so teams with offline and privacy requirements need to check the settings separately. [Deployment documentation](https://github.com/cactus-compute/needle)

In community replies, the team acknowledged that implicit intent and multistep relationships remain difficult to handle, making the model better suited to direct, explicit instructions. Chinese is also absent from the seven languages currently listed as supported. Teams building Chinese-language products should first create test sets covering Traditional Chinese, mixed-language input, and cases where the model should decline to call a tool. They can then determine confidence thresholds and human confirmation workflows, avoiding the mistake of equating correct formatting with correct actions. [Maintainer comments](https://news.ycombinator.com/item?id=49748553)