Ling-3.0-tiny Activates Just 1.3B Parameters per Token, but Local Execution Still Relies on Dedicated Runtime Branches

InclusionAI has released Ling-3.0-tiny under the MIT License, positioning it as a hybrid reasoning model designed to run locally and on edge devices. It has 7.9B parameters in total but activates only 1.3B per token. The model card provides BF16, FP8, and INT4 weights, allowing deployers to balance quality, memory usage, and throughput. [Official model card](https://huggingface.co/inclusionAI/Ling-3.0-tiny)

The architecture is not simply a scaled-down standard Transformer. Each four-layer block combines three Kimi Delta Attention (KDA) layers with one Multi-Head Latent Attention (MLA) layer. The design aims to have linear attention handle most long-context computation, while MLA supplements it with content-dependent global interactions. The FFN contains 128 routed experts; each token selects eight experts and always passes through one shared expert. The model can switch between fast responses and multi-step reasoning through `enable_thinking`. InclusionAI also provides an SGLang recipe for a 256K YaRN context window and NEXTN speculative decoding. [SGLang deployment guide](https://docs.sglang.io/cookbook/autoregressive/InclusionAI/Ling-3.0-tiny)

According to InclusionAI's hardware tests, the FP8 model generates approximately 100 to 105 tokens per second on a DGX Spark and around 86 to 90 tokens per second on an M4 Pro MacBook, with peak memory usage of roughly 8.34 GiB at an 8K context length. Its Artificial Analysis Intelligence Index v4.1.1 and Agentic Index scores are 25 and 16, respectively. However, these figures use different hardware and evaluation recipes, so they are not directly comparable. They are also insufficient to demonstrate the model's long-context quality or real-world agent reliability.

Deployment maturity is currently the most apparent limitation. SGLang uses a development image that tracks the model, while the vLLM installation instructions require InclusionAI's `ling_3_0` branch. Ollama support remains in a pull request, and the model card confirms only the MLX path on Apple Silicon. The model is also not directly hosted by any Hugging Face Inference Provider. Engineering teams should next watch whether runtime support is merged upstream, how INT4 quality performs in practice, how routed experts affect memory behavior over long sequences, and whether the community can independently reproduce the official throughput and agent-evaluation results.