開放權重模型
Solar Open 2 Supports a Million-Token Context with Hybrid Linear Attention, Activating Just 15B of Its 250B MoE Parameters per Token
South Korea’s Upstage has released Solar Open 2 for long-horizon agents, combining a 250-billion-parameter MoE, hybrid attention, and a 1-million-token context window in a single model. The company says the quantized model can be deployed on two NVIDIA H200 GPUs, though current evidence for its performance and long-context stability comes primarily from the team’s own testing.
Upstage released the Solar Open 2 weights and technical report on July 22. The model has 250 billion parameters in total but activates only 15 billion parameters for each token. Its 48-layer backbone includes 320 routed experts and one shared expert, using a sparse Mixture-of-Experts (MoE) architecture to retain the capacity of a large model while reducing the computation required for each forward pass.
Its long-context design is even more notable. Solar Open 2 inserts one standard softmax attention layer after every three linear attention layers and eliminates positional encoding. The linear layers accumulate historical information in a recurrent state, while the softmax layers provide precise token-to-token relationships. Upstage says this approach extends the context window from the previous generation’s 128K to 1 million tokens. Although it avoids the extrapolation limits of Rotary Position Embeddings (RoPE), it does not mean that all information within a million-token context can be retrieved with equal reliability. Engineering teams should still test for positional bias in long documents, loss of tool state, and KV-cache or recurrent-state memory requirements.
Post-training data covers conversational tool calling, software development, and office work, while retaining a tokenizer optimized for Korean. According to the report, tokenizers used by other global models consume approximately 1.2 to 1.9 times as many tokens when processing the same Korean text. This has direct implications for Korean-language Retrieval-Augmented Generation (RAG), agent trajectories, and API costs. Upstage also claims that the model leads comparable open-weight models on MMLU-Pro, LiveCodeBench, and APEX-Agents, and that its quantized version fits on two H200 GPUs.
However, these comparisons are still supplied by the development team, and “15B parameters activated per token” does not mean that only 15B parameters’ worth of weights need to be stored. Self-hosted serving must still handle the full 250B weights, expert routing, and inter-GPU communication. Key areas to watch next include third-party long-context stress tests, degradation in tool-calling performance after quantization, and whether runtimes such as vLLM, SGLang, and llama.cpp can efficiently support the model’s hybrid attention architecture.