推論與模型架構
MACRO Reorders Frozen Transformer Layers, Raising Average Accuracy by 5.0 Percentage Points Across Six Models
MACRO does not update model weights. Instead, it searches for a fixed execution path for each task that can skip layers, jump backward, and reuse hidden states. Across 13 benchmarks, the authors report reducing search time from 14.8 hours to 1.6 hours, though every new task still requires labeled training and validation data.
New research called MACRO rewrites a Transformer's fixed, sequential layer-by-layer forward pass as a searchable program. The model can skip a layer, return to an earlier layer and execute it again, or add a previous hidden state back into the residual stream before reconnecting to the original later portion of the network. Model weights remain frozen throughout the process, with no additional neural router trained and no new search performed for each test input. The resulting path is task-specific and shared by all samples at deployment time.
Its search mechanism is a Markov policy whose state comprises the current layer, remaining compute budget, direction of movement, and operation type. In each round, it samples 30 candidate paths, evaluates them on partitions of the training data, and updates transition probabilities using the highest-scoring samples. The process runs for at most 10 rounds. The system then uses top-k Viterbi decoding to find the five highest-probability valid paths exactly, after which an independent validation set selects one. According to the authors' measurements, this reduces Dr.LLM's search time—which relies on MCTS to generate routing labels—from 14.8 hours to 1.6 hours.
Across 13 reasoning and knowledge benchmarks using configurations including Qwen3 1.7B, 4B, 8B, and 14B; Llama 3.2 3B; Mixtral 8x7B; and DeepSeek-R1-Distill-Llama-8B, MACRO improves average accuracy by 5.0 percentage points over the standard layer order and outperforms Dr.LLM by 7.2 points. The most striking result is Qwen3-1.7B on GSM8K: after executing layers 0 through 7, the model returns to layer 3, replays five blocks, and then proceeds to the end, increasing accuracy from 43.4% to 69.5%. This path makes 33 block calls, compared with 28 in the original model.
MACRO is therefore not a cost-free acceleration method. Some paths increase layer calls, KV cache slots, and latency, while the average gains are concentrated primarily in weaker, smaller models. It also requires up to 1,000 training samples and a separate validation set for each task, which may make it impractical for open-ended or continually changing workloads. The library provides configurations for six model families, reproduction commands for three random seeds, and an MIT license. The next question is whether fixed paths can transfer across similar tasks and whether they can still offset the additional computation in real-world generation, long-context, and batched-serving scenarios.