模型編譯與本機推論
Compile by Training Compiles Natural-Language Specifications into Locally Executable Functions on a 0.6B Model
The new method uses a large teacher model to synthesize task data, then trains a function-specific LoRA for each task so subsequent requests do not need to call a remote model. It achieves 83.6% semantic accuracy on a difficult benchmark, but compilation still requires a teacher API, about 40 GB of accelerator memory, and output validation.
Large models are well suited to text-processing tasks that are easy to describe but difficult to express as rules. Calling an API for every request, however, accumulates latency, costs, and vendor dependency. Researchers from the University of Waterloo and Harvard have introduced [Compile by Training](https://arxiv.org/abs/2609.04199), which turns model adaptation into a software build-like step: developers first define the inputs, outputs, and constraints in natural language; a teacher model then synthesizes structured examples from the specification; after validating the data, the compiler uses gradient descent to fine-tune a task-specific LoRA.
All functions share a frozen, quantized Qwen3-0.6B interpreter. In the published configuration, an existing Program-as-Weights compiler first generates the initial weights and execution prompt, after which a LoRA with rank 64 and alpha 16 is trained for 100 steps. The resulting artifact is packaged as a `.paw` file containing the adapter, prompt scaffold, original specification, and interpreter metadata. The interpreter only needs to be downloaded once, after which new inputs can be processed locally, while functions can be cached, versioned, and composed. In the team’s website assistant demonstration, 30 compiled functions handle fuzzy classification and answer selection, while BM25 retrieval and flow control remain in deterministic code.
On FuzzyBench-Hard—a specially selected benchmark containing cases the previous fast compiler could not match precisely—the new method raises LLM Exact Match from 22.4% to 83.6%. The tradeoff is that build time on a B300 increases from 3.5 seconds to 50.9 seconds; measured times on an H200 and RTX hardware are 68.2 and 99.2 seconds, respectively. Semantic correctness for this metric is judged by GPT-5.5. Although it achieves 97.7% agreement accuracy against 128 author-labeled examples, this is still not formal verification.
The [open-source implementation](https://github.com/programasweights/compile-by-training) is released under the MIT License, but its core is currently only a streamlined set of compilation scripts. The default workflow requires an OpenAI API key and about 40 GB of available accelerator memory. “Local” applies only to inference after compilation; the specification and synthesis stages still interact with an external teacher, and the adapter inherits edge cases missed by that teacher. Engineering teams should next measure performance on Chinese-language tasks, out-of-distribution inputs, and behavioral drift after specification changes, while retaining deterministic validation gates for safety- or compliance-sensitive workflows.