OpenArch Hits Hacker News With Readable LLM Architecture Implementations, but Cannot Yet Replace Production Inference Frameworks

OpenArch reached the Hacker News front page on September 14, offering a different way to read LLM source code than production libraries such as Transformers. Each model is presented as a relatively self-contained PyTorch `model.py`, deliberately stripping out distributed loading, quantization kernels, inference server integrations, and extensive backward-compatibility logic so researchers can directly compare architectural choices. The currently runnable entries cover GPT-2 XL, Llama 2/3, OLMo 2, DeepSeek R1, Gemma 3, Mistral 3, Llama 4, Qwen 3, Kimi K2, GLM-4.5, GPT-OSS, and PaliGemma; Grok 2.5, Qwen3 multimodal models, and DALL-E, among others, are still marked as works in progress.

The project organizes differences along several axes with practical implications for deployment: MHA, GQA, MLA, sliding-window attention, and linear attention; pre-norm, post-norm, RMSNorm, and QK-Norm; RoPE, NoPE, and partial RoPE; as well as dense architectures, sparse MoE, shared experts, and multi-token prediction. Its main sources are the original papers, model `config.json` files, and Sebastian Raschka’s LLM Architecture Gallery. The latter currently lists 103 models, while the OpenArch README still says its target list corresponds to 72, indicating that implementation coverage has not yet caught up with the reference catalog.

For research and education, its value lies in turning design decisions that are easily buried in the conditional branches of large frameworks into implementations that can be inspected line by line. It can also serve as a starting point for writing small forward-pass tests. However, the maintainer explicitly states that it is not competing with Transformers. The repository currently has no published package, formal release, or consistent cross-architecture weight-alignment results, and some implementations are described only as capable of performing a forward pass. Engineers should not infer training stability, numerical equivalence, throughput, or memory efficiency from these implementations. The next things to watch are whether the project adds per-token comparison tests using official weights, pins dependency versions, and provides more comprehensive validation of MLA, hybrid linear attention, and MoE communication paths.