Gensyn open-1b Builds a Cross-Hardware, Bit-Level Audit Trail for 80,957 Training Steps

Gensyn has released open-1b under the Apache 2.0 license. Rather than chasing the large-model leaderboards, the project advances the idea of a “reproducible recipe” into a spot-checkable training record. The model has 1.61 billion parameters, including 1.08 billion non-embedding parameters. The base model was trained for 80,957 steps on 400 billion tokens using 48 H100 GPUs. Gensyn also provides a version that underwent continued training on an additional 93 billion tokens, as well as an SFT model. The official release includes full checkpoints every 100 steps, corpus shards, configurations, evaluation code, and per-step state hashes.

At its core are RepOps and a fixed data pipeline: matrix multiplication, normalization, gradient reduction, and random-number generation all use a fixed operation order, while data windows remain unchanged regardless of world size or rank. At every step, the hash from the previous step, the data-batch digest, weights, gradients, and optimizer state are combined into a new digest. Auditors can replay selected steps on CPUs, CUDA hardware, or Apple Silicon and check whether they obtain bit-identical results. Compared with releasing only weights and a training recipe, this adds a way to verify whether the recipe can actually produce those weights. It could support research into data-contamination tracing, model supply-chain attestation, and provenance governance for oversight models.

The costs are substantial and concrete. Gensyn says the reproducible runtime achieves roughly 5% model FLOPs utilization and is about five times slower than optimized PyTorch on the same hardware with the same recipe. Full pretraining took 27.8 compute-days. Model capability is also not the main selling point: after mid-training, it averaged 26.8 on the OLMo 2 evaluation suite, below the 43.7 achieved by OLMo 2 1B. More importantly, a successful one-step replay proves only that the specified inputs and program produce the expected digest; by itself, it cannot prove that the historical cluster actually operated that way. The RepOps source code has not yet been released, and the training-loss logs have been deliberately withheld. The next things to watch are the coverage achieved by independent audits, replay results across GPU generations, and whether the cost of fixed-order reductions can scale to larger models and tensor parallelism.