GitHub Repo
Transformers Proposes Native Integration for D1 Omni, Adding Multimodal Decision and Fine-Tuning Interfaces
On November 6, the Transformers community proposed a draft for native D1 Omni support, bringing text, image, and audio decisions into unified loading and training interfaces. The proposal has not been merged. Its authors report that CPU weight checks and output comparisons passed; CUDA validation remains incomplete.
Transformers community contributors submitted a draft PR on November 6 proposing native support for Liquid AI’s d1-omni-600M through AutoModelForDecision, AutoProcessor, and a decision pipeline. The change would let existing checkpoints load with their vision and audio branches without enabling trust_remote_code, and connect them to the standard training interface. The proposal is still under development. Integration proposal
D1 Omni was announced in October; the news here is its proposed framework integration. It uses a bidirectional encoder backbone and accepts text paired with an image or audio. In a single forward pass, it directly produces a distribution over options, returning yes/no, choice, and scoring results without text generation or answer parsing. This interface suits fixed decision tasks such as agent routing, intent classification, and content review. Official architecture overview
Deployment constraints also matter: the model has approximately 587 million parameters, and a single request cannot include both image and audio. Audio is limited to 30 seconds, and the training tasks primarily cover English user-assistant interactions. The model card does not report inference speed, so Chinese speech performance and real-world latency cannot be inferred directly. Model card
The proposal also adds a cross-entropy training loss, with support for ignored labels, option padding, and Trainer updates. The authors state that this is a training implementation for the local model interface; they do not claim to reproduce Liquid AI’s unpublished original training objective. Calibration is applied only during text inference; training uses uncalibrated logits. The authors report that the official weights load on CPU with no missing or unexpected tensors, and that the maximum probability difference from the publisher’s output is approximately 4.53×10⁻⁶. CUDA validation is not yet complete. Implementation and validation notes
For engineering teams, completing the native integration could make it easier to reuse existing data processing and fine-tuning workflows. Next, teams should track whether the proposal and its dependent decision interface are merged, GPU numerical consistency, and classification quality and probability calibration on Chinese data. No official release currently delivers this support.