Transformers Proposes Native Cellpose Integration with SAM and DINOv3 Cell Segmentation Models

Hugging Face Transformers contributor ArthurZucker proposed a native Cellpose integration on October 15, covering cell image segmentation models based on SAM and DINOv3. The proposal loads original checkpoints through weight conversion mappings and adds PyTorch post-processing with flow error quality checks. The author says that masks produced by the SAM path are identical to those from the original implementation; this is a validation claim by the proposal’s author and should not yet be treated as a broad guarantee across datasets. Integration proposal

The technical focus of the integration is preserving the processing chain from model outputs to cell instance masks. According to the official Cellpose documentation, the network predicts horizontal and vertical flows as well as cell probabilities, then simulates pixel movement along the flows and groups pixels that converge at the same locations into candidate regions. As a result, even with identical backbone weights, differences in post-processing can change cell boundaries and counts. Official settings documentation

The flow error check included in the proposal recomputes flows from candidate masks, compares them with the network’s predictions using mean squared error, and excludes regions whose shapes are inconsistent with their flows. The official default threshold is 0.4; raising it can retain more candidate regions, while lowering it filters out irregular shapes more strictly. Research workflows that compare outputs from different models should record this setting alongside weights and preprocessing to make result differences interpretable. Flow threshold and processing pipeline

If the integration is completed, engineering teams could load both model types within the Transformers ecosystem, reducing the work needed to connect a separate model interface. This is an engineering inference from the integration; the public proposal has not demonstrated improvements in throughput or memory usage. Existing image workflows will also need to check input semantics: the official Cellpose-SAM documentation says that the model uses only the first three channels and truncates any additional channels. Multistain images still require channel selection or merging beforehand. Input channel rules

This is still a proposal under review, so it should not be treated as a capability already available in an official release. Points to watch include numerical alignment with the original checkpoints, consistency of post-processing across devices, and instance segmentation evaluations on representative microscopy images. Teams preparing to try it should first fix the images and thresholds, preserve the original outputs, and then compare masks and cell counts. Proposal status