Monroe Pretrains a Graph Model on 81 Million Molecules; Frozen Embeddings with TabPFN Outperform Per-Task Fine-Tuning

The newly released molecular foundation model [Monroe](https://arxiv.org/abs/2608.18982) does not fine-tune its encoder for each drug-property task. Instead, it converts each molecule into a fixed 720-dimensional vector, then passes the labeled training vectors and query molecules together to TabPFNv3. TabPFN approximates the posterior predictive distribution in a single forward pass, eliminating the need for per-task gradient updates or hyperparameter searches. This is especially useful for ADMET and activity prediction, where labels come from expensive wet-lab experiments and datasets are typically small.

The encoder uses the GRIT graph Transformer as its backbone and computes sparse attention only over observed edges, reducing complexity from the O(|V|²) cost of full connectivity to O(|E|). It also adds a virtual node for global information exchange. Because using only rotation- and reflection-invariant features would give enantiomers identical inputs, the researchers additionally introduce directed auxiliary edges encoding R/S and E/Z stereochemistry. The training data includes quantum-chemical properties for approximately 81 million PM6 molecules and bioassay labels for 1.56 million PCBA molecules, yielding 1,152 property-prediction, classification, and conformer-denoising objectives. Full pretraining took 26 hours on four H100 GPUs.

Across 28 Polaris tasks, Monroe achieved either the best mean score or a result statistically indistinguishable from the best method on every task, with an average rank of 3.71. On 30 MoleculeACE activity-cliff tasks, it recorded a statistical win rate of 86.7%, with RMSE values of 0.629 overall and 0.737 on the activity-cliff subset. The study used Tukey’s HSD test with Benjamini–Hochberg correction for multiple comparisons and removed pretraining molecules overlapping with downstream test sets based on InChIKey. The [code and weights](https://github.com/blazejba/monroe) are publicly available, but the checkpoint outputs only molecular fingerprints; users must configure TabPFN separately to obtain task predictions.

These results should not be interpreted as having solved drug discovery. Some Polaris datasets have issues involving salt handling, incorrect structures, and stereochemical ambiguity, while errors on activity cliffs remain substantially higher than on ordinary samples. Key questions include whether independent teams can reproduce the four-H100 training run, whether Monroe retains its advantage under genuine temporal splits or on novel chemical scaffolds, and how TabPFN inference costs scale as the labeled support set grows.