評測與關聯式學習
RelArena-α Uniformly Reruns 21 Relational Data Tasks, With TabPFN-Rel Leading Among Models Using Standardized Tuning
Prior Labs has open-sourced a unified workflow for data splitting, hyperparameter tuning, and evaluation, enabling a fresh comparison of graph models, relational Transformers, and tabular methods. TabPFN-Rel shows that aggregating relational data before passing it to a tabular foundation model remains competitive, although the CPU cost of feature synthesis is not fully included in the leaderboard’s timing results.
Relational data learning has long lacked a reproducible common playing field: different papers use different temporal splits, database snapshots, and hyperparameter-tuning budgets, and some do not even release training code. Prior Labs’ [RelArena-α paper](https://arxiv.org/abs/2608.16319) and [Apache 2.0-licensed library](https://github.com/PriorLabs/relarena) place RelBench v1’s seven databases and 21 entity-level prediction tasks within a unified workflow for data loading, nested temporal validation, hyperparameter tuning, and Bradley–Terry Elo scoring.
The framework deliberately distinguishes between “models” and “systems.” A model declares only its search space, while RelArena controls the trials, validation-set selection, and final retraining. A system may bring its own sequential or conditional tuning procedure, so only end-to-end results can be compared; performance differences cannot be attributed entirely to model architecture. This distinction explains the leaderboard: TabPFN-Rel ranks first among models using standardized tuning, but RT-PluRel, which uses custom tuning, takes the top spot on the combined leaderboard.
TabPFN-Rel does not train directly on the entire relational graph. It performs deep feature synthesis along primary-key and foreign-key paths, aggregating related tables into a single feature table, which TabPFN-3 then uses to make predictions with labeled rows as context. The new version fixes a mismatch between the database timestamps used during tuning and testing, expands the available context by roughly an order of magnitude, and selects examples based on recency and diversity. The API version can also retain text columns and achieves an Elo rating of 1821. The open-source, text-free version, which does not send data to a hosted service, scores 1706; most of the gap is concentrated in two tasks with strong textual signals.
The results also challenge the intuition that relational data necessarily requires specialized graph architectures. TabPFN-Rel and RT-PluRel are the only methods that outperform a per-entity constant predictor across all 21 tasks. That zero-feature baseline even beats RelGNN and RelGT on four tasks each. This does not mean that flattening-based methods are universally superior: the number of feature-synthesis join paths grows rapidly with depth, and producing the full single-seed leaderboard already requires hundreds of hours.
The current release is explicitly labeled a research-use alpha and covers only entity-level prediction. Its timing table also excludes CPU preprocessing, whose cost varies substantially between methods; for TabPFN-Rel on large databases, this step may dominate runtime. The next things to watch are multi-seed rankings, end-to-end resource accounting, and whether third-party submissions can reproduce the current ordering under the same temporal boundaries.