最新模型
BPCO Trains an LLM Critic With a Single Rollout, Replacing GRPO’s Within-Group Comparison Across Multiple Responses
BPCO combines a constrained value head, Monte Carlo targets, and length-adaptive GAE, enabling a critic to estimate per-token advantages from a single response. The authors match or outperform multi-sample methods on models ranging from 1.5B to 30B-A3B, but have not yet shown that total training costs are lower.