DiDPO Breaks Code Diffs Into Credit Units, Raising Qwen2.5-Coder-7B’s Average on Key Benchmarks to 48.4%

Current RLVR methods can use compilation and test results to provide objective rewards for coding agents, but a single edit often modifies multiple functions or files at once. If the final success score is broadcast across the entire trajectory, fragments that fix an error and fragments that introduce unrelated changes may receive the same signal. [DiDPO](https://arxiv.org/abs/2608.07147) therefore incorporates the code changes themselves into credit assignment, rather than grouping solely by episode or environment state.

The method first divides the diff produced by each edit into sub-diffs, then searches across multiple rollouts for the same problem to find similar fragments that can serve as anchors. The authors’ groupability score balances a fragment’s semantic scope against the number of samples that can be aggregated. Aligned sub-diffs are organized into local comparison groups, a diff-level advantage is calculated, and the result is projected back onto the tokens that generated the modification. This local signal is then combined with the outcome advantage for the full trajectory, without requiring an additional critic or more environment rollouts.

On Qwen2.5-Coder-7B, DiDPO achieves an average score of 48.4% in the paper’s main code-generation results table, compared with 42.8% for GRPO and 44.2% for the state-grouping method GiGPO. The corresponding average for Qwen3.5-4B is 58.6%, versus 53.7% for GiGPO. Training uses 32 rollouts with up to eight interaction steps each. For cold-start data, GPT-5.5 is first used to expand the dataset to approximately 7,000 problems, after which Qwen3.6-27B generates trajectories that are filtered into roughly 3,000 SFT samples.

The authors have also released [verl-code](https://github.com/xuc865/verl-code) under the Apache-2.0 license. It includes entry points for DiDPO, GRPO, GiGPO, GSPO, and DAPO, along with an SFT data link and analysis tools. However, the repository does not yet provide a trained checkpoint, and its quick-install command still contains a `<YOUR_REPO_URL>` placeholder. More importantly, the experiments primarily use APPS, HumanEval, MBPP, LiveCodeBench, and competitive-programming problems. Agent actions are restricted to add, delete, or none, so the results do not directly represent real-world repository repair of the kind evaluated by SWE-bench. Future work should examine the cost of sub-diff alignment, stability under cross-file refactoring, and whether the gains can be independently reproduced under the same compute budget.