程式代理與評測
Change2Task Ports Historical PRs to Modern Code, With 900 Agent Tasks Passing Three-Stage Validation
Change2Task extracts development intent from merged PRs, then reconstructs executable tasks on a modern version of the same repository. The study generated 900 task sets from 1,130 candidate changes, although most cases still required another coding agent to complete the reconstruction.
A Microsoft research team has introduced Change2Task, an attempt to address the “expensive environments, rapidly aging tasks” problem in coding-agent training and evaluation. Conventional SWE-bench-style datasets typically pin issues to historical commits. Change2Task instead obtains descriptions, patches, and test evidence from merged PRs, then ports the same maintenance requirement to a newer version that builds successfully, allowing a single Docker/dependency environment to support multiple tasks.
The system applies three methods in sequence. It first directly reverses the original patch. If the modern code has been refactored, it maps historical code regions to their modern counterparts. If correspondence still cannot be established, it asks Claude Code to reconstruct the task using evidence from the PR and failure feedback. Every case must pass three-state “healthy baseline–faulty task–restored fix” validation: target tests must pass, fail, and then pass again, while regression tests must remain successful throughout. Distorted cases are further filtered using the scope of changes and structural similarity to the original PR.
The study covers five categories of work: debugging, feature additions, test generation, API migration, and security patches. Of 1,130 eligible source changes, 900 completed validation, for an overall success rate of 79.6%. On the same set of 621 debugging candidates, Change2Task produced 500 tasks, compared with 387 for SWE-smith PR Mirror—a relative increase of 29.2%. After sharing 388 modern baselines, environment build time and storage use fell by 58.4% and 71.2%, respectively. However, estimated spending for the complete pipeline declined by only 10.8%, indicating that agent-based reconstruction and validation remain the primary costs.
Engineering teams should pay attention to data contamination and evaluation independence: 68.3% of the 900 tasks required agent-based reconstruction, while semantic review used other large models. The paper does not yet provide links to a public implementation or dataset, so outside researchers currently cannot verify build failures, model-judge bias, or cost estimates. It is better understood as a viable data-factory design than as a benchmark package ready for direct deployment.