Back Home

AI 程式代理研究

36,710 GitHub Projects Yield Just 85 Agent Plans, Most Not Yet Maintained as Long-Term Documents

A new study distinguishes persistent AGENTS.md files from task-specific Agent Plans and examines how plans enter public version histories. The sample is heavily concentrated in a handful of projects, suggesting that treating plans as collaborative artifacts remains an early and unstable practice.

Man Alive! · CC BY 2.0 · Image source
zh-Hant

A new study accepted at ESEM 2026 is the first to examine at scale whether task plans generated by coding agents are preserved in open-source projects. Researchers scanned 36,710 GitHub projects exhibiting signs of engineering activity and found only 85 Markdown plans across 10 projects. Of these, 72 were located in `.cursor/plans` and 13 in `.claude/plans`; the `.gemini/plans` directory included in the default search produced no matches.

Agent Plans differ from persistent instruction files such as AGENTS.md and CLAUDE.md. The latter describe build commands, architecture, and coding conventions, while the former correspond to a specific task and document the intended steps, the locations to modify, and the completion criteria. After analyzing 592 first- and second-level headings, the researchers found that the most common content categories were implementation steps, files and locations, and testing and validation, appearing in 54, 46, and 40 plans, respectively. Refactoring and migration were the most common task types, followed by testing, feature development, and CI/CD.

Public preservation has not yet become a widespread convention. Salesforce’s `salesforcedx-vscode` project alone contributed 66 plans, accounting for nearly 80% of the entire sample. Excluding it leaves only 19 plans and substantially changes the task distribution. Only seven of the 85 plans were revised across multiple commits. Among the 72 commits for which complete metadata was available, 31 explicitly identified Claude or Cursor as a co-author, although the absence of such attribution does not prove that a document was written independently by a human.

The significance of these findings for agent governance is not that they demonstrate widespread adoption of planning workflows, but that they reveal a versionable intermediate layer. Teams can review scope, constraints, and validation methods before an agent modifies code, then compare the plan with the actual diff afterward. However, the search covered only three tool-specific directories and default branches, so it would have missed plans stored in `docs`, issues, pull requests, or uncommitted local files. The next step is to measure whether preserving plans actually reduces rework and defects, rather than merely adding another document that quickly becomes outdated.

Sources

  1. An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software
  2. Introducing Plan Mode
  3. salesforcedx-vscode:研究樣本中計畫檔最集中的專案