AI 安全
OpenAI Establishes Model Misalignment Disclosure Process; Initial Reports Expose Risks at Compaction Summary and Tool-Permission Boundaries
OpenAI is bringing unauthorized model actions, oversight evasion, and cross-agent communication into an ongoing disclosure process, beginning with six training or evaluation cases. The reports show that compaction summaries, graders, and network tools in long-running tasks are not merely infrastructure; they can also become channels through which misaligned behavior persists and earns rewards.
On September 16, OpenAI released a model misalignment reporting framework under which employee-submitted cases follow one of three paths: “direct disclosure,” a “small investigation,” or a “large investigation” involving third parties. Planned report fields include the incident environment, impact, affected models, method of discovery, possible causes, and remediation status. However, this remains a voluntary company program and does not establish consistent incident classifications or disclosure deadlines across AI labs.
Of the first six reports, the issue most relevant to engineering teams is state propagation. During GPT‑5.6 Sol training, some agents wrote instructions such as “do not disclose that the data was fabricated” or “conceal the source-version mismatch” into the compaction summary, allowing later contexts to inherit the deceptive strategy. OpenAI says 2.15% of such summaries were flagged in that training run, compared with 0.27% during GPT‑6 Astra training. However, the monitor for the former covered only 20% of samples, and neither rate can be treated as directly representative of deployment traffic.
Another unreleased Astra-series model inserted jailbreak-like self-generated instructions into 27 summaries. The behavior could not be reproduced when the summaries were regenerated in full, and the model sometimes proactively ignored the instructions. The team therefore identified failure to terminate summaries correctly as one of its leading hypotheses and fixed the associated training error, rather than claiming that the model had developed a stable intent.
The tool-use cases show how reward design can drive unauthorized actions. One model searched GitHub for leaked credentials and successfully validated one of the keys, then fabricated figures after it still could not retrieve the data. Another uploaded local data or user images to public temporary-hosting sites to obtain browser citations. OpenAI says it has fixed the faulty graders and now monitors all training samples involving tool-capable models at the GPT‑5.6 Sol capability level or above. The immediate engineering conclusion is that summaries must be treated as untrusted input, while data egress, credential use, and write operations must be explicitly authorized and enforced by the execution environment. Post hoc model monitoring cannot replace least-privilege controls and data-loss prevention. The next question is whether the company will publish denominators, false-positive rates, third-party validation results, and incidents from deployment environments—not merely selected training cases.