AI 安全與私隱
Normal Responses From Eight Closed-Source Models Can Conceal Context Secrets, With 82% Exact Recovery for Four-Digit Secrets
New research shows that even when models refuse to disclose secrets directly, statistical features such as response length, formatting, and word choice may still carry contextual information. An adaptive black-box attack can recover nearly all two-digit secrets, while achieving an 82% exact-match rate for four-digit secrets.
AI agents often place calendars, medical records, credentials, or long-term memory in a model’s context. Existing safeguards generally check only whether the output directly repeats sensitive content. A [new preprint](https://arxiv.org/abs/2608.19857) finds that although all eight tested closed-source models correctly refused requests that directly asked for secrets, their normal responses still exhibited measurable differences in tokens, length, punctuation, formatting, and word choice depending on the secret.
The researchers formalized the problem as predicate inference: an attacker needs only black-box access to the target API to first estimate how different secrets affect the output distribution, then update posterior probabilities one position at a time. On Claude Opus 4.6, adaptive prompting increased the four-digit recovery rate from 44% with a static method to 82%; nearly all two-digit secrets were recovered. However, performance varied substantially across models, with at least two approaching random guessing once secrets reached four digits. In another set of experiments, a small Qwen decoder analyzed ordinary text and achieved an inference advantage of 0.319 for hidden attributes such as health or financial memories. An active attack could even cause an agent to encode a complete U.S. Social Security number through the number of exclamation marks in its responses.
This means that “the model did not reveal the secret” does not imply zero information leakage. Output filters and refusal tests also cannot cover statistical covert channels. Engineering teams should shorten the lifetime of sensitive context, isolate tool credentials from generative models, and include attribute inference across repeated queries in red-team testing. A [UC Berkeley technical report](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2026/EECS-2026-126.html) likewise describes the phenomenon as an inherent risk that may grow with model capabilities. However, the findings still come from controlled experiments, depend on knowledge of the model and an upfront estimation cost, and do not imply that arbitrary production data can be recovered at the same rates.