AI 安全與評測
K-Bench Expands the Observability Surface for Machine Unlearning: A Leak-Free Final Answer Does Not Mean an Agent Has Truly Forgotten
K-Bench expands evaluation beyond a model’s final response to six agent channels, including reasoning, tool calls, and retrieval results. Experiments show that even when existing benchmarks report no leakage, deployed agents may still expose target data in 22% to 86% of queries.
Machine unlearning is typically evaluated with benchmarks such as TOFU and MUSE, which inspect a model’s final answer: if the target knowledge no longer appears in the response, it is considered removed. K-Bench, added to arXiv’s latest listings on September 14, argues that this single-channel evidence cannot be directly applied to agents equipped with RAG, tools, and reasoning scratchpads.
K-Bench injects synthetic personal data into model weights, the system context, a text retrieval store, or a structured retrieval store, then simultaneously inspects six channels: chain-of-thought, tool arguments, tool results, retrieved content, the final answer, and summaries. A disclosure in any channel counts as a leak. K-Score also compares performance on a retain set and checks whether the agent has collapsed, preventing blanket refusal from being mistaken for selective unlearning.
In one Llama-3.1-8B case from the paper, a StaR filter placed an incorrect birth date in the final answer, causing answer-only evaluation to mark the attempt as successful. The real date, however, remained verbatim in the tool results. The aggregate six-channel leakage rate was 0.857, barely below the 0.855 baseline without intervention. Overall, when data was located in prompts or retrieval stores, TOFU and MUSE reported zero leakage, yet agents still exposed it in 22% to 86% of queries. When the data was stored in model weights, none of the 20 published methods tested could be shown to have genuinely removed the knowledge.
The team released a CLI, benchmark transcripts, target LoRA adapters, and two FAISS indexes of approximately 8.4 GB each, covering Llama-3.1-8B, Qwen3.5-9B, and Mistral-7B. However, some retain sets remain incomplete, and Qwen’s thinking mode can exhaust the step budget while substantially changing leakage rates. The Hugging Face dataset viewer is also currently unable to load the data because of schema inconsistencies. Engineering teams should therefore extend deletion acceptance testing to logs, tool inputs and outputs, caches, vector databases, and summaries—not just the text visible to users.