AI 安全
Malicious Text Written into System Logs Can Make LLMs Classify Real Attacks as Normal Activity
New research demonstrates that attackers can inject prompts through log fields they control, disrupting LLM analysis workflows used by security operations centers (SOCs). The study also finds that model explanations often reveal signs of manipulation, though this signal is not yet reliable enough to serve as a dependable defense.
Researchers from institutions including the Austrian Institute of Technology in Vienna released a prompt-injection evaluation framework for LLM-based log analyzers on July 27. The threat model does not require attackers to access the model or the SOC backend. They need only cause controlled text to appear in audit, access, or application logs—through fields such as account names, URLs, User-Agent strings, filenames, or command-line arguments—to smuggle instructions into the model context.
The research workflow begins with CAM-LDS log traces generated from real-world network attacks. The researchers first create generic injection text, then refine it through rewriting and optimization for specific attack scenarios. Experiments show that several frontier models may classify malicious traces as benign even when clear indicators of compromise remain visible. This differs from traditional “log injection”: the goal is not merely to corrupt the parsing format, but to alter how a downstream language model semantically interprets an event.
The authors also observed a potentially useful but still unreliable defensive signal: although a model may output a benign label, its explanation often mentions suspicious instructions, manipulation attempts, or contradictory evidence. Engineering teams could therefore validate classifications and explanations separately, using rules or another isolated detector to identify signs of manipulation. A more fundamental approach is to treat logs as untrusted data, provide them to the model using structured fields, source labels, and explicit data/instruction boundaries, and prohibit conclusions from a single LLM from directly closing alerts.
However, the paper does not establish that every commercial SOC product is affected, and the results depend on the prompt template, model version, and log-selection method. The public code repository provides materials for reproducing the findings. Key next steps include cross-version testing across different models and RAG pipelines, and determining whether explanation-based detection can consistently block attacks without substantially increasing false positives.