AI 供應鏈安全
SkillGate Pre-Filters Agent Skills with Rules, Cutting LLM Input by 77% but Still Missing Nearly a Quarter of Attacks
The open-source SkillGate uses 530 rules to identify suspicious snippets before agents load skills, then sends them to an LLM for classification and isolation. Official benchmarks measured a 1.13% false-positive rate and 76.9% recall, showing that a low-cost gateway is feasible but cannot yet replace sandboxing and least privilege.
The research team released SkillGate on July 28, aiming to add a pre-execution inspection layer for Coding Agent `SKILL.md` files, MCP responses, and skill installation workflows. This type of Markdown does not necessarily contain traditional malware, but it can use instructions trusted by an agent to request credential access, execute commands, or exfiltrate data to external endpoints. As a result, scanners designed for source code and package binaries may not detect it.
SkillGate uses a two-stage architecture. The first stage applies 530 rules to search for signals such as remote execution, credential access, prompt injection, path traversal, and network activity. Files with no matches are allowed through without invoking a model. When a rule matches, the system extracts only the surrounding context—500 characters per window by default, with a maximum of 20 windows—and sends it to an LLM for classification as SAFE, SUSPICIOUS, or MALICIOUS. A policy engine then decides whether to allow, warn, quarantine, or block the content. The project provides an MCP proxy service, a skill installation interceptor, and configuration entry points for Cursor, Claude Code, Copilot, Gemini CLI, and Windsurf.
On SkillsBench, which contains 1,650 skills including 150 malicious samples, the paper reports an F1 score of 0.817, recall of 0.769, a false-positive rate of 1.13%, and an AUPRC of 0.830. In total, 67.2% of files bypassed the LLM entirely, while only suspicious snippets from the remainder were submitted, reducing input tokens from roughly 3.02 million to 699,000—a 76.9% decrease. Weighted average latency was approximately 818 milliseconds. The point of this design is not to let regular expressions determine guilt directly, but to use them to narrow the scope of semantic analysis.
The limitations are also clear: 76.9% recall means that nearly a quarter of malicious samples may still pass through. Treating zero rule matches as safe also creates blind spots for novel, rewritten, or cross-file attacks. The malicious cases in the benchmark were manually designed, LLM classification is nondeterministic, and the current results come primarily from the authors’ own testing. Deployers should treat SkillGate as a supply-chain filtering layer and continue to use signatures and source pinning, network egress restrictions, secret isolation, per-tool-call approval, and execution sandboxes.