SAILS Shows Backdoor Success Depends on the Poisoned Sample Set, While Random Sampling May Underestimate Worst-Case Risk

*Pick Your Poison*, released on September 14, challenges a common evaluation method in backdoor research: after fixing the number of poisoned samples, researchers randomly draw several sets from a candidate pool and use the mean attack success rate to represent model risk. Across three instruction-backdoor settings involving Llama 3 8B, the research team found that, with the model, clean data, and poison count held exactly constant, changing only which samples were selected caused the attack success rate to range from 3% to 80%. In other words, random sampling may measure only the average case rather than the worst case achievable by an attacker with search capabilities.

The study introduces SAILS, framing the problem as set optimization under a limited oracle budget. The system first runs a small number of full “fine-tune–trigger test” evaluations on randomly selected poisoned sets, obtaining costly but direct labels. It then trains a lightweight scoring model using the raw text within each set. In each round, the scorer cheaply ranks hundreds of thousands to millions of candidate sets, sends only a small batch of the highest-ranked sets back to the full fine-tuning oracle for validation, and uses the new results to recalibrate the scorer. This propose–score–audit loop can capture interactions that emerge when multiple samples occur together, unlike methods that score individual samples using gradients or influence scores and then simply select the top k.

The paper reports that, on sets not used for training, SAILS improves the attack success rate by an average of 30 percentage points over the strongest influence-based baseline and transfers from small-scale search to full fine-tuning settings. The authors also extend the method to natural-language-to-shell-command generation with Qwen3-4B, the multi-turn WebShop agent environment, and scenarios in which candidate data is generated through hosted APIs. The public repository includes LoRA fine-tuning, held-out ASR evaluation, baselines such as TRAK and SGD, and reproducible data and configurations.

This does not prove that every fine-tuning dataset can be compromised with a small number of samples. The results focus on specified triggers, controlled candidate pools, and a limited number of models. The set scorer itself also depends on hundreds of fine-tuning evaluations, with each oracle run in the full experiments taking roughly 30 to 60 minutes on an H100. As part of a responsible release, the authors did not provide preassembled poisoned sets or backdoored model weights. The more immediate implication for defenders is that data audits and red-team evaluations should report distributions across sets and extreme values found after search, rather than relying solely on mean results from randomly sampled poisoning sets.