模型訓練工具
Soup 0.74 Fixes the fp32 Loading Path for Frozen Models, Cutting Peak VRAM to 18.7 GiB in a Single LoRA Test
Soup’s text, vision, and audio loaders did not specify a dtype, causing base models excluded from optimization to remain resident in VRAM as fp32. After the fix, the project’s H100 test fell from 48,241 MiB to 18,658 MiB, but the 2.59× improvement cannot yet be generalized to all models and hardware.
Open-source model fine-tuning tool Soup has released version 0.74.0, fixing a default behavior significant enough to distort hardware capacity planning. Its three `from_pretrained` paths for text, vision, and audio previously did not explicitly pass a dtype, causing frozen base models in supervised fine-tuning to be instantiated as fp32. Even when LoRA updated only a small number of adapter parameters, the entire base model still occupied VRAM at twice the checkpoint’s bit width.
The new version loads frozen weights using the checkpoint’s original precision, while deliberately retaining fp32 for full fine-tuning. The project also consolidated the separate checks for “whether this is full fine-tuning” previously maintained by the trainer and the VRAM preflight check into a single `is_full_finetune()` function. This prevents capacity estimation from treating a model as frozen while the actual loading path makes the opposite decision.
In a test using Llama 3.1 8B with LoRA on a single H100, the maintainers reported that peak VRAM fell from 48,241 MiB to 18,658 MiB—a difference of about 28.9 GB and a 2.59× reduction. Results were byte-for-byte identical across three runs. That difference can determine whether a workload fits on a particular GPU and whether a team incorrectly concludes that parameter-efficient fine-tuning is uneconomical. However, the figures come from the project itself, cover only one model and hardware configuration, and have not yet been independently reproduced.
Version 0.74.0 also patches several execution and network boundaries. Telemetry, webhook, and OTLP validators now block alternative IPv4 representations such as `127.1`, decimal, hexadecimal, and octal forms, preventing them from bypassing address checks. The re-enabled `/v1/tools/bash` endpoint is now isolated at the operating-system level. If the service binds to a non-loopback address without a `--tool-auth-token`, it exits immediately with status code 2 instead of merely displaying a warning. The SGLang backend also no longer enables remote model code by default.
The upgrade still has a dependency pitfall: the package’s declared minimum of `torch>=2.5.0` is incompatible with `trl>=0.29` in practice. Torch 2.5.1 is known to lack the required `FSDPModule`, preventing DPO, KTO, GRPO, and BCO from loading. Adopters should record the base model’s actual dtype, frozen state, peak allocated VRAM, and reserved VRAM, then rerun capacity tests using their own model, sequence length, and GPU. Successful installation by a package resolver does not mean that the resulting version combination has been tested.