Fizgig 5.0 Uses Rotating Training Windows to Fully Fine-Tune a 33B Video Model on a 16GB GPU

Fizgig 5.0 expands the original LoRA workbench into a full-parameter fine-tuner capable of training the 33B-parameter MiniMax H3 video model and the 12.9B-parameter Krea 2 on a single consumer-grade NVIDIA GPU. Here, “full-parameter” does not mean allocating gradients for the entire model at once: the system activates only one segment of weights per epoch and rotates the training window, while the remaining blocks stay frozen on the GPU in 4-bit NF4. Gradients and optimizer states likewise exist only for the current slice. Typically, four epochs are equivalent to one complete update cycle across the full model.

To prevent the output weights from permanently inheriting quantization error, Fizgig keeps a bf16 master copy in system memory and writes the checkpoint directly from that copy after training. On a 16GB GPU, the maintainer measured peak VRAM usage of 8.8–12.3GB for H3 and 8.4–11.0GB for Krea 2. Training runs for H3 using 2.3-second, 56-frame videos were completed at each VRAM tier. Built-in tools can also compare checkpoints from before and after fine-tuning and extract a standard LoRA for loading in ComfyUI.

This sliced approach changes the VRAM requirement from “holding the model, gradients, and optimizer states simultaneously” to “holding a single active window,” but it does not eliminate the total compute, PCIe transfer, or storage requirements. In practice, Krea 2 is recommended to have at least 48GB of system memory. When system memory is insufficient for H3, the master copy spills to disk, potentially slowing training significantly. AMD/ROCm has not yet been tested, some figures for longer videos are still estimates, and the claim that a rank-64 LoRA is “perceptually indistinguishable” from a full checkpoint is based only on the author’s testing. The next step is to see whether the community can reproduce the convergence quality, measure the duration of each complete update cycle, and compare its real-world cost with QLoRA and distributed full-parameter fine-tuning.