推論系統
JustFit Compresses and Dynamically Schedules KV State, Enabling a 24 GiB MacBook to Complete a 210K-Token Workload
By combining 4-bit KV, component residency management, and cross-request state transitions in MLX, JustFit enables a 27B model to perform long-context inference within 24 GiB of unified memory. The reported 6.93× capacity increase comes from a single-machine, model-specific test, and some historical environment details and checkpoint identities cannot yet be fully reproduced.