最新模型
CompactifAI Compresses Llama 3.3 70B to About 65 GiB, Runs It on a Single Xeon 6 with vLLM CPU
Multiverse Computing has unveiled a compressed and healing-trained version of Llama 3.3 70B that can be served on Xeon 6 using vLLM CPU and Intel AMX. Official tests show nearly double the throughput, but output remains below 4 tokens per second, and neither the checkpoint nor the raw results have been released.