RAG／向量檢索
txtai 9.13 Compresses ColBERT Multi-Vectors into a Single Index, but Faiss Still Needs Retuning for Large Corpora
txtai 9.13 officially integrates LEMUR, using learned fixed-dimensional vectors to approximate per-token MaxSim scores. Initial tests balance index size and retrieval quality, but the default IVF configuration may significantly reduce recall on larger datasets.
NeuML released [txtai 9.13.0](https://github.com/neuml/txtai/releases/tag/v9.13.0) on August 27, officially adding LEMUR (Learned Multi-Vector Retrieval) to the open-source search framework. ColBERT-style late-interaction models retain a vector for each token and use MaxSim to calculate fine-grained relevance between queries and documents. They generally deliver better quality than a single dense vector, but at the cost of larger indexes and limited compatibility with conventional ANN infrastructure.
LEMUR first trains a shallow feature encoder to learn MaxSim. It then represents a query as the sum of its token features and a document as least-squares weights relative to sampled tokens. The inner product of these fixed-dimensional vectors approximates the original score, allowing them to be used directly with single-vector indexes such as Faiss. Training produces `config.json`, `model.safetensors`, and normalization statistics. However, because the resulting model is tied to the corpus distribution, additional training is still required before building a new index.
In exact-search tests using a single RTX 4080 SUPER, ColBERTv2, and three BEIR datasets, the author compared 2,048-dimensional LEMUR vectors with 10,240-dimensional MUVERA vectors. LEMUR reduced index size to roughly one-fifth while still improving NDCG@10 by 8.4% to 22.9%. The release also adds configurable mean centering for token vectors to mitigate vector anisotropy in models such as LateOn.
The limitations are equally important from an engineering perspective. The [implementation report](https://huggingface.co/blog/NeuML/txtai-lemur) covers only one model, three datasets, and exact search. Once a dataset exceeds 5,000 records, txtai switches to IVF by default; as a result, LEMUR's NDCG@10 fell by 43% in the scifact test. Teams should therefore benchmark their own Chinese-language corpora, tune IVF parameters, and compare the total cost of training and index rebuilding instead of directly extrapolating from the exact-search results.