多模態檢索
Tencent Open-Sources WeMM-Embedding: 2B Model Outperforms Some 8B Baselines in Multimodal Retrieval Benchmarks
WeMM-Embedding processes text, images, video, visual documents, and interleaved inputs in a shared vector space and is available in 2B, 4B, and 9B sizes. Official results show it outperforming several public baselines, but the model does not support audio, and some product-level results are backed only by Tencent’s internal A/B tests.
Tencent’s WeChat Vision team released WeMM-Embedding on August 25, mapping text, images, video, and scanned documents—which often require separate processing—into a shared vector space. The three models are built on Qwen3.5 and use a two-stage training process consisting of large-scale multimodal alignment and refinement. The second stage incorporates curated data, fine-grained relevance supervision, and cross-size knowledge transfer. Embeddings are extracted from the final-layer state of a dedicated `<embedding>` token and then L2-normalized.
Another practical feature is Matryoshka representation. The full 9B model outputs 4,096-dimensional vectors, but deployers can truncate them to specified dimensions such as 64, 128, or 256 and renormalize them to build smaller vector indexes. According to the official report, the 2B model averages 77.9 across MMEB-v2’s 78 datasets, narrowly exceeding Qwen3-VL-Embedding 8B’s 77.8, while the 9B model reaches 80.6. At 256 dimensions, the 2B model retains 98.7% of its full-dimensional performance on image and video tasks, indicating that storage requirements and retrieval quality can be balanced within the same model.
Code is available for Transformers, SentenceTransformers, vLLM, and SGLang. To reproduce the results, the official recommendation pins `transformers==5.2.0`, while vLLM 0.27.0 and SGLang 0.5.9 are the only versions listed as tested. This is valuable for mixed-media RAG, product search, and document agents because engineering teams do not need to maintain multiple incompatible embedding spaces.
However, the 80.6 score comes from public benchmarks run by the authors and still awaits independent reproduction. The 14 online A/B tests described in the paper and WeChat’s internal 26-task benchmark have not been made public either. MMEB-v3 assigns a score of zero to unsupported audio tasks, and the model-loading example uses `trust_remote_code=True`. Before production deployment, teams should pin the revision, audit the remote code, and verify licensing requirements item by item.