多模態 RAG
DualG-MRAG Splits a Multimodal Knowledge Graph into Two Layers, Raising MMQA R@5 to 61.9%
DualG-MRAG uses a Macro Graph to find cross-document reasoning paths, then a Micro Graph to verify local image and text evidence, preventing fine-grained nodes from rapidly inflating the knowledge graph. In MMQA, the study raises R@5 from 42.1% for the strongest graph-based baseline to 61.9%, but the code and index construction costs have not yet been disclosed.
Multimodal RAG systems often face a trade-off between two representations: creating nodes only for entire documents can lose local evidence such as image regions and table fields, while packing every detail into a single knowledge graph causes the number of nodes and edges to surge, introducing substantial noise into retrieval. DualG-MRAG, submitted on July 30, separates these two tasks. Its Macro Graph stores documents, entities, and cross-modal relationships and handles global topology routing, while its Micro Graph preserves fine-grained content and performs local matching only within the candidate scope.
After a query enters the system, the GNN Retriever updates the relevance of different pieces of evidence through message passing. The system then unfolds the GNN forward pass into a directed acyclic graph and uses dynamic programming to extract high-probability evidence paths, which are passed to Qwen3-VL-4B or Qwen3-VL-8B for answer generation, rather than merely concatenating independent top-k chunks. Micro-level retrieval first uses a vector index to obtain top-K anchors, then applies heuristic branch-and-bound within a constrained subgraph. The paper estimates that query graphs typically contain fewer than five nodes, although this analysis still depends on the data structures and parameters used.
On the MMQA benchmark, which includes text, images, and tables, DualG-MRAG achieves R@2/R@5 of 49.4%/61.9%, a substantial improvement over MMGraphRAG's 31.8%/42.1%. Its R@5 on WebQA is 58.2%. Average query latency is approximately 0.44 seconds, far below the roughly 40.5 seconds the authors measured for MMGraphRAG, though still slower than VLM2Vec-V2.0 at around 0.09 seconds. Removing the Macro Graph reduces MMQA R@5 to 21.8%, indicating that the main gains indeed come from global routing.
From an engineering perspective, key areas to watch include offline graph construction time, index capacity, incremental updates, and whether the latency results can be reproduced across different hardware and large enterprise corpora. The experiments sample only 1,000 questions each from MMQA and WebQA, and neither the code nor the complete indexes have been released. In addition, explicit paths do not deliver consistent gains in MMQA exact match (EM) for the 8B model, suggesting that stronger models may instead be constrained by fixed paths.