推論系統
PaDoc Parses Documents in Parallel with Layout Branches, Boosting Throughput by Up to 118% on a Single A800
PaDoc decodes the layout sequence and each region’s content in parallel while sharing the page prefix, avoiding serialization of the entire document into a single path. Its 2.1B model maintains a 94.24 Overall score on OmniDocBench, though the performance figures come from a single GPU under specific scheduling settings.
Document understanding models typically output layout coordinates, text, formulas, and tables sequentially. Even when different regions are independent, later content must still wait for the preceding tokens to be completed. Two-stage OCR can recognize cropped regions in parallel, but at the cost of processing the image repeatedly and losing full-page context.
PaDoc treats the predicted layout as a generation tree: the main sequence continues generating the position of the next region, while a specific fork token simultaneously starts a content branch for that region. All branches share the full page and their ancestral layout prefix, but cannot see sibling branches. During training, variable-length FlashAttention batching implements ancestor attention while retaining the standard next-token objective. On the vLLM serving side, branches are handled as independent requests, with prefix caching used to reuse the KV cache; no additional draft model or prediction head is required.
The team initialized PaDoc from Qwen3-VL-2B, conducted continued pretraining on approximately 11 million document samples, and then performed SFT on about 500,000 filtered samples using 128 A800 GPUs. On the 1,651-page OmniDocBench v1.6 test set, PaDoc achieved a layout F1 score of 91.1 and an end-to-end Overall score of 94.24. Compared with a serialized baseline of the same 2.1B size, branching caused no significant quality degradation.
Performance tests used a fixed workload of 384 pages, a single A800, and between 16 and 256 concurrent requests. PaDoc increased effective page throughput by 67.4% to 118%, reduced P95 latency by 39.2% to 54.9%, and reached 1.722 pages per second at a concurrency of 64. The key engineering insight is to extract parallelism from the output structure: the same approach could potentially be applied to forms, charts, and other multimodal tasks containing independent regions.
However, branch independence relies on the assumption that region contents are mutually independent once conditioned on the image and layout; cross-column references or tables spanning multiple pages could violate that assumption. The repository also does not provide the authors’ training data or trained checkpoints, and cross-model speed comparisons are affected by model size, vLLM scheduling, and the definition of effective pages. The next step is to determine whether the gains can be reproduced across different GPUs, at lower concurrency levels, and under real-world mixed-document traffic.