推論系統
PaDoc Parses Documents in Parallel with Layout Branches, Boosting Throughput by Up to 118% on a Single A800
PaDoc enables the layout sequence and each region’s content to share a page-level KV prefix and decode in parallel, instead of compressing the entire page into one long sequence. On OmniDocBench, the authors report quality close to leading systems and a 39.2% to 54.9% reduction in P95 latency versus a sequential baseline using the same backbone.
End-to-end document models typically read a full-page image and then sequentially generate block coordinates, text, tables, and formulas. Even when two regions are independent, an autoregressive decoder must finish the content of the first region before processing the next. An alternative two-stage approach—cropping first and then recognizing each crop—can run in parallel, but it must repeat the visual prefill for every crop and may lose some full-page context.
PaDoc treats the predicted layout as a branching structure. The main sequence continues generating the next layout region; whenever it encounters a learned fork token, it launches a content branch from the same image and layout prefix. Each branch retains the complete page image but cannot see content generated by sibling branches. During training, packed variable-length ancestor attention implements this visibility pattern while retaining the standard next-token objective, without requiring an additional detection head or draft model. At inference time, PaDoc copies the KV cache at the fork point so that the main sequence and content branches can advance in parallel.
The researchers started with Qwen3-VL-2B and performed continual pretraining and supervised fine-tuning. On OmniDocBench Full, PaDoc achieved a layout Overall F1 of 91.1 and an end-to-end Overall score of 94.24, with Text Edit at 0.038 and Formula CDM at 95.59. Efficiency tests used a 384-page subset and a single A800. Compared with Sequential SFT using the same backbone, effective page throughput increased by 67.4% to 118% across five parallelism settings, while P95 latency fell by 39.2% to 54.9%. The key is not a reduction in the total number of output tokens, but shortening the critical path from the sum of all region-content lengths to the longest layout–content branch.
The open-source implementation provides two paths: Transformers and vLLM. The former advances branches synchronously within the same GPU batch, while the latter submits branches as independent requests and uses prefix caching. In production, branch count, total generated tokens, and timeout limits still require careful management; otherwise, dense pages may amplify scheduling and memory pressure. Another fundamental limitation is the assumption that, given the full-page image and regional layout, each region’s content can be recognized independently. Cross-column reading order, tables spanning multiple pages, or caption references may violate this assumption. The reported speed results also cover only a specific model, the A800, and a subset selected by the authors, so the approach still needs validation across different GPUs, long documents, and batched serving workloads.