GitHub Repo
Transformers community proposes server-side fix to prevent output token limits from being ignored
A Transformers patch proposal submitted on November 1 addresses a chat endpoint that ignores `max_completion_tokens` and instead uses a default limit of 1,024 tokens. The proposal includes test results, but had not been merged at the time of review, so the official release cannot yet be considered fixed.
On November 1, the Transformers community submitted PR #50225, a fix for how transformers serve handles output length parameters. An earlier issue report showed that when a request to /v1/chat/completions specifies a limit using max_completion_tokens, the server accepts the request and issues a warning but does not apply that value. The latest development is that a specific fix and tests have been submitted; the original issue predates this review window. Proposed fix
The reporter provided a minimal reproduction that directly creates a processor and inspects the generation settings: passing max_completion_tokens=20 produces max_new_tokens of 1024; using max_tokens=20 produces the expected value of 20. The code path shows that the former is listed among unused fields, while the function that creates the generation settings reads only the latter. This demonstrates a mismatch in settings conversion; it does not mean every response will generate 1,024 tokens. Issue and reproduction code
The proposal removes this parameter from the unused-field list, prioritizes max_completion_tokens, and falls back to max_tokens when it is not provided. It also adds tests for using the new parameter alone and for providing both parameters. The author reports that 11 relevant settings tests passed, but that result is provided by the submitter. The page shows no reviews yet, and CI is still awaiting maintainer approval to run. Changes and verification status
The official documentation positions the Serve CLI as a tool for local and self-hosted serving, evaluation, and medium-load deployments; its chat example uses max_tokens. As a result, following the documentation example versus switching to another client may lead to different behavior for length controls. The engineering impact is that evaluation conditions and resource budgets may deviate from expectations, rather than a change in the model’s capabilities. Official serving documentation
Deployment teams can first compare the actual settings produced by the two parameters, rather than assuming the limit took effect simply because the request succeeded. They should then track review, CI, and inclusion in an official release. Public evidence currently does not establish all affected versions or quantify how much additional latency or compute this defect causes under real workloads.