Back Home

GitHub Repo

Transformers Community Reports Structural Conversion Bug That May Drop Constraints on Composite Types

When Python tool-function type hints are converted to JSON Schema, certain union types lose array-item or enum-value constraints. The report covers both the released and development versions; maintainers have yet to confirm a fix.

Andrew Wippler from Lancaster, USA · CC BY 2.0 · Image source
zh-Hant

On September 26, the Hugging Face Transformers community filed a tool-calling bug report: `get_json_schema()` may drop array-item and enum-value constraints when handling union types. The reporter said the issue is reproducible in both the released version 5.17.0 and the development version. At the time of review, the issue remained open, with no fix PR linked. [Community report](https://github.com/huggingface/transformers/issues/49122)

This conversion happens at the point where a model receives tool descriptions. The official documentation explains that developers can pass Python functions to the `tools` parameter of `apply_chat_template()`. The system reads the type hints and docstrings to generate a JSON Schema for each tool. When investigating argument errors, agent developers should check not only the prompt and model output, but also whether the structure passed to the template faithfully reflects the function signature. Otherwise, information lost in the conversion layer could be mistaken for a model capability issue. [Official tool documentation](https://huggingface.co/docs/transformers/main/en/chat_extras)

In the reported example, `str | list[str]`, which accepts a string or an array of strings, is converted to just the two broad types—string and array—without the `items` constraint requiring array elements to be strings. Another parameter combining `Literal` and integers loses its `enum`, which restricts the valid string options. These cases concern the generation of tool descriptions; no data has been provided on downstream model miscall rates. [Reproduction examples](https://github.com/huggingface/transformers/issues/49122)

A review of the 5.17.0 source code shows that in the union branch, if every child structure has a string-valued `type`, the structures are collapsed into a list of types. Arrays and enums also meet this condition, so their other constraints are discarded. The code already has an `anyOf` branch that preserves the full child structures; tightening the preceding condition is the fix proposed by the reporter. [Released-version source code](https://github.com/huggingface/transformers/blob/v5.17.0/src/transformers/utils/chat_template_utils.py)

There are limits to the trigger conditions: a single non-null type combined with `None` takes a different branch, so not all optional parameters should be considered affected. By inference from the code, if an application uses the weakened structure for validation, it may accept arguments that should be disallowed. The actual result still depends on the chat template, model, and tool execution layer's checks.

Engineering teams can review the generated structures and add tests for invalid array elements and enum values. The official interface also supports passing a handwritten JSON Schema directly, providing a way to bypass automatic conversion. Teams should watch for maintainer confirmation, regression tests, and a release containing the fix. There is currently no complete list of affected versions, and the community's proposed change should not be treated as a released fix. [Tool definition methods](https://huggingface.co/docs/transformers/main/en/chat_extras)

Sources

  1. get_json_schema drops items / enum when a Union contains list, dict or Literal — Issue #49122
  2. Transformers v5.17.0:chat_template_utils.py
  3. Transformers 官方文件:Tool use