GitHub Repo
Dify 1.19.0 Adds Nested Workflow Tracing; Rolling Upgrades Must Route Old and New Queues Separately
Released on November 4, Dify 1.19.0 lets external observability tools see execution steps and model calls inside Workflow Tools. The update also restructures the tracing pipeline, so teams using rolling deployments must manage the handoff between old and new task queues.
Dify released version 1.19.0 on November 4, adding internal steps from nested workflows to external traces. When a main workflow calls a Workflow Tool, the tool’s persisted child steps appear under that tool call’s span, preserving relationships across repeated calls, failures, and deeper nested calls. Traces also show formatted prompts, responses, and token usage. Official release notes
This change improves debugging for reusable workflows. For example, if the same tool is called several times and only one call fails, engineers can follow the execution tree to that call and inspect the context sent to the model. This use case can be inferred from the new trace structure; the official notes do not quantify any reduction in debugging time. The existing Langfuse integration is still enabled through Dify’s Monitoring settings. After creating project credentials, traces are generated by calls through the application’s interface or API. Langfuse integration documentation
The underlying implementation has also been substantially reworked: related changes consolidate overlapping tracing paths into a shared pipeline, through which providers receive a vendor-neutral span tree via a consistent interface. IDs are deterministic, so resubmissions update the same record. Delivery failures are classified as retryable, rejected, or indeterminate, providing a clear basis for retry handling. Tracing refactor proposal
Deployment teams should note that new tasks are routed to ops_trace_v2 by default. The API, worker, and Beat configurations must match, and custom queue subscriptions must include it. During a rolling upgrade, old workers should continue processing ops_trace. Retire them only after old producers have stopped and queued or delayed-retry work has drained, to avoid mixing old and new workloads. Upgrade and queue handoff notes
The fuller hierarchy applies only to new runs; historical traces are not backfilled. Standalone nested workflows still link separate traces through a parent ID. After upgrading, engineering teams should check actual exported traces using repeated-call, failure, and retry scenarios, and confirm that existing dashboards can handle the new span structure. Version limitations