Ollama Pre-release Switches to Native Decision Inference in llama.cpp; Older GGUF Models May Need to Be Re-downloaded

Ollama merged PR #18917 on October 12, updating its bundled llama.cpp to b11521. The change was included in v0.40.4-rc1, released the same day. The main adjustment routes /v1/systemone requests for GGUF decision models directly to llama.cpp, which also assembles the model prompt. This removes Ollama’s separately maintained scoring implementation and Clef patches. It reallocates inference responsibilities; public information does not yet provide performance-improvement figures. Release notes, merge proposal

Decision models can turn shared state into classifications, yes-or-no probabilities, or rubric-based scores. For example, an engineering team could put a ticket in state and ask about its category and urgency. The official documentation lists three question types: choice returns options and probabilities, noul returns the probability that a statement is true, and score calculates a weighted score from the probabilities for each rating. This interface can be used to route workflow items, but the documentation explicitly cautions that a higher confidence value does not guarantee a correct answer. Decision models guide

The new path runs questions within a single request in parallel. Clef, Laya, and LFM2-D1-Omni read the full prompt in one batch, so the scheduler sets the batch size according to the context configuration. The update also standardizes score across all models to use two to ten levels and rejects empty state. For deployment, engineers should compare scores, latency, and memory usage for the same inputs before and after the update to assess the engine switch’s effect on their workloads. Technical change details

Compatibility is the most immediate consideration in this update: decision GGUF files no longer need to declare CAPABILITY decision manually, but older files without llama.cpp decision metadata will return an error requiring a re-download. The maintainers say updated models will be pushed separately and marked as requiring 0.40.4; merging the code does not mean every model file has been updated. Migration notes

The current version is still a pre-release. The System One API returns a single JSON response and does not support streaming or tool calls; the full input must also fit within the context limit. Follow-up items include the stable release, model-file availability, and regression tests of decision results across versions. API documentation