Ollama Decision Models
A decision model evaluates an input against a set of named, typed questions and returns one typed answer per question, instead of generated text. Typical uses are classification, routing, gating and grading.
Ollama runs decision models such as nimble, tev1 and clef locally, through its System One API (Ollama v0.35.0 or later).
The DecisionModel API is experimental in LangChain4j and may change in future releases.
|
Prerequisites
Ollama Installation
To use decision models, you must have a working Ollama setup. Refer to Ollama Chat Models for details on installation.
Pull the decision model you want to use before starting the application:
ollama pull nimble| The Ollama Dev Service only pulls chat and embedding models, so decision models must be pulled manually. |
Enabling Ollama
To use Ollama decision models in your Quarkus application, add the quarkus-langchain4j-ollama extension:
<dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-ollama</artifactId>
<version>1.15.0.CR2</version>
</dependency>Even better, if you use the Quarkus platform BOM (default for projects generated), add the Quarkus Langchain4J BOM and all dependency versions will align:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>${quarkus.platform.group-id}</groupId>
<artifactId>${quarkus.platform.artifact-id}</artifactId>
<version>${quarkus.platform.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>${quarkus.platform.group-id}</groupId>
<artifactId>quarkus-langchain4j-bom</artifactId> (1)
<version>${quarkus.platform.version}</version> (2)
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-ollama</artifactId>
(3)
</dependency>
</dependencies>| 1 | In your dependencyManagement section, add the quarkus-langchain4j-bom |
| 2 | Inherit the version from your platform version |
| 3 | Voilà, no need for version alignment anymore |
Default Model
By default, the decision model is set to nimble.
You can override this using:
quarkus.langchain4j.ollama.decision-model.model-id=tev1You may also wish to configure logging during development:
quarkus.langchain4j.log-requests=true
quarkus.langchain4j.log-responses=trueUsing the Decision Model
Inject the DecisionModel bean and call decide(…) with a DecisionRequest:
import dev.langchain4j.model.decision.DecisionModel;
import dev.langchain4j.model.decision.request.ChoiceQuestion;
import dev.langchain4j.model.decision.request.DecisionRequest;
import dev.langchain4j.model.decision.request.YesNoQuestion;
import dev.langchain4j.model.decision.response.ChoiceAnswer;
import dev.langchain4j.model.decision.response.DecisionResponse;
import dev.langchain4j.model.decision.response.YesNoAnswer;
@Inject DecisionModel decisionModel;
DecisionRequest request = DecisionRequest.builder()
.input("Help! My payouts have been failing for 3 days.")
.question("team", ChoiceQuestion.builder()
.text("Which team should handle this ticket?")
.option("billing", "Payments, payouts, invoices, refunds")
.option("support", "Problems using the product")
.build())
.question("urgent", YesNoQuestion.from("Does this need attention today?"))
.build();
DecisionResponse response = decisionModel.decide(request);
ChoiceAnswer team = response.choice("team"); // team.value() is "billing", team.probabilities() has one entry per option
YesNoAnswer urgent = response.yesNo("urgent"); // urgent.probability() is the probability of "yes"The input can be text, named fields, a list of TextContent and ImageContent, or a conversation.
Images are only judged by models with vision, such as clef, and are sent alongside the input, which refers to each of them by its position, for example [image 1].
decideAsync(…) returns a CompletableFuture<DecisionResponse> for non-blocking callers.
All CDI beans implementing DecisionModelListener are automatically registered with the model, and
ModelBuilderCustomizer<OllamaDecisionModel.OllamaDecisionModelBuilder> beans can tune the underlying builder.
Multiple models
As with the other model types, multiple decision models can be configured by name and injected using @ModelName:
quarkus.langchain4j.ollama.m1.base-url=http://localhost:11434
quarkus.langchain4j.ollama.m1.decision-model.model-id=tev1@Inject @ModelName("m1") DecisionModel decisionModel;When multiple extensions providing a DecisionModel are present, select one with quarkus.langchain4j.decision-model.provider=ollama (or quarkus.langchain4j.m1.decision-model.provider for a named model).
Disabling the model
Set quarkus.langchain4j.ollama.decision-model.enabled=false at build time to stop the extension from providing a DecisionModel bean.
Set quarkus.langchain4j.ollama.enable-integration=false at runtime to keep the bean but make every call fail with a ModelDisabledException, as for the other Ollama models.
Dynamic Authorization
To provide dynamic authorization headers, implement ModelAuthProvider:
import io.quarkiverse.langchain4j.auth.ModelAuthProvider;
import jakarta.enterprise.context.ApplicationScoped;
@ApplicationScoped
public class MyAuthProvider implements ModelAuthProvider {
@Override
public String getAuthorization(Input input) {
return "Bearer " + fetchToken();
}
}Configuration Reference
Configuration property fixed at build time - All other configuration properties are overridable at runtime
Configuration property |
Type |
Default |
|---|---|---|
Whether the model should be enabled Environment variable: |
boolean |
|
Whether the model should be enabled Environment variable: |
boolean |
|
Whether the model should be enabled Environment variable: |
boolean |
|
If Dev Services for Ollama has been explicitly enabled or disabled. Dev Services are generally enabled by default, unless there is an existing configuration present. Environment variable: |
boolean |
|
The Ollama container image to use. Environment variable: |
string |
|
The port that the dev service should be exposed on. Default: A random free port. Environment variable: |
int |
|
Additional model options to apply when pulling/loading the model. These options will be propagated to the chat-model configuration. For example: Environment variable: |
Map<String,String> |
|
Model to use Environment variable: |
string |
|
Model to use. According to Ollama docs, the default value is Environment variable: |
string |
|
Model to use. Ollama runs decision models such as Environment variable: |
string |
|
Base URL where the Ollama serving is running Environment variable: |
string |
|
If set, the named TLS configuration with the configured name will be applied to the REST Client Environment variable: |
string |
|
Timeout for Ollama calls Environment variable: |
|
|
Whether the Ollama client should log requests Environment variable: |
boolean |
|
Whether the Ollama client should log responses Environment variable: |
boolean |
|
Whether the Ollama client should log requests as cURL commands Environment variable: |
boolean |
|
Whether to enable the integration. Defaults to Environment variable: |
boolean |
|
The temperature of the model. Increasing the temperature will make the model answer with more variability. A lower temperature will make the model answer more conservatively. Environment variable: |
double |
|
Maximum number of tokens to predict when generating text Environment variable: |
int |
|
Sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return Environment variable: |
list of string |
|
Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text Environment variable: |
double |
|
Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative Environment variable: |
int |
|
With a static number the result is always the same. With a random number the result varies Example: Environment variable: |
int |
|
The format to return a response in. Format can be Environment variable: |
string |
|
Whether chat model requests should be logged Environment variable: |
boolean |
|
Whether chat model responses should be logged Environment variable: |
boolean |
|
Enable thinking mode for models that support it (e.g., Qwen3). When enabled, the model will output its reasoning process. Environment variable: |
boolean |
|
Whether to return thinking/reasoning content in the response. Environment variable: |
boolean |
|
Sets the size of the context window used to generate the next token. Environment variable: |
int |
|
The last N tokens to penalize. Higher values will penalize more. Environment variable: |
int |
|
The penalty for repeated tokens. Higher values will penalize repetition more. Environment variable: |
double |
|
Enable Mirostat sampling for controlling perplexity. 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0. Environment variable: |
int |
|
Controls the learning rate of Mirostat. Lower values result in slower adjustments. Environment variable: |
double |
|
Controls the target perplexity for Mirostat. Lower values produce more focused outputs. Environment variable: |
double |
|
Sets the minimum probability relative to the most likely token. Lower values will result in more diverse outputs. Environment variable: |
double |
|
The temperature of the model. Increasing the temperature will make the model answer with more variability. A lower temperature will make the model answer more conservatively. Environment variable: |
double |
|
Maximum number of tokens to predict when generating text Environment variable: |
int |
|
Sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return Environment variable: |
list of string |
|
Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text Environment variable: |
double |
|
Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative Environment variable: |
int |
|
Whether embedding model requests should be logged Environment variable: |
boolean |
|
Whether embedding model responses should be logged Environment variable: |
boolean |
|
Whether decision model requests should be logged Environment variable: |
boolean |
|
Whether decision model responses should be logged Environment variable: |
boolean |
|
Type |
Default |
|
Model to use Environment variable: |
string |
|
Model to use. According to Ollama docs, the default value is Environment variable: |
string |
|
Model to use. Ollama runs decision models such as Environment variable: |
string |
|
Base URL where the Ollama serving is running Environment variable: |
string |
|
If set, the named TLS configuration with the configured name will be applied to the REST Client Environment variable: |
string |
|
Timeout for Ollama calls Environment variable: |
|
|
Whether the Ollama client should log requests Environment variable: |
boolean |
|
Whether the Ollama client should log responses Environment variable: |
boolean |
|
Whether the Ollama client should log requests as cURL commands Environment variable: |
boolean |
|
Whether to enable the integration. Defaults to Environment variable: |
boolean |
|
The temperature of the model. Increasing the temperature will make the model answer with more variability. A lower temperature will make the model answer more conservatively. Environment variable: |
double |
|
Maximum number of tokens to predict when generating text Environment variable: |
int |
|
Sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return Environment variable: |
list of string |
|
Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text Environment variable: |
double |
|
Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative Environment variable: |
int |
|
With a static number the result is always the same. With a random number the result varies Example: Environment variable: |
int |
|
The format to return a response in. Format can be Environment variable: |
string |
|
Whether chat model requests should be logged Environment variable: |
boolean |
|
Whether chat model responses should be logged Environment variable: |
boolean |
|
Enable thinking mode for models that support it (e.g., Qwen3). When enabled, the model will output its reasoning process. Environment variable: |
boolean |
|
Whether to return thinking/reasoning content in the response. Environment variable: |
boolean |
|
Sets the size of the context window used to generate the next token. Environment variable: |
int |
|
The last N tokens to penalize. Higher values will penalize more. Environment variable: |
int |
|
The penalty for repeated tokens. Higher values will penalize repetition more. Environment variable: |
double |
|
Enable Mirostat sampling for controlling perplexity. 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0. Environment variable: |
int |
|
Controls the learning rate of Mirostat. Lower values result in slower adjustments. Environment variable: |
double |
|
Controls the target perplexity for Mirostat. Lower values produce more focused outputs. Environment variable: |
double |
|
Sets the minimum probability relative to the most likely token. Lower values will result in more diverse outputs. Environment variable: |
double |
|
The temperature of the model. Increasing the temperature will make the model answer with more variability. A lower temperature will make the model answer more conservatively. Environment variable: |
double |
|
Maximum number of tokens to predict when generating text Environment variable: |
int |
|
Sets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return Environment variable: |
list of string |
|
Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text Environment variable: |
double |
|
Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative Environment variable: |
int |
|
Whether embedding model requests should be logged Environment variable: |
boolean |
|
Whether embedding model responses should be logged Environment variable: |
boolean |
|
Whether decision model requests should be logged Environment variable: |
boolean |
|
Whether decision model responses should be logged Environment variable: |
boolean |
|
|
About the Duration format
To write duration values, use the standard You can also use a simplified format, starting with a number:
In other cases, the simplified format is translated to the
|