TypeSafe Decision Models

A decision model evaluates an input against a set of named, typed questions and returns one typed answer per question, instead of generated text. Typical uses are classification, routing, gating and grading.

TypeSafe provides decision models, such as Jev, through its System One API.

The DecisionModel API is experimental in LangChain4j and may change in future releases.

Prerequisites

You must provide a valid TypeSafe API key.

<dependency>
  <groupId>io.quarkiverse.langchain4j</groupId>
  <artifactId>quarkus-langchain4j-typesafe</artifactId>
  <version>1.15.0.CR1</version>
</dependency>

Even better, if you use the Quarkus platform BOM (default for projects generated), add the Quarkus Langchain4J BOM and all dependency versions will align:

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>${quarkus.platform.artifact-id}</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>quarkus-langchain4j-bom</artifactId> (1)
                <version>${quarkus.platform.version}</version> (2)
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
      <dependency>
        <groupId>io.quarkiverse.langchain4j</groupId>
        <artifactId>quarkus-langchain4j-typesafe</artifactId>
        (3)
      </dependency>
    </dependencies>
1 In your dependencyManagement section, add the quarkus-langchain4j-bom
2 Inherit the version from your platform version
3 Voilà, no need for version alignment anymore

Configuration

quarkus.langchain4j.typesafe.api-key=...
quarkus.langchain4j.typesafe.decision-model.model-name=jev-latest

Other servers that implement the same API can be used by setting quarkus.langchain4j.typesafe.base-url, in which case the API key is optional.

To use it programmatically:

@Inject DecisionModel decisionModel;

DecisionRequest request = DecisionRequest.builder()
        .input("Help! My payouts have been failing for 3 days.")
        .question("team", ChoiceQuestion.builder()
                .text("Which team should handle this ticket?")
                .option("billing", "Payments, payouts, invoices, refunds")
                .option("support", "Problems using the product")
                .build())
        .question("urgent", YesNoQuestion.of("Does this need attention today?"))
        .build();

DecisionResponse response = decisionModel.decide(request);

ChoiceAnswer team = response.choice("team");
YesNoAnswer urgent = response.yesNo("urgent");

All CDI beans implementing DecisionModelListener are automatically registered with the model.

Multiple models

As with the other model types, multiple decision models can be configured by name and injected using @ModelName:

quarkus.langchain4j.typesafe.m1.api-key=...
quarkus.langchain4j.typesafe.m1.decision-model.model-name=jev-1.13.0
@Inject @ModelName("m1") DecisionModel decisionModel;

When multiple extensions providing a DecisionModel are present, select one with quarkus.langchain4j.decision-model.provider (or quarkus.langchain4j.m1.decision-model.provider for a named model).

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

quarkus.langchain4j.typesafe.decision-model.enabled

Whether the model should be enabled

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_DECISION_MODEL_ENABLED

boolean

true

quarkus.langchain4j.typesafe.base-url

Base URL of the TypeSafe System One API. Set it to use another server that implements the same API.

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_BASE_URL

string

https://api.typesafe.ai

quarkus.langchain4j.typesafe.api-key

TypeSafe API key. Required when using the default base URL, optional for other servers.

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_API_KEY

string

quarkus.langchain4j.typesafe.timeout

Timeout for TypeSafe calls

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_TIMEOUT

Duration 

10s

quarkus.langchain4j.typesafe.log-requests

Whether the TypeSafe client should log requests

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_LOG_REQUESTS

boolean

false

quarkus.langchain4j.typesafe.log-responses

Whether the TypeSafe client should log responses

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_LOG_RESPONSES

boolean

false

quarkus.langchain4j.typesafe.decision-model.model-name

Decision model to use when a request does not specify one

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_DECISION_MODEL_MODEL_NAME

string

jev-latest

quarkus.langchain4j.typesafe.decision-model.log-requests

Whether decision model requests should be logged

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_DECISION_MODEL_LOG_REQUESTS

boolean

false

quarkus.langchain4j.typesafe.decision-model.log-responses

Whether decision model responses should be logged

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE_DECISION_MODEL_LOG_RESPONSES

boolean

false

Named model config

Type

Default

quarkus.langchain4j.typesafe."model-name".base-url

Base URL of the TypeSafe System One API. Set it to use another server that implements the same API.

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__BASE_URL

string

https://api.typesafe.ai

quarkus.langchain4j.typesafe."model-name".api-key

TypeSafe API key. Required when using the default base URL, optional for other servers.

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__API_KEY

string

quarkus.langchain4j.typesafe."model-name".timeout

Timeout for TypeSafe calls

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__TIMEOUT

Duration 

10s

quarkus.langchain4j.typesafe."model-name".log-requests

Whether the TypeSafe client should log requests

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__LOG_REQUESTS

boolean

false

quarkus.langchain4j.typesafe."model-name".log-responses

Whether the TypeSafe client should log responses

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__LOG_RESPONSES

boolean

false

quarkus.langchain4j.typesafe."model-name".decision-model.model-name

Decision model to use when a request does not specify one

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__DECISION_MODEL_MODEL_NAME

string

jev-latest

quarkus.langchain4j.typesafe."model-name".decision-model.log-requests

Whether decision model requests should be logged

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__DECISION_MODEL_LOG_REQUESTS

boolean

false

quarkus.langchain4j.typesafe."model-name".decision-model.log-responses

Whether decision model responses should be logged

Environment variable: QUARKUS_LANGCHAIN4J_TYPESAFE__MODEL_NAME__DECISION_MODEL_LOG_RESPONSES

boolean

false
About the Duration format

To write duration values, use the standard java.time.Duration format. See the Duration#parse() Java API documentation for more information.

You can also use a simplified format, starting with a number:

  • If the value is only a number, it represents time in seconds.

  • If the value is a number followed by ms, it represents time in milliseconds.

In other cases, the simplified format is translated to the java.time.Duration format for parsing:

  • If the value is a number followed by h, m, or s, it is prefixed with PT.

  • If the value is a number followed by d, it is prefixed with P.