IBM watsonx.ai Embedding and Scoring Models
IBM watsonx.ai provides embedding models that turn text into vector representations and scoring models that rank a set of documents by their relevance to a query.
| This extension supports IBM watsonx as a service on IBM Cloud only. |
Prerequisites
To use watsonx.ai models, configure the following required values in your application.properties file:
Base URL
The base-url depends on the region of your service instance:
-
Dallas - https://us-south.ml.cloud.ibm.com
-
Frankfurt - https://eu-de.ml.cloud.ibm.com
-
London - https://eu-gb.ml.cloud.ibm.com
-
Tokyo - https://jp-tok.ml.cloud.ibm.com
-
Sydney - https://au-syd.ml.cloud.ibm.com
-
Toronto - https://ca-tor.ml.cloud.ibm.com
-
Mumbai - https://ap-south-1.aws.wxai.ibm.com
quarkus.langchain4j.watsonx.base-url=https://us-south.ml.cloud.ibm.com
Project ID
Obtain the Project Id via:
-
Visit https://dataplatform.cloud.ibm.com/projects/?context=wx
-
Open your project and click the Manage tab.
-
Copy the Project ID from the Details section.
quarkus.langchain4j.watsonx.project-id=23d...
| You may use the optional space-id as an alternative. |
API Key
Create an API key by visiting https://cloud.ibm.com/iam/apikeys and clicking Create +.
quarkus.langchain4j.watsonx.api-key=your-api-key
You can also use the QUARKUS_LANGCHAIN4J_WATSONX_API_KEY environment variable.
|
Dependency
Add the following dependency to your project:
<dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-watsonx</artifactId>
<version>1.13.0</version>
</dependency>
Even better, if you use the Quarkus platform BOM (default for projects generated), add the Quarkus Langchain4J BOM and all dependency versions will align:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>${quarkus.platform.group-id}</groupId>
<artifactId>${quarkus.platform.artifact-id}</artifactId>
<version>${quarkus.platform.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>${quarkus.platform.group-id}</groupId>
<artifactId>quarkus-langchain4j-bom</artifactId> (1)
<version>${quarkus.platform.version}</version> (2)
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-watsonx</artifactId>
(3)
</dependency>
</dependencies>
| 1 | In your dependencyManagement section, add the quarkus-langchain4j-bom |
| 2 | Inherit the version from your platform version |
| 3 | Voilà, no need for version alignment anymore |
Embedding Model
IBM watsonx.ai provides multiple embedding models for converting text into vector representations suitable for semantic search, RAG pipelines, similarity comparison, and vector database integrations.
Quarkus integrates the LangChain4j WatsonxEmbeddingModel, exposing it as EmbeddingModel bean.
The list of supported embedding models is available in the watsonx.ai documentation.
Configuration
Configure the embedding model by specifying its model name in application.properties.
# Base Watsonx configuration
quarkus.langchain4j.watsonx.base-url=${BASE_URL}
quarkus.langchain4j.watsonx.api-key=${API_KEY}
quarkus.langchain4j.watsonx.project-id=${PROJECT_ID}
# Embedding model configuration
quarkus.langchain4j.watsonx.embedding-model.model-name=ibm/slate-125m-english-rtrvr
If an embedding model is configured, Quarkus will automatically create and register
a EmbeddingModel bean.
Usage
To generate the embedding of a single text, use embed.
var response = embeddingModel.embed("Hello Watsonx!");
assertNotNull(response);
var embedding = response.content();
System.out.println("Embedding size: " + embedding.vector().length());
To embed more than one text segment at a time, use embedAll.
var embeddings = embeddingModel.embedAll(
List.of(
TextSegment.from("First document"),
TextSegment.from("Second document")
)
);
Scoring Model
IBM watsonx.ai provides scoring (reranking) models that evaluate the relevance between a query and a piece of text.
Quarkus integrates the LangChain4j WatsonxScoringModel, exposing it as ScoringModel implementation.
Scoring models are especially useful for RAG pipelines, document ranking, and semantic relevance evaluation.
The list of supported scoring models is available in the watsonx.ai documentation.
Configuration
Configure the model by specifying its name in application.properties.
# Base Watsonx configuration
quarkus.langchain4j.watsonx.base-url=${BASE_URL}
quarkus.langchain4j.watsonx.api-key=${API_KEY}
quarkus.langchain4j.watsonx.project-id=${PROJECT_ID}
# Scoring model configuration
quarkus.langchain4j.watsonx.scoring-model.model-name=cross-encoder/ms-marco-minilm-l-12-v2
If a scoring model is configured, Quarkus will automatically create and register
a ScoringModel bean.
Usage
To score a single text against a query, use score.
var response = scoringModel.score("Rerank this!", "Test to rerank 1");
assertNotNull(response);
assertNotNull(response.content());
double score = response.content();
System.out.println("Score: " + score);
To score more than one document at a time, use scoreAll.
var scores = scoringModel.scoreAll(
List.of(
TextSegment.from("Document A"),
TextSegment.from("Document B")
),
"User query"
);
System.out.println(scores); // list of relevance scores
Configuration
Configuration property fixed at build time - All other configuration properties are overridable at runtime
Configuration property |
Type |
Default |
|---|---|---|
Whether the model should be enabled. Environment variable: |
boolean |
|
Whether the embedding model should be enabled. Environment variable: |
boolean |
|
Whether the scoring model should be enabled. Environment variable: |
boolean |
|
Whether the moderation model should be enabled. Environment variable: |
boolean |
|
Specifies the base URL of the watsonx.ai API. A list of all available URLs is provided in the IBM Watsonx.ai documentation at the this link. Environment variable: |
string |
|
IBM Cloud API key. Environment variable: |
string |
|
Timeout for watsonx.ai calls. Environment variable: |
|
|
The version date for the API of the form YYYY-MM-DD. Environment variable: |
string |
|
The space that contains the resource. Either Environment variable: |
string |
|
The project that contains the resource. Either Environment variable: |
string |
|
Whether the watsonx.ai client should log requests. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log responses. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether to enable the integration. Defaults to Environment variable: |
boolean |
|
Base URL of the IAM Authentication API. Environment variable: |
||
Timeout for IAM authentication calls. Environment variable: |
|
|
Grant type for the IAM Authentication API. Environment variable: |
string |
|
Base URL of the Cloud Object Storage API. Environment variable: |
string |
required |
The id of the connection asset that contains the credentials required to access the data. Environment variable: |
string |
required |
The name of the bucket containing the input document. Environment variable: |
string |
required |
The id of the connection asset used to store the extracted results. Environment variable: |
string |
required |
The name of the bucket where the output files will be written. Environment variable: |
string |
required |
Whether text extraction requests should be logged. Environment variable: |
boolean |
|
Whether text extraction responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Base URL of the Cloud Object Storage API. Environment variable: |
string |
required |
The id of the connection asset that contains the credentials required to access the data. Environment variable: |
string |
required |
The name of the bucket containing the input document. Environment variable: |
string |
required |
Whether text extraction requests should be logged. Environment variable: |
boolean |
|
Whether text extraction responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Base URL of the Cloud Object Storage API. Environment variable: |
string |
required |
The id of the connection asset that contains the credentials required to access the data. Environment variable: |
string |
required |
The name of the bucket containing the input document. Environment variable: |
string |
required |
Whether create schema requests should be logged. Environment variable: |
boolean |
|
Whether create schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether improve schema requests should be logged. Environment variable: |
boolean |
|
Whether improve schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether merge schema requests should be logged. Environment variable: |
boolean |
|
Whether merge schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether cluster schema requests should be logged. Environment variable: |
boolean |
|
Whether cluster schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Specifies how the model should choose which tool to call during a request. This value can be:
If Setting this value influences the tool-calling behavior of the model when no specific tool is required. Environment variable: |
|
|
Specifies the name of a specific tool that the model must call. When set, the model will be forced to call the specified tool. The name must exactly match one of the available tools defined for the service. Environment variable: |
string |
|
Positive values penalize new tokens based on their existing frequency in the generated text, reducing the likelihood of the model repeating the same lines verbatim. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies whether to return the log probabilities of the output tokens. If set to The parameter is sent to the model only when it is set. Environment variable: |
boolean |
|
An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option Possible values: Environment variable: |
int |
|
The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. Set to 0 for the model’s configured max generated tokens. Environment variable: |
int |
|
Applies a penalty to new tokens based on whether they already appear in the generated text so far, encouraging the model to introduce new topics rather than repeat itself. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Random number generator seed to use in sampling mode for experimental repeatability. Environment variable: |
int |
|
Defines one or more stop sequences that will cause the model to stop generating further tokens if any of them are encountered in the output. This allows control over where the model should end its response. If a stop sequence is encountered before the minimum number of tokens has been generated, it will be ignored. Possible values: Environment variable: |
list of string |
|
Specifies the sampling temperature to use in the generation process. Higher values (e.g. Possible values: Environment variable: |
double |
|
An alternative to sampling with The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies the desired format for the model’s output. Allowable values: Environment variable: |
|
|
Whether the JSON Schema sent to the model should use the strict mode. When enabled, the model is constrained to return a response that exactly matches the given JSON Schema. To satisfy the restrictions of the strict mode, all the properties of the schema are marked as required, the optional ones are made nullable and Set this property to Environment variable: |
boolean |
|
Whether chat model requests should be logged. Environment variable: |
boolean |
|
Whether chat model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Specifies a set of allowed output choices. When this parameter is set, the model is constrained to return exactly one of the provided choices. Environment variable: |
list of string |
|
Constrains the model output to follow a context-free grammar. If specified, the generated output will conform to the defined grammar. Environment variable: |
string |
|
Constrains the model output to match a regular expression pattern. If specified, the generated output must conform to the provided regex. Environment variable: |
string |
|
Sets the length penalty to be applied during text generation. This penalty influences the length of the generated text. A length penalty discourages the model from generating overly long responses, or conversely, it can encourage more extended outputs. When the penalty value is greater than 1.0, it discourages generating longer responses. Conversely, a value less than 1.0 incentivizes the model to generate longer text. A value of 1.0 means no penalty, and the length of the output will be determined by other factors, such as the input prompt and model’s natural completion behavior. Environment variable: |
double |
|
Sets the repetition penalty to be applied during text generation. This penalty helps to discourage the model from repeating the same words or phrases too often. The penalty value should be greater than 1.0 for repetition discouragement. A value of 1.0 means no penalty, and values above 1.0 increase the strength of the penalty. Environment variable: |
double |
|
Enables or disables reasoning. Environment variable: |
boolean |
|
The opening delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The closing delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The opening delimiter for the model’s final response section. Environment variable: |
string |
required |
The closing delimiter for the model’s final response section. Environment variable: |
string |
required |
Controls the reasoning effort level for models that separate reasoning and response automatically. Example values: Environment variable: |
|
|
Determines whether the reasoning portion returned by the model should be included in the final response provided to the application. Environment variable: |
boolean |
|
Specifies the model to use for the chat completion. A list of all available models is provided in the IBM watsonx.ai documentation at the this link. To use a model, locate the Environment variable: |
string |
|
Specifies how the model should choose which tool to call during a request. This value can be:
If Setting this value influences the tool-calling behavior of the model when no specific tool is required. Environment variable: |
|
|
Specifies the name of a specific tool that the model must call. When set, the model will be forced to call the specified tool. The name must exactly match one of the available tools defined for the service. Environment variable: |
string |
|
Positive values penalize new tokens based on their existing frequency in the generated text, reducing the likelihood of the model repeating the same lines verbatim. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies whether to return the log probabilities of the output tokens. If set to The parameter is sent to the model only when it is set. Environment variable: |
boolean |
|
An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option Possible values: Environment variable: |
int |
|
The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. Set to 0 for the model’s configured max generated tokens. Environment variable: |
int |
|
Applies a penalty to new tokens based on whether they already appear in the generated text so far, encouraging the model to introduce new topics rather than repeat itself. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Random number generator seed to use in sampling mode for experimental repeatability. Environment variable: |
int |
|
Defines one or more stop sequences that will cause the model to stop generating further tokens if any of them are encountered in the output. This allows control over where the model should end its response. If a stop sequence is encountered before the minimum number of tokens has been generated, it will be ignored. Possible values: Environment variable: |
list of string |
|
Specifies the sampling temperature to use in the generation process. Higher values (e.g. Possible values: Environment variable: |
double |
|
An alternative to sampling with The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies the desired format for the model’s output. Allowable values: Environment variable: |
|
|
Whether the JSON Schema sent to the model should use the strict mode. When enabled, the model is constrained to return a response that exactly matches the given JSON Schema. To satisfy the restrictions of the strict mode, all the properties of the schema are marked as required, the optional ones are made nullable and Set this property to Environment variable: |
boolean |
|
Whether chat model requests should be logged. Environment variable: |
boolean |
|
Whether chat model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Specifies a set of allowed output choices. When this parameter is set, the model is constrained to return exactly one of the provided choices. Environment variable: |
list of string |
|
Constrains the model output to follow a context-free grammar. If specified, the generated output will conform to the defined grammar. Environment variable: |
string |
|
Constrains the model output to match a regular expression pattern. If specified, the generated output must conform to the provided regex. Environment variable: |
string |
|
Sets the length penalty to be applied during text generation. This penalty influences the length of the generated text. A length penalty discourages the model from generating overly long responses, or conversely, it can encourage more extended outputs. When the penalty value is greater than 1.0, it discourages generating longer responses. Conversely, a value less than 1.0 incentivizes the model to generate longer text. A value of 1.0 means no penalty, and the length of the output will be determined by other factors, such as the input prompt and model’s natural completion behavior. Environment variable: |
double |
|
Sets the repetition penalty to be applied during text generation. This penalty helps to discourage the model from repeating the same words or phrases too often. The penalty value should be greater than 1.0 for repetition discouragement. A value of 1.0 means no penalty, and values above 1.0 increase the strength of the penalty. Environment variable: |
double |
|
Enables or disables reasoning. Environment variable: |
boolean |
|
The opening delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The closing delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The opening delimiter for the model’s final response section. Environment variable: |
string |
required |
The closing delimiter for the model’s final response section. Environment variable: |
string |
required |
Controls the reasoning effort level for models that separate reasoning and response automatically. Example values: Environment variable: |
|
|
Determines whether the reasoning portion returned by the model should be included in the final response provided to the application. Environment variable: |
boolean |
|
The deployment ID of the model deployed in watsonx.ai. Setting this property routes all chat requests to the deployment chat API. Environment variable: |
string |
|
Specifies how the model should choose which tool to call during a request. This value can be:
If Setting this value influences the tool-calling behavior of the model when no specific tool is required. Environment variable: |
|
|
Specifies the name of a specific tool that the model must call. When set, the model will be forced to call the specified tool. The name must exactly match one of the available tools defined for the service. Environment variable: |
string |
|
Positive values penalize new tokens based on their existing frequency in the generated text, reducing the likelihood of the model repeating the same lines verbatim. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies whether to return the log probabilities of the output tokens. If set to The parameter is sent to the model only when it is set. Environment variable: |
boolean |
|
An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option Possible values: Environment variable: |
int |
|
The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. Set to 0 for the model’s configured max generated tokens. Environment variable: |
int |
|
Applies a penalty to new tokens based on whether they already appear in the generated text so far, encouraging the model to introduce new topics rather than repeat itself. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Random number generator seed to use in sampling mode for experimental repeatability. Environment variable: |
int |
|
Defines one or more stop sequences that will cause the model to stop generating further tokens if any of them are encountered in the output. This allows control over where the model should end its response. If a stop sequence is encountered before the minimum number of tokens has been generated, it will be ignored. Possible values: Environment variable: |
list of string |
|
Specifies the sampling temperature to use in the generation process. Higher values (e.g. Possible values: Environment variable: |
double |
|
An alternative to sampling with The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies the desired format for the model’s output. Allowable values: Environment variable: |
|
|
Whether the JSON Schema sent to the model should use the strict mode. When enabled, the model is constrained to return a response that exactly matches the given JSON Schema. To satisfy the restrictions of the strict mode, all the properties of the schema are marked as required, the optional ones are made nullable and Set this property to Environment variable: |
boolean |
|
Whether chat model requests should be logged. Environment variable: |
boolean |
|
Whether chat model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
The identifier of the model to use, as configured in the Model Gateway (for example Setting this property routes all chat requests to the Model Gateway. Environment variable: |
string |
|
Specifies the latency tier used to serve the request. Allowable values: Environment variable: |
|
|
Constrains the effort spent on reasoning for reasoning models. Reducing the reasoning effort can result in faster responses and fewer tokens used on reasoning. Allowable values: Environment variable: |
|
|
Whether the semantic cache is enabled. Environment variable: |
boolean |
|
The similarity threshold a cached entry must reach to be served instead of calling the model. Environment variable: |
double |
|
The output types that the model is requested to generate. Most models are only able to generate Environment variable: |
list of string |
|
Whether the generated output should be stored for model distillation or evaluations. Environment variable: |
boolean |
|
Whether the model is allowed to run tool calls in parallel. Environment variable: |
boolean |
|
A stable identifier of the end user issuing the request, used by the backing provider to detect and prevent abuse. Environment variable: |
string |
|
A set of key/value pairs that is attached to the request and returned with the response. Environment variable: |
Map<String,String> |
|
Specifies the ID of the model to be used. A list of all available models is provided in the IBM watsonx.ai documentation at the this link. To use a model, locate the Environment variable: |
string |
|
Whether embedding model requests should be logged. Environment variable: |
boolean |
|
Whether embedding model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
The id of the model to be used. All available models are listed in the IBM Watsonx.ai documentation at the link: following link. To use a model, locate the Environment variable: |
string |
|
Whether embedding model requests should be logged. Environment variable: |
boolean |
|
Whether embedding model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Indicates whether the PII moderation model is enabled. Environment variable: |
boolean |
required |
Indicates whether the HAP moderation model is enabled. Environment variable: |
boolean |
required |
Threshold value for HAP moderation model. Environment variable: |
double |
|
Indicates whether the GraniteGuardian moderation model is enabled. Environment variable: |
boolean |
required |
Threshold value for Granite Guardian moderation model. Environment variable: |
double |
|
Whether moderation model requests should be logged. Environment variable: |
boolean |
|
Whether moderation model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Base URL for the built-in service. All available URLs are listed in the IBM Watsonx.ai documentation at the following link. Note: If empty, the URL is automatically calculated based on the Environment variable: |
string |
|
Timeout for built-in tools APIs. If empty, the api key inherits the value from the Environment variable: |
|
|
Whether the built-in rest client should log requests. Environment variable: |
boolean |
|
Whether the built-in rest client should log responses. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Tavily API key. Environment variable: |
string |
|
Deployment id. Environment variable: |
string |
|
Vector index ids Environment variable: |
list of string |
|
Type |
Default |
|
Specifies the base URL of the watsonx.ai API. A list of all available URLs is provided in the IBM Watsonx.ai documentation at the this link. Environment variable: |
string |
|
IBM Cloud API key. Environment variable: |
string |
|
Timeout for watsonx.ai calls. Environment variable: |
|
|
The version date for the API of the form YYYY-MM-DD. Environment variable: |
string |
|
The space that contains the resource. Either Environment variable: |
string |
|
The project that contains the resource. Either Environment variable: |
string |
|
Whether the watsonx.ai client should log requests. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log responses. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether to enable the integration. Defaults to Environment variable: |
boolean |
|
Base URL of the IAM Authentication API. Environment variable: |
||
Timeout for IAM authentication calls. Environment variable: |
|
|
Grant type for the IAM Authentication API. Environment variable: |
string |
|
Base URL of the Cloud Object Storage API. Environment variable: |
string |
required |
The id of the connection asset that contains the credentials required to access the data. Environment variable: |
string |
required |
The name of the bucket containing the input document. Environment variable: |
string |
required |
The id of the connection asset used to store the extracted results. Environment variable: |
string |
required |
The name of the bucket where the output files will be written. Environment variable: |
string |
required |
Whether text extraction requests should be logged. Environment variable: |
boolean |
|
Whether text extraction responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Base URL of the Cloud Object Storage API. Environment variable: |
string |
required |
The id of the connection asset that contains the credentials required to access the data. Environment variable: |
string |
required |
The name of the bucket containing the input document. Environment variable: |
string |
required |
Whether text extraction requests should be logged. Environment variable: |
boolean |
|
Whether text extraction responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Base URL of the Cloud Object Storage API. Environment variable: |
string |
required |
The id of the connection asset that contains the credentials required to access the data. Environment variable: |
string |
required |
The name of the bucket containing the input document. Environment variable: |
string |
required |
Whether create schema requests should be logged. Environment variable: |
boolean |
|
Whether create schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether improve schema requests should be logged. Environment variable: |
boolean |
|
Whether improve schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether merge schema requests should be logged. Environment variable: |
boolean |
|
Whether merge schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Whether cluster schema requests should be logged. Environment variable: |
boolean |
|
Whether cluster schema responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Specifies how the model should choose which tool to call during a request. This value can be:
If Setting this value influences the tool-calling behavior of the model when no specific tool is required. Environment variable: |
|
|
Specifies the name of a specific tool that the model must call. When set, the model will be forced to call the specified tool. The name must exactly match one of the available tools defined for the service. Environment variable: |
string |
|
Positive values penalize new tokens based on their existing frequency in the generated text, reducing the likelihood of the model repeating the same lines verbatim. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies whether to return the log probabilities of the output tokens. If set to The parameter is sent to the model only when it is set. Environment variable: |
boolean |
|
An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option Possible values: Environment variable: |
int |
|
The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. Set to 0 for the model’s configured max generated tokens. Environment variable: |
int |
|
Applies a penalty to new tokens based on whether they already appear in the generated text so far, encouraging the model to introduce new topics rather than repeat itself. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Random number generator seed to use in sampling mode for experimental repeatability. Environment variable: |
int |
|
Defines one or more stop sequences that will cause the model to stop generating further tokens if any of them are encountered in the output. This allows control over where the model should end its response. If a stop sequence is encountered before the minimum number of tokens has been generated, it will be ignored. Possible values: Environment variable: |
list of string |
|
Specifies the sampling temperature to use in the generation process. Higher values (e.g. Possible values: Environment variable: |
double |
|
An alternative to sampling with The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies the desired format for the model’s output. Allowable values: Environment variable: |
|
|
Whether the JSON Schema sent to the model should use the strict mode. When enabled, the model is constrained to return a response that exactly matches the given JSON Schema. To satisfy the restrictions of the strict mode, all the properties of the schema are marked as required, the optional ones are made nullable and Set this property to Environment variable: |
boolean |
|
Whether chat model requests should be logged. Environment variable: |
boolean |
|
Whether chat model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Specifies a set of allowed output choices. When this parameter is set, the model is constrained to return exactly one of the provided choices. Environment variable: |
list of string |
|
Constrains the model output to follow a context-free grammar. If specified, the generated output will conform to the defined grammar. Environment variable: |
string |
|
Constrains the model output to match a regular expression pattern. If specified, the generated output must conform to the provided regex. Environment variable: |
string |
|
Sets the length penalty to be applied during text generation. This penalty influences the length of the generated text. A length penalty discourages the model from generating overly long responses, or conversely, it can encourage more extended outputs. When the penalty value is greater than 1.0, it discourages generating longer responses. Conversely, a value less than 1.0 incentivizes the model to generate longer text. A value of 1.0 means no penalty, and the length of the output will be determined by other factors, such as the input prompt and model’s natural completion behavior. Environment variable: |
double |
|
Sets the repetition penalty to be applied during text generation. This penalty helps to discourage the model from repeating the same words or phrases too often. The penalty value should be greater than 1.0 for repetition discouragement. A value of 1.0 means no penalty, and values above 1.0 increase the strength of the penalty. Environment variable: |
double |
|
Enables or disables reasoning. Environment variable: |
boolean |
|
The opening delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The closing delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The opening delimiter for the model’s final response section. Environment variable: |
string |
required |
The closing delimiter for the model’s final response section. Environment variable: |
string |
required |
Controls the reasoning effort level for models that separate reasoning and response automatically. Example values: Environment variable: |
|
|
Determines whether the reasoning portion returned by the model should be included in the final response provided to the application. Environment variable: |
boolean |
|
Specifies the model to use for the chat completion. A list of all available models is provided in the IBM watsonx.ai documentation at the this link. To use a model, locate the Environment variable: |
string |
|
Specifies how the model should choose which tool to call during a request. This value can be:
If Setting this value influences the tool-calling behavior of the model when no specific tool is required. Environment variable: |
|
|
Specifies the name of a specific tool that the model must call. When set, the model will be forced to call the specified tool. The name must exactly match one of the available tools defined for the service. Environment variable: |
string |
|
Positive values penalize new tokens based on their existing frequency in the generated text, reducing the likelihood of the model repeating the same lines verbatim. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies whether to return the log probabilities of the output tokens. If set to The parameter is sent to the model only when it is set. Environment variable: |
boolean |
|
An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option Possible values: Environment variable: |
int |
|
The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. Set to 0 for the model’s configured max generated tokens. Environment variable: |
int |
|
Applies a penalty to new tokens based on whether they already appear in the generated text so far, encouraging the model to introduce new topics rather than repeat itself. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Random number generator seed to use in sampling mode for experimental repeatability. Environment variable: |
int |
|
Defines one or more stop sequences that will cause the model to stop generating further tokens if any of them are encountered in the output. This allows control over where the model should end its response. If a stop sequence is encountered before the minimum number of tokens has been generated, it will be ignored. Possible values: Environment variable: |
list of string |
|
Specifies the sampling temperature to use in the generation process. Higher values (e.g. Possible values: Environment variable: |
double |
|
An alternative to sampling with The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies the desired format for the model’s output. Allowable values: Environment variable: |
|
|
Whether the JSON Schema sent to the model should use the strict mode. When enabled, the model is constrained to return a response that exactly matches the given JSON Schema. To satisfy the restrictions of the strict mode, all the properties of the schema are marked as required, the optional ones are made nullable and Set this property to Environment variable: |
boolean |
|
Whether chat model requests should be logged. Environment variable: |
boolean |
|
Whether chat model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Specifies a set of allowed output choices. When this parameter is set, the model is constrained to return exactly one of the provided choices. Environment variable: |
list of string |
|
Constrains the model output to follow a context-free grammar. If specified, the generated output will conform to the defined grammar. Environment variable: |
string |
|
Constrains the model output to match a regular expression pattern. If specified, the generated output must conform to the provided regex. Environment variable: |
string |
|
Sets the length penalty to be applied during text generation. This penalty influences the length of the generated text. A length penalty discourages the model from generating overly long responses, or conversely, it can encourage more extended outputs. When the penalty value is greater than 1.0, it discourages generating longer responses. Conversely, a value less than 1.0 incentivizes the model to generate longer text. A value of 1.0 means no penalty, and the length of the output will be determined by other factors, such as the input prompt and model’s natural completion behavior. Environment variable: |
double |
|
Sets the repetition penalty to be applied during text generation. This penalty helps to discourage the model from repeating the same words or phrases too often. The penalty value should be greater than 1.0 for repetition discouragement. A value of 1.0 means no penalty, and values above 1.0 increase the strength of the penalty. Environment variable: |
double |
|
Enables or disables reasoning. Environment variable: |
boolean |
|
The opening delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The closing delimiter for the model’s internal reasoning section. Environment variable: |
string |
required |
The opening delimiter for the model’s final response section. Environment variable: |
string |
required |
The closing delimiter for the model’s final response section. Environment variable: |
string |
required |
Controls the reasoning effort level for models that separate reasoning and response automatically. Example values: Environment variable: |
|
|
Determines whether the reasoning portion returned by the model should be included in the final response provided to the application. Environment variable: |
boolean |
|
The deployment ID of the model deployed in watsonx.ai. Setting this property routes all chat requests to the deployment chat API. Environment variable: |
string |
|
Specifies how the model should choose which tool to call during a request. This value can be:
If Setting this value influences the tool-calling behavior of the model when no specific tool is required. Environment variable: |
|
|
Specifies the name of a specific tool that the model must call. When set, the model will be forced to call the specified tool. The name must exactly match one of the available tools defined for the service. Environment variable: |
string |
|
Positive values penalize new tokens based on their existing frequency in the generated text, reducing the likelihood of the model repeating the same lines verbatim. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies whether to return the log probabilities of the output tokens. If set to The parameter is sent to the model only when it is set. Environment variable: |
boolean |
|
An integer specifying the number of most likely tokens to return at each token position, each with an associated log probability. The option Possible values: Environment variable: |
int |
|
The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. Set to 0 for the model’s configured max generated tokens. Environment variable: |
int |
|
Applies a penalty to new tokens based on whether they already appear in the generated text so far, encouraging the model to introduce new topics rather than repeat itself. The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Random number generator seed to use in sampling mode for experimental repeatability. Environment variable: |
int |
|
Defines one or more stop sequences that will cause the model to stop generating further tokens if any of them are encountered in the output. This allows control over where the model should end its response. If a stop sequence is encountered before the minimum number of tokens has been generated, it will be ignored. Possible values: Environment variable: |
list of string |
|
Specifies the sampling temperature to use in the generation process. Higher values (e.g. Possible values: Environment variable: |
double |
|
An alternative to sampling with The parameter is sent to the model only when it is set. Possible values: Environment variable: |
double |
|
Specifies the desired format for the model’s output. Allowable values: Environment variable: |
|
|
Whether the JSON Schema sent to the model should use the strict mode. When enabled, the model is constrained to return a response that exactly matches the given JSON Schema. To satisfy the restrictions of the strict mode, all the properties of the schema are marked as required, the optional ones are made nullable and Set this property to Environment variable: |
boolean |
|
Whether chat model requests should be logged. Environment variable: |
boolean |
|
Whether chat model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
The identifier of the model to use, as configured in the Model Gateway (for example Setting this property routes all chat requests to the Model Gateway. Environment variable: |
string |
|
Specifies the latency tier used to serve the request. Allowable values: Environment variable: |
|
|
Constrains the effort spent on reasoning for reasoning models. Reducing the reasoning effort can result in faster responses and fewer tokens used on reasoning. Allowable values: Environment variable: |
|
|
Whether the semantic cache is enabled. Environment variable: |
boolean |
|
The similarity threshold a cached entry must reach to be served instead of calling the model. Environment variable: |
double |
|
The output types that the model is requested to generate. Most models are only able to generate Environment variable: |
list of string |
|
Whether the generated output should be stored for model distillation or evaluations. Environment variable: |
boolean |
|
Whether the model is allowed to run tool calls in parallel. Environment variable: |
boolean |
|
A stable identifier of the end user issuing the request, used by the backing provider to detect and prevent abuse. Environment variable: |
string |
|
A set of key/value pairs that is attached to the request and returned with the response. Environment variable: |
Map<String,String> |
|
Specifies the ID of the model to be used. A list of all available models is provided in the IBM watsonx.ai documentation at the this link. To use a model, locate the Environment variable: |
string |
|
Whether embedding model requests should be logged. Environment variable: |
boolean |
|
Whether embedding model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
The id of the model to be used. All available models are listed in the IBM Watsonx.ai documentation at the link: following link. To use a model, locate the Environment variable: |
string |
|
Whether embedding model requests should be logged. Environment variable: |
boolean |
|
Whether embedding model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
Indicates whether the PII moderation model is enabled. Environment variable: |
boolean |
required |
Indicates whether the HAP moderation model is enabled. Environment variable: |
boolean |
required |
Threshold value for HAP moderation model. Environment variable: |
double |
|
Indicates whether the GraniteGuardian moderation model is enabled. Environment variable: |
boolean |
required |
Threshold value for Granite Guardian moderation model. Environment variable: |
double |
|
Whether moderation model requests should be logged. Environment variable: |
boolean |
|
Whether moderation model responses should be logged. Environment variable: |
boolean |
|
Whether the watsonx.ai client should log requests as cURL commands. Environment variable: |
boolean |
|
|
About the Duration format
To write duration values, use the standard You can also use a simplified format, starting with a number:
In other cases, the simplified format is translated to the
|