Anthropic

Anthropic is an AI safety and research company based in San Francisco which provides the Claude LLM models.

Using Anthropic Models

To employ Anthropic LLMs, integrate the following dependency into your project:

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-anthropic</artifactId>
    <version>0.22.0</version>
</dependency>

If no other LLM extension is installed, AI Services will automatically utilize the configured Anthropic model.

Configuration

Configuring Anthropic models mandates an API key, obtainable by creating an account on the Claude platform.

The API key can be set in the application.properties file:

quarkus.langchain4j.anthropic.api-key=...
Alternatively, leverage the QUARKUS_LANGCHAIN4J_ANTHROPIC_API_KEY environment variable.

Several configuration properties are available:

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

Whether the model should be enabled

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_ENABLED

boolean

true

Base URL of the Anthropic API

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_BASE_URL

string

https://api.anthropic.com/v1/

Anthropic API key

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_API_KEY

string

dummy

The Anthropic version

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_VERSION

string

2023-06-01

Timeout for Anthropic calls

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_TIMEOUT

Duration

10s

Whether the Anthropic client should log requests

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_LOG_REQUESTS

boolean

false

Whether the Anthropic client should log responses

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_LOG_RESPONSES

boolean

false

Whether to enable the integration. Defaults to true, which means requests are made to the Anthropic provider. Set to false to disable all requests.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_ENABLE_INTEGRATION

boolean

true

Model name to use

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_MODEL_NAME

string

claude-3-haiku-20240307

What sampling temperature to use, between 0.0 and 1.0. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

It is generally recommended to set this or the top-k property but not both.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_TEMPERATURE

double

0.7

The maximum number of tokens to generate in the completion.

The token count of your prompt plus max_tokens cannot exceed the model’s context length

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_MAX_TOKENS

int

1024

Double (0.0-1.0). Nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

It is generally recommended to set this or the temperature property but not both.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_TOP_P

double

1.0

Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_TOP_K

int

40

The maximum number of times to retry. 1 means exactly one attempt, with retrying disabled.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_MAX_RETRIES

int

1

The custom text sequences that will cause the model to stop generating

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_STOP_SEQUENCES

list of string

Whether chat model requests should be logged

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_LOG_REQUESTS

boolean

false

Whether chat model responses should be logged

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC_CHAT_MODEL_LOG_RESPONSES

boolean

false

Named model config

Type

Default

Base URL of the Anthropic API

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__BASE_URL

string

https://api.anthropic.com/v1/

Anthropic API key

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__API_KEY

string

dummy

The Anthropic version

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__VERSION

string

2023-06-01

Timeout for Anthropic calls

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__TIMEOUT

Duration

10s

Whether the Anthropic client should log requests

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__LOG_REQUESTS

boolean

false

Whether the Anthropic client should log responses

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__LOG_RESPONSES

boolean

false

Whether to enable the integration. Defaults to true, which means requests are made to the Anthropic provider. Set to false to disable all requests.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__ENABLE_INTEGRATION

boolean

true

Model name to use

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_MODEL_NAME

string

claude-3-haiku-20240307

What sampling temperature to use, between 0.0 and 1.0. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

It is generally recommended to set this or the top-k property but not both.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_TEMPERATURE

double

0.7

The maximum number of tokens to generate in the completion.

The token count of your prompt plus max_tokens cannot exceed the model’s context length

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_MAX_TOKENS

int

1024

Double (0.0-1.0). Nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

It is generally recommended to set this or the temperature property but not both.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_TOP_P

double

1.0

Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_TOP_K

int

40

The maximum number of times to retry. 1 means exactly one attempt, with retrying disabled.

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_MAX_RETRIES

int

1

The custom text sequences that will cause the model to stop generating

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_STOP_SEQUENCES

list of string

Whether chat model requests should be logged

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_LOG_REQUESTS

boolean

false

Whether chat model responses should be logged

Environment variable: QUARKUS_LANGCHAIN4J_ANTHROPIC__MODEL_NAME__CHAT_MODEL_LOG_RESPONSES

boolean

false

About the Duration format

To write duration values, use the standard java.time.Duration format. See the Duration#parse() Java API documentation for more information.

You can also use a simplified format, starting with a number:

  • If the value is only a number, it represents time in seconds.

  • If the value is a number followed by ms, it represents time in milliseconds.

In other cases, the simplified format is translated to the java.time.Duration format for parsing:

  • If the value is a number followed by h, m, or s, it is prefixed with PT.

  • If the value is a number followed by d, it is prefixed with P.