Enabling and Disabling Integrations

By default, all integrations with AI providers (OpenAI, HuggingFace, Azure OpenAI, etc.) are enabled. This means that live calls are made to the configured AI provider.

Each provider has an enable-integration property (i.e. quarkus.langchain4j.openai.enable-integration, quarkus.langchain4j.huggingface.enable-integration, etc.) that can be set to false to disable the integration. This property is read at runtime.

When disabled, any call made to the AI provider will end up in an dev.langchain4j.model.ModelDisabledException runtime exception being thrown.

In this case your AI Service could use fault tolerance to handle this and serve a fallback response without making a live call to the AI provider. Below are a few examples of how it could be used:

Handling disabled integration separately from other error conditions
package io.quarkiverse.langchain4j.samples;

import org.eclipse.microprofile.faulttolerance.Fallback;

import dev.langchain4j.model.ModelDisabledException;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService
public interface AiServiceWithFaultToleranceAndOnDisabledIntegration {
    @SystemMessage("You are a professional poet")
    @UserMessage("Write a poem about {topic}. The poem should be {lines} lines long.")
    @Fallback(fallbackMethod = "fallback")
    String writeAPoem(String topic, int lines);

    // This fallback is only called when a ModelDisabledException is thrown due to disabled integration
    default String fallback(String topic, int lines, ModelDisabledException mde) {
        return "I'm sorry, but the integration with the AI provider because " + mde.getMessage();
    }

    // This fallback is called for any other exception, except for ModelDisabledException
    default String fallback(String topic, int lines) {
        return "I'm sorry, I can't write a poem about " + topic;
    }
}
Handling only disabled integration
package io.quarkiverse.langchain4j.samples;

import org.eclipse.microprofile.faulttolerance.Fallback;

import dev.langchain4j.model.ModelDisabledException;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService
public interface AiServiceWithFaultToleranceOnlyOnDisabledIntegration {
    @SystemMessage("You are a professional poet")
    @UserMessage("Write a poem about {topic}. The poem should be {lines} lines long.")
    @Fallback(fallbackMethod = "fallback", applyOn = ModelDisabledException.class)
    String writeAPoem(String topic, int lines);

    // In this case, the fallback is only applied if a ModelDisabledException is thrown
    default String fallback(String topic, int lines) {
        return "I'm sorry, I can't write a poem about " + topic;
    }
}