Enabling or Disabling AI Model Integrations

By default, all AI provider integrations (e.g., OpenAI, HuggingFace, Azure OpenAI) are enabled. This means that live API calls are made to the respective providers when an AI service is invoked.

You can disable any provider at runtime using the corresponding configuration property:

  • quarkus.langchain4j.openai.enable-integration

  • quarkus.langchain4j.huggingface.enable-integration

  • quarkus.langchain4j.azure-openai.enable-integration

Setting this property to false will prevent any live call to the model, and a call to the provider will result in a dev.langchain4j.model.ModelDisabledException being thrown.

It disables the whole integration, so all the models provided by the extension are disabled.

In such cases, your AI Service can leverage fault tolerance mechanisms to handle this exception and provide a fallback response.

The following examples illustrate how to handle disabled integrations using fault-tolerant within AI services:

Handling disabled integration separately from other error conditions
package io.quarkiverse.langchain4j.samples;

import org.eclipse.microprofile.faulttolerance.Fallback;

import dev.langchain4j.model.ModelDisabledException;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService
public interface AiServiceWithFaultToleranceAndOnDisabledIntegration {
    @SystemMessage("You are a professional poet")
    @UserMessage("Write a poem about {topic}. The poem should be {lines} lines long.")
    @Fallback(fallbackMethod = "fallback")
    String writeAPoem(String topic, int lines);

    // This fallback is only called when a ModelDisabledException is thrown due to disabled integration
    default String fallback(String topic, int lines, ModelDisabledException mde) {
        return "I'm sorry, but the integration with the AI provider because " + mde.getMessage();
    }

    // This fallback is called for any other exception, except for ModelDisabledException
    default String fallback(String topic, int lines) {
        return "I'm sorry, I can't write a poem about " + topic;
    }
}
Handling only disabled integration
package io.quarkiverse.langchain4j.samples;

import org.eclipse.microprofile.faulttolerance.Fallback;

import dev.langchain4j.model.ModelDisabledException;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService
public interface AiServiceWithFaultToleranceOnlyOnDisabledIntegration {
    @SystemMessage("You are a professional poet")
    @UserMessage("Write a poem about {topic}. The poem should be {lines} lines long.")
    @Fallback(fallbackMethod = "fallback", applyOn = ModelDisabledException.class)
    String writeAPoem(String topic, int lines);

    // In this case, the fallback is only applied if a ModelDisabledException is thrown
    default String fallback(String topic, int lines) {
        return "I'm sorry, I can't write a poem about " + topic;
    }
}