Making AI interactions more robust
When implementing an AI service, leveraging Quarkus fault tolerance annotations can enhance the reliability of your interactions. To integrate fault tolerance, follow these steps:
-
1. Start by adding the
quarkus-smallrye-fault-tolerance
dependency to yourpom.xml
file:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-fault-tolerance</artifactId>
</dependency>
-
2. Utilize the
@Fallback
or@Timeout
annotation within theAIService
class. These annotations help manage exceptional scenarios:
package io.quarkiverse.langchain4j.samples;
import java.time.temporal.ChronoUnit;
import org.eclipse.microprofile.faulttolerance.Fallback;
import org.eclipse.microprofile.faulttolerance.Timeout;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;
@RegisterAiService
public interface AiServiceWithFaultTolerance {
@SystemMessage("You are a professional poet")
@UserMessage("Write a poem about {topic}. The poem should be {lines} lines long.")
@Timeout(value = 60, unit = ChronoUnit.SECONDS)
@Fallback(fallbackMethod = "fallback")
String writeAPoem(String topic, int lines);
default String fallback(String topic, int lines) {
return "I'm sorry, I can't write a poem about " + topic;
}
}
A Note on Utilizing Tools
When employing tools in your interactions, multiple requests and responses occur in the background, potentially extending the time required to obtain a response from the AI service. It’s essential to consider this while configuring a timeout to ensure robustness in your system’s interactions.