Testing Quarkus LangChain4j Models

An AI service can depend on a model provider, a memory or embedding store, and tool servers. Each dependency has its own failure path and timeout. Test which stage fails, what the caller receives, and how many model HTTP attempts the application makes.

This guide shows how to:

  • Address a model, or the AI service that uses it

  • Apply faults to a model provider

  • Count model HTTP attempts, including retries

  • Cut or stall a streamed response

  • Fault the chat memory store and the embedding stores behind an AI service

  • Fault the MCP servers an AI service calls tools on

The snippets are taken from the integration-tests/langchain4j module, some shortened; those of the last two sections from integration-tests/langchain4j-stores and integration-tests/langchain4j-mcp. The ordinary model-call scenarios run against local OpenAI, Ollama, and Anthropic API stubs, with @Timeout, @CircuitBreaker, and @Fallback around each AI service call. Streaming scenarios use the OpenAI stub; stores and MCP clients have their own test modules. The model tests do not call a paid provider or verify its billing policy.

Dependency to fault Target example What to check

Model provider

target(ChatModel.class, ModelName.Literal.of("m-openai"))

Fallback response, timeout, retries, and recovery.

Streaming model

target(OpenAiStreamingSummarizer.class)

Tokens received before failure and the consumer’s idle timeout.

Memory or embedding store

target(ChatMemoryStore.class) or target(EmbeddingStore.class)

Which client fails and whether a model call is made afterward.

MCP server

target(McpClient.class, McpClientName.Literal.of("weather"))

Tool errors, tool-call timeout, and HTTP request count after session setup.

Prerequisites

The test class uses the NetworkFaultsProfile test profile and injects NetworkFaults, as described in the Quick start.

The support for Quarkus LangChain4j is added to the build automatically when quarkus-langchain4j-core is present, and that for MCP clients when quarkus-langchain4j-mcp is. The snippets belong inside test methods unless they declare an interface or show configuration. Use Stopwatch from the extension’s JUnit module for elapsed-time assertions.

The model-call examples depend on this sample setup:

Setting or component Sample behavior

Named models

m-openai, m-ollama, and m-anthropic, each using its provider’s local stub URL.

Client timeout

quarkus.langchain4j.timeout=2s.

Application timeout

@Timeout(1000) on Summaries.summarize.

Circuit breaker

Opens after two failing service calls: requestVolumeThreshold=2, failureRatio=1.0, delay=60000.

Fallback

The resource returns 503 model unavailable when the service returns no summary.

Streaming consumer

Uses its own three-second idle timeout and returns 502 with the tokens collected before failure.

The retry counts and timing bounds below describe these integration tests. Check them against your provider, client configuration, and library version before using the same assertions.

Addressing a Model

The extension recognises provider base URLs such as quarkus.langchain4j.openai.m-openai.base-url. The model name in the client configuration, the AI service, and the target qualifier must agree. For example, m-openai is the name of this AI service’s chat model:

@RegisterAiService(modelName = "m-openai")
public interface OpenAiSummarizer {

    @UserMessage("Summarize in one sentence: {text}")
    String summarize(String text);
}
Target What it is

faults.target(ChatModel.class)

The default chat model

faults.target(ChatModel.class, ModelName.Literal.of("m-openai"))

A named chat model

faults.target(OpenAiSummarizer.class)

The chat model behind an AI service

faults.target(StreamingChatModel.class, ModelName.Literal.of("m-openai"))

The same target as the chat model of that name

Embedding, image, moderation, scoring and audio transcription models are addressed the same way, by their bean type and @ModelName.

A model target is the provider connection, not the complete set of dependencies behind an AI service. Address its memory store, embedding store, or MCP client separately when that dependency is the one you want to fail.

The Provider Is Down

try (ActiveFault f = faults.target(ChatModel.class).unavailable()) {
    when()
            .get("/summarize/default?text=long+text")
            .then()
            .statusCode(503)
            .body(is("model unavailable"));
}

unavailable() refuses connections, so the sample reaches its fallback without waiting for the full client read timeout. The extension resets circuit breakers when the fault ends, allowing a subsequent request to try the provider again. The 503 assertion checks the fallback response; it does not test the breaker’s natural recovery delay.

The Provider Hangs

Target model = faults.target(ChatModel.class, ModelName.Literal.of("m-openai"));
try (ActiveFault f = model.hangs()) {
    long ms = Stopwatch.millis(() -> when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(503));
    assertTrue(ms >= 1_900,
            "@Timeout alone cannot cut a blocking model call, took " + ms + "ms");
    assertTrue(ms < 9_000,
            "the client timeout must bound the hang, took " + ms + "ms");
}
The sample blocking client outlives the application timeout

In these tests, SmallRye Fault Tolerance interrupts the thread after one second, but the blocking model client keeps waiting. The fallback reaches the caller only after the configured two-second client timeout has ended the attempt; a client retry can add another wait. That is why the assertion allows more than two seconds but rejects an unbounded hang.

Set an explicit client timeout and verify the elapsed time seen by the caller. If you use an asynchronous AI service, test its cancellation behavior separately rather than assuming it behaves like this blocking example.

The Response Is Cut

diesMidRequest(afterBytes(40)) resets the connection after relaying 40 response bytes. The client sees an incomplete response and, in the sample setup, retries once:

Target model = faults.target(ChatModel.class, ModelName.Literal.of("m-openai"));
long requestsBefore = model.traffic().httpRequestsSeen();
try (ActiveFault f = model.diesMidRequest(afterBytes(40))) {
    when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(503);
}
// one attempt plus the retry of the client
assertEquals(2, model.traffic().httpRequestsSeen() - requestsBefore);

The count of two is an observation of the configured OpenAI, Ollama, and Anthropic clients in these tests: the first attempt plus one retry. Retry behavior can change with client settings and versions.

Two HTTP attempts do not prove that a provider processed or billed two prompts. The proxy records requests from the client, not provider-side token usage or billing. Use the count to detect unexpected retries, and provider usage data to assess cost.

The Provider Rate Limits

http().respondsWith(429) answers the model call with that status:

Target model = faults.target(ChatModel.class, ModelName.Literal.of("m-openai"));
long requestsBefore = model.traffic().httpRequestsSeen();
try (ActiveFault f = model.http().respondsWith(429)
        .contentType("application/json")
        .body("{\"error\":{\"message\":\"rate limited\"}}")) {
    when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(503)
            .body(is("model unavailable"));
}
// a 429 is retried like an I/O error
assertEquals(2, model.traffic().httpRequestsSeen() - requestsBefore);

In the sample clients, two HTTP attempts occur before a RateLimitException reaches the fallback. The test asserts the HTTP count and the application’s response; a separate failure-log assertion in LangChain4jScenariosTest checks the exception type.

To test a transient rate limit, fail only the first HTTP attempt. The sample client’s retry then reaches the healthy stub while the fault is still active:

Target model = faults.target(ChatModel.class, ModelName.Literal.of("m-openai"));
try (ActiveFault f = model.http().forFirst(1).http().respondsWith(429)) {
    when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(200)
            .body(is("summary from openai"));
}
Like every HTTP-aware fault, this needs a plaintext http:// base URL. Applying it to a model configured with an https:// base URL throws UnsupportedOperationException.

Counting Model Attempts

traffic().httpRequestsSeen() counts plaintext HTTP/1.x requests observed by the proxy, including attempts stopped before the upstream receives them. Take a baseline and assert the difference for the operation being tested; the total includes earlier calls to that target. One user interaction can cause several requests through retries, retrieval, or tool use.

The sample’s breaker opens after two failed service calls. Each of those calls can itself involve a client retry. A third service call should return the fallback without increasing the HTTP request count:

Target model = faults.target(ChatModel.class, ModelName.Literal.of("m-openai"));
try (ActiveFault f = model.diesMidRequest(onNextWrite())) {
    when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(503);

    when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(503);

    long before = model.traffic().httpRequestsSeen();
    // breaker open: no HTTP attempt sent
    when()
            .get("/summarize/openai?text=long+text")
            .then()
            .statusCode(503);
    assertEquals(before, model.traffic().httpRequestsSeen());
    String failure = when().get("/summarize/last-failure")
            .then().statusCode(200).extract().asString();
    assertTrue(failure.contains("CircuitBreakerOpen"));
}

The unchanged count confirms that the third call made no model HTTP attempt. The recorded cause confirms that the breaker, rather than another early failure, prevented it.

Streamed Responses

The streaming form of an AI service, one returning Multi<String>, is tested with the moments that read a Server-Sent Events stream. StreamingScenariosTest runs against the OpenAI stub. The stub emits summary, ` from`, and ` openai` as three token fragments, one Server-Sent Event per fragment. The resource collects them into summary from openai on success and reports the partial list on failure:

One SSE event is not necessarily one token in a real provider response. Use the event count and payload format of your test fixture when choosing a fault moment. 
Target model = faults.target(OpenAiStreamingSummarizer.class);
try (ActiveFault f = model.diesMidRequest(afterSseEvents(1))) { (1)
    when()
            .get("/summarize/stream/openai?text=long+text")
            .then()
            .statusCode(502)
            .body(containsString(" after [summary]"));
}

try (ActiveFault f = model.hangs(afterSseEvents(1))) { (2)
    when()
            .get("/summarize/stream/openai?text=long+text")
            .then()
            .statusCode(502)
            .body(containsString(" after [summary]"));
}
1 The stream is cut after the first event, containing summary. The sample streaming client does not retry after that fragment is emitted; the source test verifies one HTTP attempt.
2 The stream stalls after the same event. The resource’s three-second idle timeout ends the wait.

After each fault, the source test makes a healthy request and asserts the complete summary. Keep that recovery check when adapting the scenario.

In the sample’s streaming client, the request read timeout is cancelled once response headers arrive. The resource adds ifNoItem().after(Duration.ofSeconds(3)).fail() to the token Multi to bound silence between items. That idle timeout resets on each item and does not limit the duration of a continuously active stream. See Testing Streamed Responses for the consumer code and timing assertions.

Memory and Embedding Stores

Choose the target of the store operation you want to fail:

  • Redis and MongoDB chat memory stores resolve to their underlying Redis or MongoDB client.

  • Pgvector and Oracle embedding stores resolve to their datasource; a Redis embedding store resolves to its Redis client.

  • Chroma has its own HTTP target, configured through quarkus.langchain4j.chroma[.<store>].url and named langchain4j:chroma:<store>.

A store and its underlying client can therefore be two ways of addressing the same target. A fault on that target can affect other application operations using the same client or datasource too. Only plaintext HTTP stores, such as a Chroma test endpoint, take HTTP-aware faults like respondsWith(…​); use connection faults for database or Redis targets.

Target What it is

faults.target(ChatMemoryStore.class)

The client of the chat memory store, redis:<default> for the Redis store on the default client

faults.target(EmbeddingStore.class)

The default embedding store: the datasource behind a pgvector store, datasource:<default>:jdbc, the client behind a Redis store

faults.target(EmbeddingStore.class, EmbeddingStoreName.Literal.of("c1"))

A named store, langchain4j:chroma:c1 for a Chroma one

In the sample chat service, memory is read before the model is called. Fail the memory store and check that the model’s request count stays unchanged:

Target model = faults.target(ChatModel.class);
long promptsBefore = model.traffic().httpRequestsSeen();
try (ActiveFault f = faults.target(ChatMemoryStore.class).unavailable()) {
    when()
            .get("/chat/s1?text=hello")
            .then()
            .statusCode(503);
}
// the failed memory read prevented any model HTTP attempt
assertEquals(promptsBefore, model.traffic().httpRequestsSeen());

The model client timeout does not govern a Redis memory operation. The sample’s hung Redis memory store is bounded by quarkus.redis.timeout; configure and test the timeout of whichever client your store uses.

The integration-tests/langchain4j-stores module runs the Redis memory store, a pgvector store and a Chroma store, all from Dev Services, and checks that each store and the client behind it are one target.

With several embedding store extensions in one application, a named store belongs to the extension whose configuration names it, and the default store to the extension whose default store is enabled. When that does not single one out, target says so; address the datasource or client behind the store then. The Infinispan, Neo4j, Qdrant, Milvus, Weaviate, Pinecone and Hibernate stores are not recognised yet.

MCP Servers

An MCP server is a dependency an AI service calls in the middle of a conversation, when the model asks for a tool. It fails like any other HTTP dependency: it is down, it hangs, it answers with an error. Decide which layer your test exercises: a resource calling McpClient directly, or an AI service using that client during tool execution. The examples below call the MCP client directly; they do not run a complete AI conversation.

The integration-tests/langchain4j-mcp module lists and calls tools on a local stub using streamable HTTP. Its resource returns 503 if the client throws or returns a tool result with isError() == true.

Addressing an MCP Server

A client over the streamable HTTP or the WebSocket transport is configured with a URL, and that property is recognised as the target mcp:<client>:

quarkus.langchain4j.mcp.weather.transport-type=streamable-http
quarkus.langchain4j.mcp.weather.url=http://localhost:8081/mcp
quarkus.langchain4j.mcp.weather.tool-execution-timeout=2s
quarkus.langchain4j.mcp.weather.auto-health-check=false
quarkus.langchain4j.mcp.weather.protocol-version=2025-11-25

The URL is illustrative; the source test obtains its actual port from McpServerStub. The protocol version matches that stub. Choose the version supported by your test server.

The server is addressed by its client bean and @McpClientName:

Target server = faults.target(McpClient.class, McpClientName.Literal.of("weather"));

Every MCP client has a name, so faults.target(McpClient.class) alone is refused. A client over stdio has no network URL and is not a network fault target.

Before each failure scenario, initialise the session while the server is healthy:

when().get("/tools").then().statusCode(200).body(is("forecast"));

In the sample, /tools calls weather.listTools(). Warming up before applying the fault makes the scenario about an established client’s tool call, rather than a failure during initialisation. The snippets below assume this warm-up and the server target declared above.

The Server Is Down

try (ActiveFault f = server.unavailable()) {
    when()
            .post("/tools/forecast?city=Athens")
            .then()
            .statusCode(503);
}

The direct client call fails and the sample resource maps it to 503. For an AI service, tool errors may instead be passed back to the model as tool results, allowing the conversation to continue. Test your service’s tool-error policy and count model HTTP attempts separately; the direct-client assertion above does not establish the AI service’s final response.

The Server Hangs

try (ActiveFault f = server.hangs()) {
    long ms = Stopwatch.millis(() -> when()
            .post("/tools/forecast?city=Athens")
            .then()
            .statusCode(503)
            .body(containsStringIgnoringCase("timeout")));
    assertTrue(ms >= 1_900, "the tool call must wait for tool-execution-timeout, took " + ms + "ms");
    assertTrue(ms < 9_000, "tool-execution-timeout must bound the hang, took " + ms + "ms");
}

The sample explicitly sets quarkus.langchain4j.mcp.weather.tool-execution-timeout=2s. The timing bounds check that the tool timeout governs this wait, with slack for cleanup and scheduling. In this scenario the client returns an error result instead of throwing, and the resource inspects isError() to produce 503. A resource that checks exceptions only could accidentally treat that timeout as a successful tool response.

The Server Answers With an Error

long before = server.traffic().httpRequestsSeen();
try (ActiveFault f = server.http().respondsWith(500)) {
    when()
            .post("/tools/forecast?city=Athens")
            .then()
            .statusCode(503);
}
assertEquals(1, server.traffic().httpRequestsSeen() - before);

Counting Tool Calls

In this fixture, a tool call sends one HTTP request once the session is established. Initialising the streamable HTTP client sends two additional requests, which is why the tests call listTools() before taking a counter baseline.

The configuration above disables background health checks and pins the protocol version supported by the stub. Without those settings, health checks or protocol probes can increase the count independently of the tool call. Use a counter delta after warm-up, and keep other calls to the same target out of the measured interval.

Adapting the Examples

For each dependency, make the test’s expected outcome explicit:

  • Match the model or client name to its configuration and confirm that the target sees the calls you expect.

  • Separate application timeouts, client timeouts, stream idle timeouts, and tool execution timeouts.

  • Assert the HTTP attempt count your retry policy allows, using a baseline taken after any required warm-up.

  • Check partial data and recovery as well as the failure response.

  • Test stores and tool servers independently of model failures, then add an AI-service test for their effect on the full conversation.

Going Further