End-to-end example: prove your resilience annotations work

This walkthrough uses the sample application of the integration-tests module to verify, for real, that MicroProfile Fault Tolerance annotations react to failures: a @Timeout on a slow business method, then a @Retry + @Fallback on a failing database.

A fault injected at the inbound REST boundary (HTTP_IN) hits the request before the resource runs, so the Fault Tolerance annotations below it never see it. Both scenarios therefore arm a deeper layer — SERVICE or DATABASE — whose faults are injected inside or below the guarded methods (see Chaos layers).

Start the sample application

From the root of the repository, install the extension once, then start the sample application in dev mode:

mvn install -DskipTests
mvn -pl integration-tests quarkus:dev

Open the Dev UI at http://localhost:8080/q/dev and the Goblin Chaos Dashboard. The sample application has an in-memory H2 datasource, so every layer is available. Its application.properties starts with chaos active and the Latency assault enabled (100-500 ms, level 100).

The dashboard settings are saved to integration-tests/.goblin-state.json and restored at the next start instead of application.properties. If you already used the dashboard, the toggles, layers and parameters below may differ from the defaults: delete the file before starting, or use Reset to defaults in the Danger zone (latency on with 100-5000 ms, java.lang.RuntimeException, layers Inbound REST + Outbound HTTP).

Scenario 1: a slow service triggers @Timeout

SampleService.timed() is guarded by @Timeout(400ms); GET /api/service/timeout calls it.

  1. In Chaos layers, switch Service on and make sure Database is off (the default Inbound REST and Outbound HTTP can stay on). At level 100 the deepest armed layer wins the whole request: with Database armed, this request — which never touches the database — would resolve to the database layer and not be assaulted at all.

  2. Latency is already enabled: set its range to 600-700 ms, above the 400 ms timeout, and click Save. Keep the level at 100.

  3. Call the endpoint:

    curl -w "\nstatus=%{http_code} time=%{time_total}s\n" http://localhost:8080/api/service/timeout

  4. The call fails with a 500 after about 0.4 s instead of 0.6 s: @Timeout interrupted the injected delay. The Assault History shows a latency entry on io.quarkiverse.goblin.it.SampleService.timed whose duration is the delay actually endured (about 400 ms), and the status bar counts it under service.

Scenario 2: a failing database triggers @Retry then @Fallback

SampleService.databasePingWithRetry() is guarded by @Retry(maxRetries = 2) and @Fallback; it calls SampleRepository, which acquires a JDBC connection. GET /api/db/retry calls it.

  1. In Chaos layers, switch Database on.

  2. Disable Latency and enable Exception (the default java.lang.RuntimeException is fine; any exception class works, since @Retry retries on every exception by default). Keep the level at 100.

  3. Call the endpoint:

    curl -w "\nstatus=%{http_code}\n" http://localhost:8080/api/db/retry

  4. The response is 200 with database fallback reply. The Assault History shows three exception entries on Database <default> connection: the first attempt and the two retries each acquired a connection and failed, then the fallback answered; the status bar counts them under database. Compare with GET /api/db/ping, which has no Fault Tolerance annotation and fails with a 500.

Make it intermittent

Set the level to 30 and call the endpoints repeatedly: some attempts now succeed, so @Retry recovers before the fallback, and only part of the calls hit the timeout — the intermittent, non-deterministic failures that are the hardest to get right in production.

The same scenarios can run in a @QuarkusTest: set quarkus.goblin.test.enabled=true, arm the layer and the assault through the injected AssaultEngine (engine.getMutableConfig()), and assert on the response and on engine.getHistory(). See GoblinServiceLayerIntegrationTest and GoblinDatabaseLayerIntegrationTest in the integration-tests module.