Structured Logging

Quarkus Flow can emit all workflow and task lifecycle events as structured JSON logs to stdout. This enables you to export complete workflow execution data to external databases, analytics platforms, or audit systems without coupling your runtime to specific storage technologies.

Overview

Structured logging follows the logs-as-transport pattern: your workflow runtime emits JSON events to stdout, and a log forwarder (like FluentBit or Vector) routes those events to your chosen destination—PostgreSQL, Elasticsearch, S3, Kafka, or any combination.

This approach provides several advantages:

  • Zero transport ownership – Your application doesn’t manage database connections, retries, or buffering. The log forwarder handles all transport concerns.

  • Flexibility – The same log stream can feed multiple destinations simultaneously (e.g., PostgreSQL for queries + S3 for compliance archives).

  • Clear support boundary – If data isn’t reaching your database, the issue is either "logs not being emitted" (your code) or "logs not being forwarded" (infrastructure). No ambiguity.

  • Scalability – Log forwarders are designed for high-volume event streaming and can scale independently of your application.

Use Cases

  • Query APIs – Build GraphQL or REST APIs on top of PostgreSQL to query workflow instances and execution history.

  • Analytics – Feed workflow events into data warehouses (BigQuery, Snowflake, Redshift) for business intelligence.

  • Compliance auditing – Maintain long-term audit trails with detailed execution history.

  • Custom dashboards – Power monitoring UIs with workflow execution data from Elasticsearch.

  • Event-driven integrations – Stream events to Kafka for downstream processing.

Configuration

Structured logging is disabled by default. Enable it via configuration:

# Enable structured logging (REQUIRED)
quarkus.flow.structured-logging.enabled=true

# Event filtering (default: all events)
quarkus.flow.structured-logging.events=workflow.*

# Payload inclusion (default: workflow payloads included, task payloads excluded)
quarkus.flow.structured-logging.include-workflow-payloads=true
quarkus.flow.structured-logging.include-task-payloads=false

# Always include full context in error events (default: true)
quarkus.flow.structured-logging.include-error-context=true

# Truncation for large payloads (default: 10KB)
quarkus.flow.structured-logging.payload-max-size=10240
quarkus.flow.structured-logging.truncate-preview-size=1024

# Log level (default: INFO)
quarkus.flow.structured-logging.log-level=INFO

# Timestamp format (default: ISO8601)
quarkus.flow.structured-logging.timestamp-format=iso8601
# For custom format, specify the pattern (java.time.format.DateTimeFormatter)
# quarkus.flow.structured-logging.timestamp-pattern=yyyy-MM-dd'T'HH:mm:ss.SSSXXX
When you enable structured logging, Quarkus Flow automatically configures a log handler based on the handler.mode setting. By default, events are written to a file (target/quarkus-flow-events.log in dev/test mode, /var/log/quarkus-flow/events.log in production). For containerized deployments, use handler.mode=container to write to stdout instead. See Handler Mode Configuration for details.

Handler Mode Configuration

Quarkus Flow provides three handler modes to control where structured logging events are written. This allows you to adapt the logging output to your deployment environment.

FILE Mode (Default)

Writes events to a dedicated file, separate from application logs:

quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.handler.mode=file  # Default, can be omitted

File locations:

  • Dev/Test mode: target/quarkus-flow-events.log

  • Production mode: /var/log/quarkus-flow/events.log

When to use FILE mode:

  • Running on traditional servers or VMs

  • You want events in a separate file from application logs

  • Using file-based log forwarders (Filebeat, FluentBit with file input)

  • File system is writable and persistent

CONTAINER Mode

Writes events to stdout for containerized deployments (Kubernetes, Docker, Podman):

quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.handler.mode=container
quarkus.flow.structured-logging.timestamp-format=epoch-seconds  # Recommended for container environments

Benefits:

  • No file write permissions needed

  • Logs automatically captured by container runtime (kubectl logs, docker logs)

  • Compatible with log aggregation tools (FluentBit, Fluentd, Promtail, Vector)

  • Works with read-only filesystems

  • Follows cloud-native logging best practices

  • Simpler configuration (3 lines vs 8 lines for manual workaround)

When to use CONTAINER mode:

  • Running in containers (Docker, Kubernetes, Podman, OpenShift)

  • Using container runtime log collection

  • Log aggregation tools collect from stdout

  • File systems are ephemeral or read-only

  • Following twelve-factor app principles

Example Kubernetes deployment:

With CONTAINER mode, your logs flow like this:

  1. Quarkus Flow writes JSON events to stdout

  2. Container runtime captures stdout → /var/log/containers/*.log

  3. Log forwarder (FluentBit, Fluentd) reads from /var/log/containers/

  4. Events routed to destination (PostgreSQL, Elasticsearch, S3, Kafka)

No special volume mounts or file permissions needed!

NONE Mode

Disables automatic handler creation for advanced use cases:

quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.handler.mode=none

# You must manually configure handlers
quarkus.log.handler.console."CUSTOM_FLOW".enable=true
quarkus.log.handler.console."CUSTOM_FLOW".format=%s%n
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=CUSTOM_FLOW
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

When to use NONE mode:

  • You need full control over logging configuration

  • Using custom handlers (syslog, GELF, custom formatters)

  • Integrating with specialized logging infrastructure

  • Default handlers don’t fit your deployment requirements

The logger category for structured events is io.quarkiverse.flow.structuredlogging.

Timestamp Format Configuration

Different log processors and downstream systems have varying requirements for timestamp formats. Quarkus Flow allows you to configure how timestamps are formatted in structured logging events.

# ISO 8601 format (default) - human-readable, widely compatible
quarkus.flow.structured-logging.timestamp-format=iso8601

# Unix epoch seconds with fractional nanoseconds (e.g., 1776807366.427833)
# Best for: PostgreSQL TIMESTAMP WITH TIME ZONE, InfluxDB
quarkus.flow.structured-logging.timestamp-format=epoch-seconds

# Unix epoch milliseconds as long (e.g., 1776807366428)
# Best for: Elasticsearch date fields, Kafka
quarkus.flow.structured-logging.timestamp-format=epoch-millis

# Unix epoch nanoseconds as long (e.g., 1776807366427832969)
# Best for: High-precision time-series databases
quarkus.flow.structured-logging.timestamp-format=epoch-nanos

# Custom format using java.time.format.DateTimeFormatter pattern
quarkus.flow.structured-logging.timestamp-format=custom
quarkus.flow.structured-logging.timestamp-pattern=yyyy-MM-dd'T'HH:mm:ss.SSSXXX

Format Examples:

Format Example Value

iso8601

"2026-04-21T21:36:06.427832969Z"

epoch-seconds

1776807366.427833

epoch-millis

1776807366428

epoch-nanos

1776807366427832969

custom

"2026-04-21T21:36:06.427-03:00"
The timestamp format applies to all timestamp fields in events: timestamp, startTime, endTime, and lastUpdateTime.

Event Filtering

Control which events are logged using glob patterns:

# All events (default)
quarkus.flow.structured-logging.events=workflow.*

# Only workflow-level events (no task details)
quarkus.flow.structured-logging.events=workflow.instance.*

# Workflow events + task failures (recommended for most use cases)
quarkus.flow.structured-logging.events=workflow.instance.*,workflow.task.faulted

# Specific events only
quarkus.flow.structured-logging.events=\
  workflow.instance.started,\
  workflow.instance.completed,\
  workflow.instance.faulted

Payload Inclusion Strategy

By default, structured logging captures execution graphs (what executed when) but not task payloads (input/output data). This keeps log volume low while providing enough information for execution visualization.

Default behavior:

  • Workflow events: Include input/output (needed for instance queries)

  • Task events: Only metadata (taskName, position, status, timing)

  • Error events: Always include full context (overrides task payload setting)

This produces ~5KB of logs per workflow (compared to 50-500KB if all task payloads were included).

When to enable task payloads:

# Enable full audit trail with all task input/output
quarkus.flow.structured-logging.include-task-payloads=true

Use this for:

  • Compliance requirements mandating complete execution records

  • Debugging specific workflows in non-production environments

  • Workflows with small payloads where volume isn’t a concern

Large Payload Handling

For agentic workflows with large contexts (conversation history, retrieved documents, etc.), payloads exceeding the configured threshold are automatically truncated:

{
  "input": {
    "__truncated__": true,
    "__originalSize__": 157000,
    "__preview__": "First 1KB of data..."
  }
}

This prevents overwhelming log systems while preserving metadata about what was truncated.

Event Schema

All events follow a consistent JSON schema:

{
  "eventType": "workflow.instance.started",
  "timestamp": "2026-04-13T14:30:00.123Z",  // Format depends on configuration
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "workflowNamespace": "default",
  "workflowName": "greetings",
  "workflowVersion": "1.0.0",
  ...event-specific fields...
}
Timestamp fields can be formatted as ISO 8601 strings, Unix epoch values, or custom formats depending on your timestamp format configuration.

Workflow Instance Events

  • workflow.instance.started – Workflow execution begins

  • workflow.instance.completed – Workflow finishes successfully

  • workflow.instance.faulted – Workflow fails with error

  • workflow.instance.cancelled – Workflow is cancelled

  • workflow.instance.suspended – Workflow is suspended (waiting)

  • workflow.instance.resumed – Workflow resumes after suspension

  • workflow.instance.status.changed – Workflow status changes

Task Events

  • workflow.task.started – Task execution begins

  • workflow.task.completed – Task finishes successfully

  • workflow.task.faulted – Task fails with error

  • workflow.task.cancelled – Task is cancelled

  • workflow.task.suspended – Task is suspended

  • workflow.task.resumed – Task resumes after suspension

  • workflow.task.retried – Task is retried after failure

Example Events

Workflow Started (ISO 8601 format):

{
  "eventType": "io.serverlessworkflow.workflow.started.v1",
  "timestamp": "2026-04-13T14:30:00.123Z",
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "workflowNamespace": "default",
  "workflowName": "greetings",
  "workflowVersion": "1.0.0",
  "status": "RUNNING",
  "startTime": "2026-04-13T14:30:00.123Z",
  "input": {
    "name": "Alice"
  }
}

Workflow Started (epoch-seconds format):

{
  "eventType": "io.serverlessworkflow.workflow.started.v1",
  "timestamp": 1744642200.123,
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "workflowNamespace": "default",
  "workflowName": "greetings",
  "workflowVersion": "1.0.0",
  "status": "RUNNING",
  "startTime": 1744642200.123,
  "input": {
    "name": "Alice"
  }
}

Workflow Failed:

{
  "eventType": "io.serverlessworkflow.workflow.faulted.v1",
  "timestamp": "2026-04-13T14:30:05.789Z",
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "FAULTED",
  "endTime": "2026-04-13T14:30:05.789Z",
  "error": {
    "message": "Service unavailable",
    "type": "java.net.ConnectException",
    "stackTrace": "..."
  },
  "input": {
    "name": "Alice"
  }
}

Task Started (no payloads):

{
  "eventType": "io.serverlessworkflow.task.started.v1",
  "timestamp": "2026-04-13T14:30:01.000Z",
  "taskExecutionId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "taskName": "callGreetingService",
  "taskPosition": "do/0",
  "status": "RUNNING",
  "startTime": "2026-04-13T14:30:01.000Z"
}

Integration with quarkus-logging-json

When you use quarkus-logging-json in your application, Quarkus Flow automatically configures a separate file handler for structured events to avoid double JSON serialization.

The Problem (and Automatic Solution)

When quarkus-logging-json is enabled, it wraps all log messages in a JSON structure:

{
  "timestamp": "2026-04-13T21:00:40.475075-03:00",
  "level": "ERROR",
  "loggerName": "io.quarkiverse.flow.structuredlogging",
  "message": "{\"instanceId\":\"...\",\"eventType\":\"...\"}", // ← JSON string inside JSON
  "threadName": "pool-11-thread-1"
}

Notice how the message field contains a JSON string, not a JSON object. This requires log consumers to:

  1. Parse the outer JSON (from quarkus-logging-json)

  2. Parse the inner JSON string (our workflow event)

This "double serialization" defeats the purpose of structured logging.

The Solution: Automatic Handler Configuration

Quarkus Flow automatically configures a log handler when structured logging is enabled. No manual configuration required!

Default Configuration (FILE mode):

When you enable structured logging:

# Enable Quarkus Flow structured logging
quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.events=workflow.*
# handler.mode=file is the default

Quarkus Flow automatically:

  • Creates a file handler named FLOW_EVENTS

  • Writes events to:

    • Dev/Test mode: target/quarkus-flow-events.log

    • Production mode: /var/log/quarkus-flow/events.log

  • Uses raw JSON format (%s%n - no timestamps, just the event JSON)

  • Prevents events from appearing in console (no double logging)

Alternative: CONTAINER mode for containerized deployments:

If you’re running in containers and want events in stdout instead:

quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.handler.mode=container

This writes events to stdout instead of a file. See Handler Mode Configuration for details.

Customizing the Configuration:

You can override any of the defaults:

# Custom file path
quarkus.log.handler.file."FLOW_EVENTS".path=/custom/path/workflow-events.log

# Or disable the file handler and use console only (not recommended with quarkus-logging-json)
quarkus.log.handler.file."FLOW_EVENTS".enable=false
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=true
Quarkus Flow handles formatting programmatically at runtime. Even if quarkus-logging-json takes over the global loggers, Quarkus Flow automatically pierces through the logging wrappers to strictly enforce raw string formatting for the FLOW_EVENTS file handler. This guarantees that your event stream remains pure, single-level JSON without any configuration effort on your part.

With the default configuration:

  • Console logs: Application logs in JSON format (from quarkus-logging-json)

  • Event file: Pure workflow event JSON (one event per line, safe from double-wrapping)

Why Separate Files?

This follows the logs-as-transport pattern correctly:

  • Application logs: Diagnostic information for debugging (stdout/stderr)

  • Event streams: Structured data for analytics/auditing (dedicated file)

Event streams and diagnostic logs serve different purposes and should be treated separately. Log forwarders can then: - Parse application logs with the appropriate schema - Parse workflow events as pure JSON - Route each to different destinations (e.g., Elasticsearch for logs, PostgreSQL for events)

Automatic Detection and Configuration

Quarkus Flow automatically detects quarkus-logging-json at build time and: - Emits an informational message about the auto-configured file handler - Shows the default file path being used - Reminds you that the path can be customized

Check your build logs for:

INFO  [io.qua.flo.dep.FlowProcessor] Quarkus Flow structured logging file handler auto-configured.
      Events will be written to: target/quarkus-flow-events.log
      (override with quarkus.log.handler.file."FLOW_EVENTS".path)

Log Forwarder Integration

FluentBit Example

FluentBit is a lightweight, high-performance log forwarder. Configuration depends on your handler mode:

FILE Mode Configuration

When using handler.mode=file (default), FluentBit reads from the log file:

[INPUT]
    Name              tail
    Path              /var/log/quarkus-flow/events.log  # Default production path
    Parser            json
    Tag               flow.events

[FILTER]
    Name              modify
    Match             flow.events
    Add               kubernetes.namespace ${K8S_NAMESPACE}
    Add               kubernetes.pod ${K8S_POD_NAME}

[OUTPUT]
    Name              pgsql
    Match             flow.events
    Host              postgres.database.svc
    Port              5432
    User              flowuser
    Password          ${DB_PASSWORD}
    Database          workflow_data
    Table             workflow_events
    Timestamp_Key     timestamp

CONTAINER Mode Configuration (Recommended for Kubernetes/Docker)

When using handler.mode=container, FluentBit collects logs from stdout via the container runtime:

[INPUT]
    Name              tail
    Path              /var/log/containers/*_${NAMESPACE}_${POD_NAME}-*.log
    Parser            docker  # Or cri/containerd depending on runtime
    Tag               kube.*

[FILTER]
    Name              kubernetes
    Match             kube.*
    Kube_URL          https://kubernetes.default.svc:443
    Merge_Log         On
    Keep_Log          Off

[FILTER]
    Name              grep
    Match             kube.*
    Regex             log {"eventType":"io.serverlessworkflow.*

[OUTPUT]
    Name              pgsql
    Match             kube.*
    Host              postgres.database.svc
    Port              5432
    User              flowuser
    Password          ${DB_PASSWORD}
    Database          workflow_data
    Table             workflow_events
    Timestamp_Key     timestamp

This configuration:

  1. Collects all container logs from /var/log/containers/

  2. Enriches with Kubernetes metadata (namespace, pod, labels)

  3. Filters for Quarkus Flow events using the eventType field

  4. Routes to PostgreSQL

For container mode, the structured events are already JSON in the log stream, so FluentBit can parse them directly without additional processing.

Dual Output Pattern

Send the same logs to multiple destinations:

[OUTPUT]
    Name              pgsql
    Match             flow.events
    # PostgreSQL config...

[OUTPUT]
    Name              es
    Match             flow.events
    Host              elasticsearch.svc
    Port              9200
    Index             workflow-events
    Type              _doc

Production Recommendations

What to Log

Recommended (default):

  • All workflow-level events (workflow.instance.*)

  • Task failures (workflow.task.faulted)

This captures complete workflow state while keeping volume low.

Optional (high-volume):

  • All task events (workflow.task.*) – Only if you need complete task execution history or are debugging specific workflows.

Log Rotation

Configure log rotation to prevent disk fill:

quarkus.log.handler.file."FLOW_EVENTS".rotation.max-file-size=100M
quarkus.log.handler.file."FLOW_EVENTS".rotation.max-backup-index=7
quarkus.log.handler.file."FLOW_EVENTS".rotation.file-suffix=.yyyy-MM-dd
quarkus.log.handler.file."FLOW_EVENTS".rotation.rotate-on-boot=true

Retention Strategy

  • Active workflows: Hot storage (PostgreSQL/Redis)

  • Completed workflows (<30 days): Warm storage (PostgreSQL)

  • Completed workflows (>30 days): Cold storage (S3/object storage)

  • Completed workflows (>1 year): Archive or delete (configurable by compliance needs)

Implement this via your log forwarder’s routing rules or database policies.

Security Considerations

  • Sensitive data: Configure truncation aggressively for workflows handling PII or credentials

  • Log access controls: Restrict access to structured log files and destination databases

  • Encryption: Enable encryption in transit (TLS) and at rest for your log destinations

Performance Impact

Structured logging is designed to be lightweight:

  • CPU overhead: <1% (JSON serialization is fast, truncation is efficient)

  • Memory overhead: Negligible (events are streamed, not buffered)

  • Log volume:

    • Default (workflow + task failures): ~5KB per workflow

    • With task payloads: ~50-500KB per workflow (depends on data size)

The logs-as-transport pattern ensures your application performance isn’t affected by database connectivity issues or backpressure.

Comparison with Custom Listeners

If you’re considering writing a custom listener for audit logging or data export, structured logging may be a simpler alternative:

Approach Pros Cons

Structured Logging

✅ No code required
✅ Standard log infrastructure
✅ Multiple destinations
✅ Zero transport ownership

⚠️ Eventual consistency (log → database delay)
⚠️ Requires log forwarder setup

Custom Listener

✅ Synchronous writes
✅ Custom business logic
✅ Direct database access

❌ You own database connections
❌ More code to maintain
❌ Harder to add new destinations

For most use cases, structured logging is recommended. Reserve custom listeners for scenarios requiring synchronous writes or complex business logic.

See Also