Structured Logging

Quarkus Flow can emit all workflow and task lifecycle events as structured JSON logs to stdout. This enables you to export complete workflow execution data to external databases, analytics platforms, or audit systems without coupling your runtime to specific storage technologies.

Overview

Structured logging follows the logs-as-transport pattern: your workflow runtime emits JSON events to stdout, and a log forwarder (like FluentBit or Vector) routes those events to your chosen destination—PostgreSQL, Elasticsearch, S3, Kafka, or any combination.

This approach provides several advantages:

Zero transport ownership – Your application doesn’t manage database connections, retries, or buffering. The log forwarder handles all transport concerns.
Flexibility – The same log stream can feed multiple destinations simultaneously (e.g., PostgreSQL for queries + S3 for compliance archives).
Clear support boundary – If data isn’t reaching your database, the issue is either "logs not being emitted" (your code) or "logs not being forwarded" (infrastructure). No ambiguity.
Scalability – Log forwarders are designed for high-volume event streaming and can scale independently of your application.

Use Cases

Query APIs – Build GraphQL or REST APIs on top of PostgreSQL to query workflow instances and execution history.
Analytics – Feed workflow events into data warehouses (BigQuery, Snowflake, Redshift) for business intelligence.
Compliance auditing – Maintain long-term audit trails with detailed execution history.
Custom dashboards – Power monitoring UIs with workflow execution data from Elasticsearch.
Event-driven integrations – Stream events to Kafka for downstream processing.

Configuration

Structured logging is disabled by default. To enable it, you need to:

Enable the structured logging feature
Configure a log handler to capture events

Basic Configuration

# 1. Enable structured logging (REQUIRED)
quarkus.flow.structured-logging.enabled=true

# 2. Configure where events should be written (REQUIRED - see Handler Configuration below)
# Example: Write to a file
quarkus.log.handler.file."FLOW_EVENTS".enable=true
quarkus.log.handler.file."FLOW_EVENTS".format=%s%n
quarkus.log.handler.file."FLOW_EVENTS".path=target/quarkus-flow-events.log
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=FLOW_EVENTS
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

# Event filtering (default: all events)
quarkus.flow.structured-logging.events=workflow.*

# Payload inclusion (default: workflow payloads included, task payloads excluded)
quarkus.flow.structured-logging.include-workflow-payloads=true
quarkus.flow.structured-logging.include-task-payloads=false

# Always include full context in error events (default: true)
quarkus.flow.structured-logging.include-error-context=true

# Truncation for large payloads (default: 10KB)
quarkus.flow.structured-logging.payload-max-size=10240
quarkus.flow.structured-logging.truncate-preview-size=1024

# Log level (default: INFO)
quarkus.flow.structured-logging.log-level=INFO

# Timestamp format (default: ISO8601)
quarkus.flow.structured-logging.timestamp-format=iso8601
# For custom format, specify the pattern (java.time.format.DateTimeFormatter)
# quarkus.flow.structured-logging.timestamp-pattern=yyyy-MM-dd'T'HH:mm:ss.SSSXXX

Structured logging requires manual handler configuration. Unlike application logs, structured events use standard Quarkus logging properties (quarkus.log.handler.*) which gives you full control over output destination and format. This is intentional – it allows runtime configuration changes (e.g., switching from file to stdout) without rebuilding your application. See Handler Configuration for complete examples.

Handler Configuration

Quarkus Flow uses standard Quarkus logging configuration to control where structured events are written. You configure handlers using quarkus.log.* properties, which gives you full runtime flexibility – you can change output destinations without rebuilding your application.

All handler configurations shown below are runtime-configurable. This is essential for pre-built container images that need different logging setups per environment (dev/staging/production).

FILE Handler (Traditional Deployments)

Writes events to a dedicated file, separate from application logs:

# Enable structured logging
quarkus.flow.structured-logging.enabled=true

# Configure FILE handler
quarkus.log.handler.file."FLOW_EVENTS".enable=true
quarkus.log.handler.file."FLOW_EVENTS".format=%s%n
quarkus.log.handler.file."FLOW_EVENTS".path=target/quarkus-flow-events.log
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=FLOW_EVENTS
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

Recommended file paths:

Dev/Test mode: target/quarkus-flow-events.log
Production mode: /var/log/quarkus-flow/events.log

When to use FILE handler:

Running on traditional servers or VMs
You want events in a separate file from application logs
Using file-based log forwarders (Filebeat, FluentBit with file input)
File system is writable and persistent

CONSOLE Handler (Container Deployments)

Writes events to stdout for containerized deployments (Kubernetes, Docker, Podman):

# Enable structured logging
quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.timestamp-format=epoch-seconds  # Recommended for container environments

# Configure CONSOLE handler
quarkus.log.handler.console."FLOW_EVENTS".enable=true
quarkus.log.handler.console."FLOW_EVENTS".format=%s%n
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=FLOW_EVENTS
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

Benefits:

No file write permissions needed
Logs automatically captured by container runtime (kubectl logs, docker logs)
Compatible with log aggregation tools (FluentBit, Fluentd, Promtail, Vector)
Works with read-only filesystems
Follows cloud-native logging best practices

When to use CONSOLE handler:

Running in containers (Docker, Kubernetes, Podman, OpenShift)
Using container runtime log collection
Log aggregation tools collect from stdout
File systems are ephemeral or read-only
Following twelve-factor app principles

Example Kubernetes deployment:

With CONSOLE handler, your logs flow like this:

Quarkus Flow writes JSON events to stdout
Container runtime captures stdout → /var/log/containers/*.log
Log forwarder (FluentBit, Fluentd) reads from /var/log/containers/
Events routed to destination (PostgreSQL, Elasticsearch, S3, Kafka)

No special volume mounts or file permissions needed!

Custom Handler Configuration

For advanced use cases, you can configure any Quarkus-supported log handler:

# Enable structured logging
quarkus.flow.structured-logging.enabled=true

# Example: Custom console handler with different name
quarkus.log.handler.console."CUSTOM_FLOW".enable=true
quarkus.log.handler.console."CUSTOM_FLOW".format=%s%n
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=CUSTOM_FLOW
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

When to use custom handlers:

You need full control over logging configuration
Using specialized handlers (syslog, GELF, custom formatters)
Integrating with specialized logging infrastructure
Standard file/console handlers don’t fit your requirements

The logger category for structured events is always io.quarkiverse.flow.structuredlogging.

Runtime Configuration Changes

All handler configurations are runtime-configurable. This is particularly valuable for pre-built container images:

Example: Switch from FILE to CONSOLE at runtime

# Built with FILE handler, deployed with CONSOLE handler
# Just override at runtime via environment variables:

QUARKUS_LOG_HANDLER_FILE_FLOW_EVENTS_ENABLE=false
QUARKUS_LOG_HANDLER_CONSOLE_FLOW_EVENTS_ENABLE=true
QUARKUS_LOG_HANDLER_CONSOLE_FLOW_EVENTS_FORMAT=%s%n
QUARKUS_LOG_CATEGORY__IO_QUARKIVERSE_FLOW_STRUCTUREDLOGGING__HANDLERS=FLOW_EVENTS
QUARKUS_LOG_CATEGORY__IO_QUARKIVERSE_FLOW_STRUCTUREDLOGGING__USE_PARENT_HANDLERS=false

All quarkus.log.* properties support runtime override via environment variables. No application rebuild required!

Timestamp Format Configuration

Different log processors and downstream systems have varying requirements for timestamp formats. Quarkus Flow allows you to configure how timestamps are formatted in structured logging events.

# ISO 8601 format (default) - human-readable, widely compatible
quarkus.flow.structured-logging.timestamp-format=iso8601

# Unix epoch seconds with fractional nanoseconds (e.g., 1776807366.427833)
# Best for: PostgreSQL TIMESTAMP WITH TIME ZONE, InfluxDB
quarkus.flow.structured-logging.timestamp-format=epoch-seconds

# Unix epoch milliseconds as long (e.g., 1776807366428)
# Best for: Elasticsearch date fields, Kafka
quarkus.flow.structured-logging.timestamp-format=epoch-millis

# Unix epoch nanoseconds as long (e.g., 1776807366427832969)
# Best for: High-precision time-series databases
quarkus.flow.structured-logging.timestamp-format=epoch-nanos

# Custom format using java.time.format.DateTimeFormatter pattern
quarkus.flow.structured-logging.timestamp-format=custom
quarkus.flow.structured-logging.timestamp-pattern=yyyy-MM-dd'T'HH:mm:ss.SSSXXX

Format Examples:

Format Example Value

Format	Example Value
`iso8601`	`"2026-04-21T21:36:06.427832969Z"`
`epoch-seconds`	`1776807366.427833`
`epoch-millis`	`1776807366428`
`epoch-nanos`	`1776807366427832969`
`custom`	`"2026-04-21T21:36:06.427-03:00"`

iso8601

"2026-04-21T21:36:06.427832969Z"

epoch-seconds

1776807366.427833

epoch-millis

1776807366428

epoch-nanos

1776807366427832969

custom

"2026-04-21T21:36:06.427-03:00"

The timestamp format applies to all timestamp fields in events: timestamp, startTime, endTime, and lastUpdateTime.

Event Filtering

Control which events are logged using glob patterns:

# All events (default)
quarkus.flow.structured-logging.events=workflow.*

# Only workflow-level events (no task details)
quarkus.flow.structured-logging.events=workflow.instance.*

# Workflow events + task failures (recommended for most use cases)
quarkus.flow.structured-logging.events=workflow.instance.*,workflow.task.faulted

# Specific events only
quarkus.flow.structured-logging.events=\
  workflow.instance.started,\
  workflow.instance.completed,\
  workflow.instance.faulted

Payload Inclusion Strategy

By default, structured logging captures execution graphs (what executed when) but not task payloads (input/output data). This keeps log volume low while providing enough information for execution visualization.

Default behavior:

Workflow events: Include input/output (needed for instance queries)
Task events: Only metadata (taskName, position, status, timing)
Error events: Always include full context (overrides task payload setting)

This produces ~5KB of logs per workflow (compared to 50-500KB if all task payloads were included).

When to enable task payloads:

# Enable full audit trail with all task input/output
quarkus.flow.structured-logging.include-task-payloads=true

Use this for:

Compliance requirements mandating complete execution records
Debugging specific workflows in non-production environments
Workflows with small payloads where volume isn’t a concern

Large Payload Handling

For agentic workflows with large contexts (conversation history, retrieved documents, etc.), payloads exceeding the configured threshold are automatically truncated:

{
  "input": {
    "__truncated__": true,
    "__originalSize__": 157000,
    "__preview__": "First 1KB of data..."
  }
}

This prevents overwhelming log systems while preserving metadata about what was truncated.

Event Schema

All events follow a consistent JSON schema:

{
  "eventType": "workflow.instance.started",
  "timestamp": "2026-04-13T14:30:00.123Z",  // Format depends on configuration
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "workflowNamespace": "default",
  "workflowName": "greetings",
  "workflowVersion": "1.0.0",
  ...event-specific fields...
}

Timestamp fields can be formatted as ISO 8601 strings, Unix epoch values, or custom formats depending on your timestamp format configuration.

Workflow Instance Events

workflow.instance.started – Workflow execution begins
workflow.instance.completed – Workflow finishes successfully
workflow.instance.faulted – Workflow fails with error
workflow.instance.cancelled – Workflow is cancelled
workflow.instance.suspended – Workflow is suspended (waiting)
workflow.instance.resumed – Workflow resumes after suspension
workflow.instance.status.changed – Workflow status changes

Task Events

workflow.task.started – Task execution begins
workflow.task.completed – Task finishes successfully
workflow.task.faulted – Task fails with error
workflow.task.cancelled – Task is cancelled
workflow.task.suspended – Task is suspended
workflow.task.resumed – Task resumes after suspension
workflow.task.retried – Task is retried after failure

Example Events

Workflow Started (ISO 8601 format):

{
  "eventType": "io.serverlessworkflow.workflow.started.v1",
  "timestamp": "2026-04-13T14:30:00.123Z",
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "workflowNamespace": "default",
  "workflowName": "greetings",
  "workflowVersion": "1.0.0",
  "status": "RUNNING",
  "startTime": "2026-04-13T14:30:00.123Z",
  "input": {
    "name": "Alice"
  }
}

Workflow Started (epoch-seconds format):

{
  "eventType": "io.serverlessworkflow.workflow.started.v1",
  "timestamp": 1744642200.123,
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "workflowNamespace": "default",
  "workflowName": "greetings",
  "workflowVersion": "1.0.0",
  "status": "RUNNING",
  "startTime": 1744642200.123,
  "input": {
    "name": "Alice"
  }
}

Workflow Failed:

{
  "eventType": "io.serverlessworkflow.workflow.faulted.v1",
  "timestamp": "2026-04-13T14:30:05.789Z",
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "FAULTED",
  "endTime": "2026-04-13T14:30:05.789Z",
  "error": {
    "message": "Service unavailable",
    "type": "java.net.ConnectException",
    "stackTrace": "..."
  },
  "input": {
    "name": "Alice"
  }
}

Task Started (no payloads):

{
  "eventType": "io.serverlessworkflow.task.started.v1",
  "timestamp": "2026-04-13T14:30:01.000Z",
  "taskExecutionId": "7c9e6679-7425-40de-944b-e07fc1f90ae7",
  "instanceId": "550e8400-e29b-41d4-a716-446655440000",
  "taskName": "callGreetingService",
  "taskPosition": "do/0",
  "status": "RUNNING",
  "startTime": "2026-04-13T14:30:01.000Z"
}

Integration with quarkus-logging-json

When you use quarkus-logging-json in your application, you need to configure a separate handler for structured events to avoid double JSON serialization.

The Problem

When quarkus-logging-json is enabled, it wraps all log messages in a JSON structure:

{
  "timestamp": "2026-04-13T21:00:40.475075-03:00",
  "level": "ERROR",
  "loggerName": "io.quarkiverse.flow.structuredlogging",
  "message": "{\"instanceId\":\"...\",\"eventType\":\"...\"}", // ← JSON string inside JSON
  "threadName": "pool-11-thread-1"
}

Notice how the message field contains a JSON string, not a JSON object. This requires log consumers to:

Parse the outer JSON (from quarkus-logging-json)
Parse the inner JSON string (our workflow event)

This "double serialization" defeats the purpose of structured logging.

The Solution: Dedicated Handler with Raw Format

Configure a dedicated handler for structured events with raw format (%s%n):

FILE Handler (Recommended):

# Enable quarkus-logging-json for application logs
quarkus.log.console.json=true

# Enable Quarkus Flow structured logging
quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.events=workflow.*

# Configure dedicated FILE handler with raw format
quarkus.log.handler.file."FLOW_EVENTS".enable=true
quarkus.log.handler.file."FLOW_EVENTS".format=%s%n  # Raw JSON, no quarkus-logging-json wrapper
quarkus.log.handler.file."FLOW_EVENTS".path=target/quarkus-flow-events.log
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=FLOW_EVENTS
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

CONSOLE Handler (Containerized deployments):

If you prefer stdout in containers:

# Enable quarkus-logging-json for application logs
quarkus.log.console.json=true

# Enable Quarkus Flow structured logging
quarkus.flow.structured-logging.enabled=true
quarkus.flow.structured-logging.events=workflow.*

# Configure dedicated CONSOLE handler with raw format
quarkus.log.handler.console."FLOW_EVENTS".enable=true
quarkus.log.handler.console."FLOW_EVENTS".format=%s%n  # Raw JSON, no quarkus-logging-json wrapper
quarkus.log.category."io.quarkiverse.flow.structuredlogging".handlers=FLOW_EVENTS
quarkus.log.category."io.quarkiverse.flow.structuredlogging".use-parent-handlers=false

With this configuration:

Console logs: Application logs in JSON format (from quarkus-logging-json)
Event output: Pure workflow event JSON (one event per line, no double-wrapping)

Why Separate Handlers?

This follows the logs-as-transport pattern correctly:

Application logs: Diagnostic information for debugging (stdout/stderr, formatted by quarkus-logging-json)
Event streams: Structured data for analytics/auditing (dedicated handler with raw JSON format)

Event streams and diagnostic logs serve different purposes and should be treated separately. Log forwarders can then: - Parse application logs with the quarkus-logging-json schema - Parse workflow events as pure JSON (no wrapper) - Route each to different destinations (e.g., Elasticsearch for logs, PostgreSQL for events)

The format=%s%n setting ensures events are written as raw JSON strings without any additional formatting from quarkus-logging-json.

Log Forwarder Integration

FluentBit Example

FluentBit is a lightweight, high-performance log forwarder. Configuration depends on your handler type:

FILE Handler Configuration

When using a FILE handler, FluentBit reads from the configured log file:

[INPUT]
    Name              tail
    Path              /var/log/quarkus-flow/events.log  # Default production path
    Parser            json
    Tag               flow.events

[FILTER]
    Name              modify
    Match             flow.events
    Add               kubernetes.namespace ${K8S_NAMESPACE}
    Add               kubernetes.pod ${K8S_POD_NAME}

[OUTPUT]
    Name              pgsql
    Match             flow.events
    Host              postgres.database.svc
    Port              5432
    User              flowuser
    Password          ${DB_PASSWORD}
    Database          workflow_data
    Table             workflow_events
    Timestamp_Key     timestamp

CONSOLE Handler Configuration (Recommended for Kubernetes/Docker)

When using a CONSOLE handler, FluentBit collects logs from stdout via the container runtime:

[INPUT]
    Name              tail
    Path              /var/log/containers/*_${NAMESPACE}_${POD_NAME}-*.log
    Parser            docker  # Or cri/containerd depending on runtime
    Tag               kube.*

[FILTER]
    Name              kubernetes
    Match             kube.*
    Kube_URL          https://kubernetes.default.svc:443
    Merge_Log         On
    Keep_Log          Off

[FILTER]
    Name              grep
    Match             kube.*
    Regex             log {"eventType":"io.serverlessworkflow.*

[OUTPUT]
    Name              pgsql
    Match             kube.*
    Host              postgres.database.svc
    Port              5432
    User              flowuser
    Password          ${DB_PASSWORD}
    Database          workflow_data
    Table             workflow_events
    Timestamp_Key     timestamp

This configuration:

Collects all container logs from /var/log/containers/
Enriches with Kubernetes metadata (namespace, pod, labels)
Filters for Quarkus Flow events using the eventType field
Routes to PostgreSQL

For CONSOLE handler, the structured events are already JSON in the log stream, so FluentBit can parse them directly without additional processing.

Dual Output Pattern

Send the same logs to multiple destinations:

[OUTPUT]
    Name              pgsql
    Match             flow.events
    # PostgreSQL config...

[OUTPUT]
    Name              es
    Match             flow.events
    Host              elasticsearch.svc
    Port              9200
    Index             workflow-events
    Type              _doc

Production Recommendations

What to Log

Recommended (default):

All workflow-level events (workflow.instance.*)
Task failures (workflow.task.faulted)

This captures complete workflow state while keeping volume low.

Optional (high-volume):

All task events (workflow.task.*) – Only if you need complete task execution history or are debugging specific workflows.

Log Rotation

Configure log rotation to prevent disk fill:

quarkus.log.handler.file."FLOW_EVENTS".rotation.max-file-size=100M
quarkus.log.handler.file."FLOW_EVENTS".rotation.max-backup-index=7
quarkus.log.handler.file."FLOW_EVENTS".rotation.file-suffix=.yyyy-MM-dd
quarkus.log.handler.file."FLOW_EVENTS".rotation.rotate-on-boot=true

Retention Strategy

Active workflows: Hot storage (PostgreSQL/Redis)
Completed workflows (<30 days): Warm storage (PostgreSQL)
Completed workflows (>30 days): Cold storage (S3/object storage)
Completed workflows (>1 year): Archive or delete (configurable by compliance needs)

Implement this via your log forwarder’s routing rules or database policies.

Security Considerations

Sensitive data: Configure truncation aggressively for workflows handling PII or credentials
Log access controls: Restrict access to structured log files and destination databases
Encryption: Enable encryption in transit (TLS) and at rest for your log destinations

Performance Impact

Structured logging is designed to be lightweight:

CPU overhead: <1% (JSON serialization is fast, truncation is efficient)
Memory overhead: Negligible (events are streamed, not buffered)
Log volume:
- Default (workflow + task failures): ~5KB per workflow
- With task payloads: ~50-500KB per workflow (depends on data size)

The logs-as-transport pattern ensures your application performance isn’t affected by database connectivity issues or backpressure.

Comparison with Custom Listeners

If you’re considering writing a custom listener for audit logging or data export, structured logging may be a simpler alternative:

Approach	Pros	Cons
Structured Logging	✅ No code required ✅ Standard log infrastructure ✅ Multiple destinations ✅ Zero transport ownership	⚠️ Eventual consistency (log → database delay) ⚠️ Requires log forwarder setup
Custom Listener	✅ Synchronous writes ✅ Custom business logic ✅ Direct database access	❌ You own database connections ❌ More code to maintain ❌ Harder to add new destinations

Approach

Pros

Cons

Structured Logging

✅ No code required
✅ Standard log infrastructure
✅ Multiple destinations
✅ Zero transport ownership

⚠️ Eventual consistency (log → database delay)
⚠️ Requires log forwarder setup

Custom Listener

✅ Synchronous writes
✅ Custom business logic
✅ Direct database access

❌ You own database connections
❌ More code to maintain
❌ Harder to add new destinations

For most use cases, structured logging is recommended. Reserve custom listeners for scenarios requiring synchronous writes or complex business logic.

Structured Logging

Overview

Use Cases

Configuration

Basic Configuration

Handler Configuration

FILE Handler (Traditional Deployments)

CONSOLE Handler (Container Deployments)

Custom Handler Configuration

Runtime Configuration Changes

Timestamp Format Configuration

Event Filtering

Payload Inclusion Strategy

Large Payload Handling

Event Schema

Workflow Instance Events

Task Events

Example Events

Integration with quarkus-logging-json

The Problem

The Solution: Dedicated Handler with Raw Format

Why Separate Handlers?

Log Forwarder Integration

FluentBit Example

FILE Handler Configuration

CONSOLE Handler Configuration (Recommended for Kubernetes/Docker)

Dual Output Pattern

Production Recommendations

What to Log

Log Rotation

Retention Strategy

Security Considerations

Performance Impact

Comparison with Custom Listeners

See Also