Google Cloud Services - Logging

This extension allows you to integrate your logs directly into Google Cloud Logging, including the ability to set resource labels and log record labels, include trace ID and to use structured logging.

You can integrate your logs into Cloud Logging via a standard logger of the com.google.cloud.logging.Logging object that you can inject in your code.

Bootstrapping the project

First, we need a new project. Create a new project with the following command (replace the version placeholder with the correct one):

mvn io.quarkus:quarkus-maven-plugin:<quarkusVersion>:create \
    -DprojectGroupId=org.acme \
    -DprojectArtifactId=firestore-quickstart \
    -Dextensions="resteasy-reactive-jackson,quarkus-google-cloud-logging"
cd firestore-quickstart

This command generates a Maven project, importing the Google Cloud Logging extension.

If you already have your Quarkus project configured, you can add the quarkus-google-cloud-logging extension to your project by running the following command in your project base directory:

./mvnw quarkus:add-extension -Dextensions="quarkus-google-cloud-logging"

This will add the following to your pom.xml:

<dependency>
    <groupId>io.quarkiverse.googlecloudservices</groupId>
    <artifactId>quarkus-google-cloud-logging</artifactId>
</dependency>

How-to: Log

When configured, all that is needed is to log data as you wish. You can use JUL, jboss-logging or Slf4j, or even Google Cloud Logging directly. You can also inject the logger to use, for example:

package mypackage;

import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import org.jboss.logging.Logger;

@Path("/logging")
public class LoggingResource {

    @Inject
    Logger logger;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String tryLog() {
        logger.info("Greeting the World");
        return "Hello World!";
    }
}

Configuration

Basic

Currently, the logging system uses a single Google Cloud Operations log and this is a mandatory configuration parameter.

quarkus.google.cloud.logging.default-log
quarkus.google.cloud.project-id

If you want to disable the logging to Google Cloud you can do so:

quarkus.google.cloud.logging.enabled=false

This is useful for testing not to write to Google Operations during development:

%test.quarkus.google.cloud.logging.enabled=false

Formatting

This logging system can either use a standard text based format, Elastic Common Schema, or a completely custom schema.

quarkus.google.cloud.logging.format=[TEXT|JSON]

Structured Logging

By setting the logging format to JSON this library will use the ECS format. As local record parameters and MDC values are not official part of the ECS specification, these default to parameters and mdc respectively, but this can be configured (see below).

quarkus.google.cloud.logging.structured.stack-trace.included=[true|false] - Should stack traces be included? Default to true
quarkus.google.cloud.logging.structured.parameters.included=[true|false] - Should log record parameters be included? Defaults to true
quarkus.google.cloud.logging.structured.parameters.field-name - This is the field name to use in ESC for parameters, defaults to parameters
quarkus.google.cloud.logging.structured.mdc.included=[true|false] - Should the MDC values be included? Defaults to true
quarkus.google.cloud.logging.structured.mdc.field-name - This is the field name to use in ESC for MDC values, defaults to mdc

Resource type and labels

You can configure the resource type and the resource labels. This is used by Google Cloud Operations as dimensions when analysing logs. Please refer to the [Google Cloud documentation](https://cloud.google.com/logging/docs/api/v2/resource-list#resource-types) for the allowable resource types and their associated labels.

quarkus.google.cloud.logging.resource.type - The type string, defaults to generic_node
quarkus.google.cloud.logging.resource.label.<string> - This is a string map with key/value pairs

Log record labels

You can configure default labels for each log record:

quarkus.google.cloud.logging.default-label.<string> - This is a string map with key/value pairs

Tracing

In order to include trace and span ID in the log, you must bind a TraceInfoExtractor to the CDI context, e.g.:

package mypackage;

import javax.enterprise.context.ApplicationScoped;

import org.jboss.logmanager.ExtLogRecord;

import io.quarkiverse.googlecloudservices.logging.runtime.TraceInfo;
import io.quarkiverse.googlecloudservices.logging.runtime.TraceInfoExtractor;

@ApplicationScoped
public class TestTraceInfoExtractor implements TraceInfoExtractor {

    @Override
    public TraceInfo extract(ExtLogRecord record) {
        return new TraceInfo(/* something, something */);
    }
}

This will include the trace and span ID in the ESC log if they are not null. By default, this will also be set on the Google Cloud Operations log entry as a link to Google Cloud Operations tracing, but this is configurable:

quarkus.google.cloud.logging.gcp-tracing.enabled=[true|false]

Custom Labels

In order to include additional labels in the log, you must bind a LogRecordLabelExtractor to the CDI context, e.g.:

package mypackage;

import java.util.Map;

import javax.enterprise.context.ApplicationScoped;

import org.jboss.logmanager.ExtLogRecord;

import io.quarkiverse.googlecloudservices.logging.runtime.LogRecordLabelExtractor;

@ApplicationScoped
public class LogLabelExtractor implements LogRecordLabelExtractor {

    @Override
    public Map<String, String> getCustomLabels(ExtLogRecord extLogRecord) {
        return Map.of(/* some label, some value */);
    }
}

This will include additional labels in the log.

Injecting GCP Logging

You can inject a com.google.cloud.logging.Logging instance directly. If you do so, the configuration for the project to use, still apply.

import java.time.Instant;
import java.util.List;

import com.google.cloud.logging.Logging;
import com.google.cloud.logging.LogEntry;
import com.google.cloud.logging.Payload;
import com.google.cloud.logging.Severity;

@Inject
Logging gcpLogging;


public void log(String s) {
    gcpLogging.write(List.of(LogEntry.newBuilder(Payload.StringPayload.of(s))
        .setSeverity(Severity.DEBUG)
        .setTimestamp(Instant.now())
        .build()));
}

Configuration Reference

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property	Type	Default
`quarkus.google.cloud.logging.default-log` Which Google Operations log should be used by default. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_DEFAULT_LOG`	string	required
`quarkus.google.cloud.logging.enabled` Enable or disable the Google Cloud logging. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_ENABLED`	boolean	`true`
`quarkus.google.cloud.logging.format` Configure base formatting to be either plain text or structured json. Allowed values: TEXT\|JSON Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_FORMAT`	`text`, `json`	`text`
`quarkus.google.cloud.logging.synchronicity` Configure GCP logging synchronicity. Allowed values: SYNC\|ASYNC Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_SYNCHRONICITY`	`sync`, `async`
`quarkus.google.cloud.logging.flush-level` Configure auto flush level. Allowed values: DEBUG\|INFO\|WARN\|ERROR\|FATAL Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_FLUSH_LEVEL`	`debug`, `info`, `warn`, `error`, `fatal`
`quarkus.google.cloud.logging.default-label."default-label"` Configure default labels. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_DEFAULT_LABEL__DEFAULT_LABEL_`	Map<String,String>
`quarkus.google.cloud.logging.resource.type` The resource type of the log. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_RESOURCE_TYPE`	string	`global`
`quarkus.google.cloud.logging.resource.label."label"` Resource labels. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_RESOURCE_LABEL__LABEL_`	Map<String,String>
`quarkus.google.cloud.logging.gcp-tracing.enabled` Use this setting to determine if extracted trace ID’s should also be forwarded to GCP for linking with GCP Operations Tracing. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_GCP_TRACING_ENABLED`	boolean	`true`
`quarkus.google.cloud.logging.gcp-tracing.project-id` If the GCP Operations Tracing is in another project, configure it here. By default the logging project will be used. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_GCP_TRACING_PROJECT_ID`	string
`quarkus.google.cloud.logging.structured.stack-trace.included` Include stack traces when exceptions are thrown. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_STACK_TRACE_INCLUDED`	boolean	`true`
`quarkus.google.cloud.logging.structured.mdc.included` Include MDC values in the log. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_MDC_INCLUDED`	boolean	`true`
`quarkus.google.cloud.logging.structured.mdc.field-name` Field name for MDC values, defaults to 'mdc'. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_MDC_FIELD_NAME`	string	`mdc`
`quarkus.google.cloud.logging.structured.parameters.included` Include parameter values in the log. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_PARAMETERS_INCLUDED`	boolean	`true`
`quarkus.google.cloud.logging.structured.parameters.field-name` Field name for parameter values, defaults to 'parameters'. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_PARAMETERS_FIELD_NAME`	string	`parameters`
`quarkus.google.cloud.logging.log-target` Configures if logs should be written to stdout or stderr instead of using Google Cloud Operations API. Useful if app is deployed to managed Google Cloud Platform environment with installed logger agent. Possible values: STDOUT, STDERR and CLOUD_LOGGING. Environment variable: `QUARKUS_GOOGLE_CLOUD_LOGGING_LOG_TARGET`	`cloud-logging`, `stdout`, `stderr`	`cloud-logging`

Configuration property

Type

Default

quarkus.google.cloud.logging.default-log

Which Google Operations log should be used by default.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_DEFAULT_LOG

string

required

quarkus.google.cloud.logging.enabled

Enable or disable the Google Cloud logging.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_ENABLED

boolean

true

quarkus.google.cloud.logging.format

Configure base formatting to be either plain text or structured json. Allowed values: TEXT|JSON

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_FORMAT

text, json

text

quarkus.google.cloud.logging.synchronicity

Configure GCP logging synchronicity. Allowed values: SYNC|ASYNC

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_SYNCHRONICITY

sync, async

quarkus.google.cloud.logging.flush-level

Configure auto flush level. Allowed values: DEBUG|INFO|WARN|ERROR|FATAL

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_FLUSH_LEVEL

debug, info, warn, error, fatal

quarkus.google.cloud.logging.default-label."default-label"

Configure default labels.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_DEFAULT_LABEL__DEFAULT_LABEL_

Map<String,String>

quarkus.google.cloud.logging.resource.type

The resource type of the log.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_RESOURCE_TYPE

string

global

quarkus.google.cloud.logging.resource.label."label"

Resource labels.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_RESOURCE_LABEL__LABEL_

Map<String,String>

quarkus.google.cloud.logging.gcp-tracing.enabled

Use this setting to determine if extracted trace ID’s should also be forwarded to GCP for linking with GCP Operations Tracing.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_GCP_TRACING_ENABLED

boolean

true

quarkus.google.cloud.logging.gcp-tracing.project-id

If the GCP Operations Tracing is in another project, configure it here. By default the logging project will be used.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_GCP_TRACING_PROJECT_ID

string

quarkus.google.cloud.logging.structured.stack-trace.included

Include stack traces when exceptions are thrown.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_STACK_TRACE_INCLUDED

boolean

true

quarkus.google.cloud.logging.structured.mdc.included

Include MDC values in the log.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_MDC_INCLUDED

boolean

true

quarkus.google.cloud.logging.structured.mdc.field-name

Field name for MDC values, defaults to 'mdc'.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_MDC_FIELD_NAME

string

mdc

quarkus.google.cloud.logging.structured.parameters.included

Include parameter values in the log.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_PARAMETERS_INCLUDED

boolean

true

quarkus.google.cloud.logging.structured.parameters.field-name

Field name for parameter values, defaults to 'parameters'.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_STRUCTURED_PARAMETERS_FIELD_NAME

string

parameters

quarkus.google.cloud.logging.log-target

Configures if logs should be written to stdout or stderr instead of using Google Cloud Operations API. Useful if app is deployed to managed Google Cloud Platform environment with installed logger agent. Possible values: STDOUT, STDERR and CLOUD_LOGGING.

Environment variable: QUARKUS_GOOGLE_CLOUD_LOGGING_LOG_TARGET

cloud-logging, stdout, stderr

cloud-logging