Quarkiverse Java Operator SDK

This extension integrates the Java Operator SDK project (JOSDK) with Quarkus, making it even easier to use both.

For an introduction to this project, please read the blog series that we wrote on how to write operators in Java with Quarkus. Please also refer to the JOSDK documentation for more details.

Features

  • Automatically generates a main class, so that the only thing that’s required is to write Reconciler implementation(s)

  • Automatically makes a Kubernetes/OpenShift client available for CDI injection

  • Automatically sets up an Operator instance, also available for CDI injection

  • Automatically processes the reconcilers' configuration at build time, exposing all the available configuration of JOSDK via application properties

  • Automatically registers reconcilers with the Operator and start them

  • Automatically generates CRDs for all CustomResource implementations used by reconcilers

  • Automatically generates Kubernetes descriptors

  • Automatically generates the bundle manifests for all reconcilers (using the quarkus-operator-sdk-bundle-generator extension)

  • Integrates with the Dev mode:

    • Watches your code for changes and reload automatically your operator if needed without having to hit an endpoint

    • Only re-generates the CRDs if a change impacting its generation is detected

    • Only re-processes a reconciler’s configuration if needed

    • Automatically apply the CRD to the cluster when it has changed

  • Supports micrometer registry extensions (adding a Quarkus-supported micrometer registry extension will automatically inject said registry into the operator)

  • Automatically adds a SmallRye health check

  • Sets up reflection for native binary generation

  • [Deprecated] Customize the JSON serialization that the Fabric8 client relies on by providing an ObjectMapperCustomizer implementation, qualified with the @KubernetesClientSerializationCustomizer annotation

    • The Quarkus kubernetes client extension now provides an official mechanism to do so by implementing the io.quarkus.kubernetes.client.KubernetesClientObjectMapperCustomizer interface instead so this mechanism should be used moving forward.

Installation

If you want to use this extension, you need to add the quarkus-operator-sdk extension first.

You need to add minimally, the following to your pom.xml file:

<dependency>
    <groupId>io.quarkiverse.operatorsdk</groupId>
    <artifactId>quarkus-operator-sdk</artifactId>
    <version>6.5.0.Beta2</version>
</dependency>

However, it might be more convenient to use the quarkus-operator-sdk-bom dependency to ensure that all dependency versions are properly aligned:

<dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>io.quarkiverse.operatorsdk</groupId>
        <artifactId>quarkus-operator-sdk-bom</artifactId>
        <version>6.5.0.Beta2</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>

      <!-- other dependencies as needed by your project -->

    </dependencies>
  </dependencyManagement>

If you do use the BOM, please do make sure to use the same Quarkus version as the one defined in the BOM when configuring the Quarkus plugin as the Quarkus Dev Mode will not work properly otherwise, failing with an error:

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalStateException: Hot deployment of the application is not supported when updating the Quarkus version. The application needs to be stopped and dev mode started up again
        at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:138)
        at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:62)

If you want to use the Bundle generator, you will first need to use Quarkus 2.3.0.Final or above and add the quarkus-operator-sdk-bundle-generator extension first:

<dependency>
    <groupId>io.quarkiverse.operatorsdk</groupId>
    <artifactId>quarkus-operator-sdk-bundle-generator</artifactId>
    <version>6.5.0.Beta2</version>
</dependency>

Deployment

This section explains how to deploy your operator using the Operator Lifecycle Manager (OLM) by following the next steps:

Requirements

Make sure you have installed the opm command tool and are connected to a Kubernetes cluster on which OLM is installed.

Generate the Operator image and bundle manifests

Quarkus provides several extensions to build the container image. For example, the Joke sample uses the Quarkus Jib container image extension to build the image. So, you first need to configure one of these extensions as you prefer. Then, you need to add the quarkus-operator-sdk-bundle-generator extension:

<dependency>
    <groupId>io.quarkiverse.operatorsdk</groupId>
    <artifactId>quarkus-operator-sdk-bundle-generator</artifactId>
    <version>6.5.0.Beta2</version>
</dependency>

This extension generates the Operator bundle manifests in the target/bundle directory.

Finally, to generate the operator image and the bundle manifests at once, you simply need to run the next Maven command:

mvn clean package -Dquarkus.container-image.build=true \
    -Dquarkus.container-image.push=true \
    -Dquarkus.container-image.registry=<your container registry. Example: quay.io> \
    -Dquarkus.container-image.group=<your container registry namespace> \
    -Dquarkus.kubernetes.namespace=<the kubernetes namespace where you will deploy the operator> \
    -Dquarkus.operator-sdk.bundle.package-name=<the name of the package that bundle image belongs to> \
    -Dquarkus.operator-sdk.bundle.channels=<the list of channels that bundle image belongs to>

For example, if we want to name the package my-operator and use the alpha channels, we would need to append the properties -Dquarkus.operator-sdk.bundle.package-name=my-operator -Dquarkus.operator-sdk.bundle.channels=alpha.

Find more information about channels and packages here.

If you’re using an insecure container registry, you’ll also need to append the next property to the Maven command -Dquarkus.container-image.insecure=true.

Build the Operator Bundle image

An Operator Bundle is a container image that stores Kubernetes manifests and metadata associated with an operator. You can find more information about this here. In the previous step, we generated the bundle manifests at target/bundle which includes a ready-to-use target/bundle/bundle.Dockerfile Dockerfile that you will use to build and push the final Operator Bundle image to your container registry:

MY_BUNDLE_IMAGE=<your container registry>/<your container registry namespace>/<bundle image name>:<tag>
docker build -t $MY_BUNDLE_IMAGE -f target/bundle/bundle.Dockerfile target/bundle
docker push $MY_BUNDLE_IMAGE

For example, if we want to name our bundle image as my-manifest-bundle, our container registry is quay.io, our Quay user is myuser and the tag we’re releasing is 1.0, the final MY_BUNDLE_IMAGE property would be quay.io/myuser/my-manifest-bundle:1.0.

Make your operator available within a Catalog

OLM uses catalogs to discover and install Operators and their dependencies. So, a catalog is similar to a repository of operators and their associated versions that can be installed on a cluster. Moreover, the catalog is also a container image that contains a collection of bundles and channels. Therefore, we’d need to create a new catalog (or update an existing one if you’re already have one), build/push the catalog image and then install it on our cluster.

So far, we have already built the Operator bundle image at $MY_BUNDLE_IMAGE (see above) and next, we need to add this Operator bundle image into our catalog. For doing this, we’ll use the olm tool as follows:

CATALOG_IMAGE=<catalog container registry>/<catalog container registry namespace>/<catalog name>:<tag>
opm index add \
    --bundles $MY_BUNDLE_IMAGE \
    --tag $CATALOG_IMAGE \
    --build-tool docker
docker push $CATALOG_IMAGE

For example, if our catalog name is my-catalog, our container registry for the catalog is quay.io, our Quay user is myuser and the container tag we’re releasing is 59.0, the final CATALOG_IMAGE property would be quay.io/myuser/my-catalog:59.0.

If you’re using an insecure registry, you’d need to append the argument --skip-tls to the opm index command.

Once we have our catalog image built and pushed at $CATALOG_IMAGE, we need to install it in the same namespace where OLM is running (by default, OLM is running in the operators namespace, we will use the OLM_NAMESPACE property to represent this namespace) on our cluster using the CatalogSource resource by doing the next command:

cat <<EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: my-catalog-source
  namespace: $OLM_NAMESPACE
spec:
  sourceType: grpc
  image: $CATALOG_IMAGE
EOF

Once the catalog is installed, you should see the catalog pod up and running:

kubectl get pods -n $OLM_NAMESPACE --selector=olm.catalogSource=my-catalog-source

Install your operator via OLM

OLM deploys operators via subscriptions. Creating a Subscription will trigger the operator deployment. You can simply create the Subscription resource that contains the operator name and channel to install by running the following command:

cat <<EOF | kubectl create -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: my-subscription
  namespace: <Kubernetes namespace where your operator will be installed>
spec:
  channel: alpha
  name: my-operator-name
  source: my-catalog-source
  sourceNamespace: $OLM_NAMESPACE
EOF

We’ll install the operator in the target namespace defined in the metadata object. The sourceNamespace value is the Kubernetes namespace where the catalog was installed on.

Once the subscription is created, you should see your operator pod up and running:

kubectl get csv -n $OLM_NAMESPACE my-operator-name

Extension Configuration Reference

Remove this section if you don’t have Quarkus configuration properties in your extension.

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

Whether the operator should check that the CRD is properly deployed and that the associated CustomResource implementation matches its information before registering the associated controller.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_VALIDATE

boolean

true

Whether the extension should automatically generate the CRD based on CustomResource implementations.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_GENERATE

boolean

Whether the extension should automatically apply updated CRDs when they change. When running on DEV mode, the CRD changes will always be applied automatically.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_APPLY

boolean

Comma-separated list of which CRD versions should be generated.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_VERSIONS

list of string

v1

The directory where the CRDs will be generated, defaults to the kubernetes directory of the project’s output directory.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_OUTPUT_DIRECTORY

string

Whether the extension should generate all CRDs even if some are not tied to a Reconciler.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_GENERATE_ALL

boolean

false

Whether the CRDs should be generated in parallel. Please note that this feature is experimental and it may lead to unexpected results.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_GENERATE_IN_PARALLEL

boolean

false

A comma-separated list of fully-qualified class names implementing custom resources to exclude from the CRD generation process.

Environment variable: QUARKUS_OPERATOR_SDK_CRD_EXCLUDE_RESOURCES

list of string

Whether controllers should only process events if the associated resource generation has increased since last reconciliation, otherwise will process all events. Sets the default value for all controllers.

Environment variable: QUARKUS_OPERATOR_SDK_GENERATION_AWARE

boolean

true

Whether Role-Based Access Control (RBAC) resources generated by the Kubernetes extension should be augmented by this extension.

Environment variable: QUARKUS_OPERATOR_SDK_DISABLE_RBAC_GENERATION

boolean

false

Whether the operator should be automatically started or not. Mostly useful for testing scenarios.

Environment variable: QUARKUS_OPERATOR_SDK_START_OPERATOR

boolean

Whether the injected Kubernetes client should be stopped when the operator is stopped.

Environment variable: QUARKUS_OPERATOR_SDK_CLOSE_CLIENT_ON_STOP

boolean

true

Whether the operator should stop if an informer error (such as one caused by missing / improper RBACs) occurs during startup.

Environment variable: QUARKUS_OPERATOR_SDK_STOP_ON_INFORMER_ERROR_DURING_STARTUP

boolean

true

Whether to fail or emit a debug-level (warning-level when misalignment is at the minor or above version level) log when the extension detects that there are misaligned versions.

The following versions are checked for alignment:

  • declared Quarkus version used to build the extension vs. actually used Quarkus version at runtime

  • Fabric8 client version used by JOSDK vs. actually used Fabric8 client version

  • Fabric8 client version used by Quarkus vs. actually used Fabric8 client version

Environment variable: QUARKUS_OPERATOR_SDK_FAIL_ON_VERSION_CHECK

boolean

false

The list of profile names for which leader election should be activated. This is mostly useful for testing scenarios where leader election behavior might lead to issues.

Environment variable: QUARKUS_OPERATOR_SDK_ACTIVATE_LEADER_ELECTION_FOR_PROFILES

list of string

prod

The optional Server-Side Apply (SSA) related configuration.

Environment variable: QUARKUS_OPERATOR_SDK_ENABLE_SSA

boolean

true

An optional list of comma-separated watched namespace names that will be used to generate manifests at build time if controllers do NOT specify a value individually. See BuildTimeControllerConfiguration#generateWithWatchedNamespaces for more information.

Environment variable: QUARKUS_OPERATOR_SDK_GENERATE_WITH_WATCHED_NAMESPACES

list of string

Can be used to disable helm chart generation.

Environment variable: QUARKUS_OPERATOR_SDK_HELM_ENABLED

boolean

false

The max number of concurrent dispatches of reconciliation requests to controllers.

Environment variable: QUARKUS_OPERATOR_SDK_CONCURRENT_RECONCILIATION_THREADS

int

Amount of seconds the SDK waits for reconciliation threads to terminate before shutting down.

Environment variable: QUARKUS_OPERATOR_SDK_TERMINATION_TIMEOUT_SECONDS

int

An optional list of comma-separated namespace names all controllers will watch if they do not specify their own list. If a controller specifies its own list either via the io.javaoperatorsdk.operator.api.reconciler.ControllerConfiguration annotation or via the associated application.properties property, that value will be used instead of the operator-level default value that this configuration option provides.

If this property is left empty then controllers will watch all namespaces by default (which is equivalent to setting this property to Constants#WATCH_ALL_NAMESPACES, assuming they do not provide their own list of namespaces to watch. . The value can be set to Constants#WATCH_CURRENT_NAMESPACE to make all controllers watch the current namespace as specified by the kube config file the operator uses.

Environment variable: QUARKUS_OPERATOR_SDK_NAMESPACES

list of string

QOSDK_USE_BUILDTIME_NAMESPACES

The max number of concurrent workflow processing requests.

Environment variable: QUARKUS_OPERATOR_SDK_CONCURRENT_WORKFLOW_THREADS

int

How long the operator will wait for informers to finish synchronizing their caches on startup before timing out.

Environment variable: QUARKUS_OPERATOR_SDK_CACHE_SYNC_TIMEOUT

Duration

2M

Whether the controller should only process events if the associated resource generation has increased since last reconciliation, otherwise will process all events.

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__GENERATION_AWARE

boolean

An optional list of comma-separated watched namespace names that will be used to generate manifests at build time.

Note that this is provided as a means to quickly deploy a specific controller to test it by applying the generated manifests to the target cluster. If empty, no manifests will be generated. The namespace in which the controller will be deployed will be the currently configured namespace as specified by your .kube/config file, unless you specify the target deployment namespace using the quarkus.kubernetes.namespace property.

As this functionality cannot handle namespaces that are not know until runtime (because the generation happens during build time), we recommend that you use a different mechanism such as OLM or Helm charts to deploy your operator in production.

This replaces the previous namespaces property which was confusing and against Quarkus best practices since it existed both at build time and runtime. That property wasn’t also adequately capturing the fact that namespaces that wouldn’t be known until runtime would render whatever got generated at build time invalid as far as generated manifests were concerned.

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__GENERATE_WITH_WATCHED_NAMESPACES

list of string

Indicates whether the primary resource for the associated controller is unowned, meaning that another controller is the principal controller handling resources of this type. By default, controllers are assumed to own their primary resource but there are cases where this might not be the case, for example, when extra processing of a given resource type is required even though another controller already handles reconciliations of resources of that type. Set this property to true if you want to indicate that the controller doesn’t own its primary resource

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__UNOWNED_PRIMARY

boolean

false

An optional list of comma-separated namespace names the controller should watch. If this property is left empty then the controller will watch all namespaces. The value can be set to "JOSDK_WATCH_CURRENT" to watch the current (default) namespace from kube config. Constant(s) can be found in at `io.javaoperatorsdk.operator.api.reconciler.Constants`".

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__NAMESPACES

list of string

The optional name of the finalizer for the controller. If none is provided, one will be automatically generated.

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__FINALIZER

string

How many times an operation should be retried before giving up

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__RETRY_MAX_ATTEMPTS

int

The initial interval that the controller waits for before attempting the first retry

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__RETRY_INTERVAL_INITIAL

long

2000

The value by which the initial interval is multiplied by for each retry

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__RETRY_INTERVAL_MULTIPLIER

double

1.5

The maximum interval that the controller will wait for before attempting a retry, regardless of all other configuration

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__RETRY_INTERVAL_MAX

long

An optional list of comma-separated label selectors that Custom Resources must match to trigger the controller. See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ for more details on selectors.

Environment variable: QUARKUS_OPERATOR_SDK_CONTROLLERS__CONTROLLERS__SELECTOR

string

About the Duration format

To write duration values, use the standard java.time.Duration format. See the Duration#parse() javadoc for more information.

You can also use a simplified format, starting with a number:

  • If the value is only a number, it represents time in seconds.

  • If the value is a number followed by ms, it represents time in milliseconds.

In other cases, the simplified format is translated to the java.time.Duration format for parsing:

  • If the value is a number followed by h, m, or s, it is prefixed with PT.

  • If the value is a number followed by d, it is prefixed with P.