Crossplane v2.2 — More Capable, More Reliable, More Observable

Adam Wolfe Gordon -


We are excited to announce that Crossplane v2.2.0 has been released and is now available for installation into your control planes. This release is a regular quarterly release focused on maturing key areas of functionality across the project, as Crossplane continues to become more capable, more reliable, and more performant for your production workloads. In this post, we'll dive into the highlights.

Debugging with the Pipeline Inspector (Alpha)

Composition functions give platform builders the power to express arbitrarily complex resource generation logic, but debugging that logic inside a running control plane has historically been difficult. You can write tests and use crossplane render locally, but viewing the requests are going into each function step and the responses coming out required updating the pipeline or the functions themselves.

Crossplane v2.2 introduces a new alpha feature to address this: the pipeline inspector. When enabled, the inspector intercepts every function request and response in a composition pipeline and forwards them over gRPC to a user-configured socket path. This opens the door to a range of debugging and observability use cases — from a local observer that dumps requests to stdout during development, to a production-grade system that stores pipeline traces for auditing or troubleshooting.

The pipeline inspector is disabled by default. To enable it, pass the --enable-pipeline-inspector flag to Crossplane. The default socket address is /var/run/pipeline-inspector/socket, which can be overridden by passing the --pipeline-inspector-socket flag. The full Helm values required to enable the inspector and run a basic pipeline logging sidecar are:

# Enable the pipeline inspector feature flag
args:
  - --enable-pipeline-inspector
  - --pipeline-inspector-socket=/var/run/pipeline-inspector/socket

# Inject the pipeline inspector sidecar
sidecarsCrossplane:
  - name: pipeline-inspector
    image: xpkg.crossplane.io/crossplane/pipeline-inspector-sidecar:v0.1.0
    args:
      - --socket=/var/run/pipeline-inspector/socket
      - --max-recv-msg-size=8388608  # 8MB
    volumeMounts:
      - name: pipeline-inspector-socket
        mountPath: /var/run/pipeline-inspector
    resources:
      requests:
        cpu: 10m
        memory: 64Mi
      limits:
        cpu: 100m
        memory: 128Mi

# Add the shared volume for Unix socket communication
extraVolumes:
  - name: pipeline-inspector-socket
    emptyDir: {}

extraVolumeMountsCrossplane:
  - name: pipeline-inspector-socket
    mountPath: /var/run/pipeline-inspector

With this configured, Crossplane will forward a copy of each RunFunctionRequest and RunFunctionResponse to the inspector sidecar as the pipeline runs, giving you full visibility into the data flowing through your function pipeline. This feature remains Alpha in v2.2 and will continue to be matured in future releases.

Runtime Configuration for Package Dependencies

Previously, if you needed to configure a package's runtime you had to set a runtimeConfigRef directly on each package resource. This worked for explicitly installed packages, but had no effect on packages installed as dependencies. This made it difficult to leverage Configuration packages as a top-level construct.

ImageConfig, introduced in Crossplane v1.18, has grown into a powerful centralized mechanism for configuring package installation. New in v2.2, ImageConfig can configure the DeploymentRuntimeConfig used by packages matching a given image prefix, including packages installed as dependencies. The new spec.runtime.configRef field lets you specify a runtime configuration and it applies to all matching packages regardless of how they are installed.

Consider a scenario where you're running on Azure and need your provider pods to use Azure Workload Identity. Previously, you might have disabled dependency resolution to install the provider manually. Now you can express this intent once, declaratively:

---
apiVersion: pkg.crossplane.io/v1beta1
kind: ImageConfig
metadata:
  name: azure-workload-identity
spec:
  matchImages:
    - prefix: xpkg.crossplane.io/crossplane-contrib/provider-azure-
    - prefix: xpkg.crossplane.io/crossplane-contrib/provider-family-azure
  runtime:
    configRef:
      name: azure-workload-identity

---
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: azure-workload-identity
spec:
  serviceAccountTemplate:
    metadata:
      annotations:
        azure.workload.identity/client-id: "12345678-1234-1234-1234-123456789012"
  deploymentTemplate:
    metadata:
      labels:
        azure.workload.identity/use: "true"
    spec:
      selector: {}
      template:
        metadata:
          labels:
            azure.workload.identity/use: "true"
        spec:
          containers:
            - name: package-runtime
              args: []

With this ImageConfig in place, all Azure family providers — whether installed explicitly or as dependencies — will get the azure-workload-identity runtime configuration applied, and therefore have Workload Identity configured on their service accounts. Note that a matching ImageConfig runtime takes precedence over any runtimeConfigRef specified directly on a package's spec.

To confirm the configuration was applied, check the appliedImageConfigRefs field in the package revision's status:

$ kubectl get providerrevision crossplane-contrib-provider-family-azure-e57d1e7b1ce7 -o yaml
# ...
status:
  appliedImageConfigRefs:
  - name: azure-workload-identity
    reason: ConfigureRuntime
# ...

XRD Validation Beyond the Spec

Crossplane XRDs let you define rich OpenAPI schemas for the resources your platform users create. Until now, x-kubernetes-validations (CEL validation rules) could only be used to validate fields within the spec of a composite resource. This left a real gap for platform builders who wanted to enforce naming conventions, require specific labels, or validate other metadata properties at admission time.

Crossplane v2.2 closes this gap by allowing XRDs to configure x-kubernetes-validations outside of spec. This lets you write CEL rules that validate things like the resource's name format or required label values and provide a more complete API contract for your composite resources.

For example, suppose your platform requires all XR names to follow a specific pattern. You can now enforce this directly in the XRD:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: databases.platform.example.org
spec:
  group: platform.example.org
  names:
    kind: Database
    plural: databases
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          x-kubernetes-validations:
            - rule: "self.metadata.name.startsWith('db-')"
              message: "Database names must start with 'db-'"
          properties:
            spec:
              type: object
              properties:
                region:
                  type: string

Any attempt to create an Database with a name that doesn't start with db- will now be rejected at the Kubernetes API server level, giving your users clear, immediate feedback rather than a confusing reconciliation error later.

Using OpenAPI Schemas in Functions

Composition and operation functions may wish to know the OpenAPI schemas for the resources they're working with, for example to make schema-aware decisions, validate resources, or dynamically generate resources. Previously a function could indirectly get the OpenAPI schema for a CRD by requesting the CRD as a RequiredResource (or ExtraResource in Crossplane 1.x). However, schemas were not available for built-in Kubernetes kinds and some manipulation was necessary to extract the OpenAPI schema from the CRD.

Crossplane v2.2 introduces RequiredSchemas in the RunFunctionResponse, allowing functions to request schemas. Analogous to RequiredResources, a function can populate the new field with the API group/version/kind of any resources whose OpenAPI schema it needs and on the next reconciliation Crossplane will include the requested schemas in the function request.

With this expansion of the function gRPC API, Crossplane v2.2 also introduces capability advertisement. The RunFunctionRequest now includes a Capabilities field that tells the function what the running version of Crossplane supports. For example, a function that can optionally use OpenAPI schemas may check whether Capabilities contains CAPABILITY_REQUIRED_SCHEMAS to determine whether schemas are available.

Improvements to crossplane beta trace

Many platform engineers use the crossplane beta trace command is a to understand the state of resources in a Crossplane control plane. v2.2 ships two significant enhancements to make it even more useful.

Trace all resources of a given kind. Previously, crossplane beta trace could take only a single resource instance. Now you can pass a kind (and optionally a namespace) to trace all resources of that type at once. This is helpful when you want to quickly understand the health of every Database or every Provider in your cluster:

# Trace all Database resources in the platform namespace
crossplane beta trace databases.platform.example.org -n platform

# Trace all Providers
crossplane beta trace providers

Watch mode. The trace command now supports a --watch flag (shorthand -w), which keeps the trace output live and refreshes it as resources change, similar to kubectl get --watch. This is especially useful during debugging sessions when you're waiting for a resource to reconcile and want to see the dependency tree update in real time:

crossplane beta trace databases.platform.example.org db-production --watch

Notable Breaking Changes

Before upgrading, be aware of two breaking changes in v2.2:

Function input CRDs are no longer installed. Input CRDs included in Function packages are no longer installed by the package manager, following the xpkg specification. Additionally, unknown or disallowed resource types within a package are now silently ignored instead of causing package installation to fail.

Package cache on-disk structure has changed. The internal structure of the package cache has been updated. This breaks an undocumented behavior that allowed packages to be "side-loaded" directly into Crossplane, which was sometimes used for testing. If you relied on this behavior, please see #6981 and #7147 for details on the necessary changes.

Get Involved

We love contributions from the community in any form: code contributions, issues, questions, feedback on proposals, and many more. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: