Crossplane providers have always paid a memory tax that grows with the size of the cluster they run in, not with the number of resources they manage. A provider watching ten managed resources on a cluster with two thousand CRDs and a few thousand Secrets could sit at well over a gigabyte of RSS while doing almost nothing. With crossplane-runtime v2.4.0, released on 2026-08-20 alongside Crossplane v2.4, and the provider releases that consume it, that tax is largely gone: on a cluster with ~2,000 CRDs, provider memory drops by roughly 80-95% depending on the provider.

The crossplane-runtime side of this story is a community contribution. Rafał Jan (@rafal-jan) debugged where provider memory was going, wrote it up in crossplane-runtime#1056, implemented the fix in crossplane-runtime#1058 and carried it into provider-template. Thank you, Rafał.

Where the Memory Went

Every provider reads its own control plane through controller-runtime informer caches, and two of those caches were far bigger than intended:

• The CRD cache. Providers that implement safe-start watch CustomResourceDefinitions, and the default cache stores every CRD in the cluster in full, complete OpenAPI schema included. The safe-start gate only needs the CRD's group, kind and Established condition. With the Upjet AWS family installed that is 2,045 CRDs of schema held by every provider pod, whether or not it owns any of them.
• The Secret cache. The first Secret read through the manager client starts a cluster-wide Secret informer that caches every Secret in the cluster, most of which the provider will never look at again.

All figures below come from a reconcile load test: a kind cluster with the provider-upjet-aws CRD set (~2,000 CRDs) and 3,000 Secrets, 300 managed resources (Certificate.applications.azuread.m.upbound.io) each referencing a Secret and publishing a connection Secret, three update rounds, averaged over the settle phase and the rounds. baseline is v2.4.0 release before the crossplane-runtime update and runtime v2.4 adds the runtime v2.4.0 defaults (CRD schema strip on, Secret cache off).

Figure 1. provider-upjet-azure, baseline versus the runtime v2.4 defaults. Peak heap in use falls by 83% and peak RSS by 78%; requests per round grow by 12-18% because Secret reads leave the informer.

What Changed in crossplane-runtime v2.4

The CRD fix is a cache transform, customresourcesgate.TransformStripCRDSchema, that drops the schema, managedFields and the last-applied annotation from every CRD before it enters the informer cache. A provider opts in with one entry in its manager's cache options:

Cache: cache.Options{
    ByObject: map[client.Object]cache.ByObject{
        &apiextensionsv1.CustomResourceDefinition{}: {
            Transform: customresourcesgate.TransformStripCRDSchema,
        },
    },
},

On a cluster with 2,045 Upjet AWS CRDs that alone takes a provider from ~630 MiB to ~83 MiB. The one thing to remember is that a CRD read through this cache is stripped: never write it back with a full-object Update.

The Secret cache is a trade-off, so it stays on by default and each provider gains a flag, --enable-secret-cache. With --no-enable-secret-cache, Secret reads through the manager client become live API calls: memory stops scaling with the number of Secrets in the cluster, and the provider issues one extra request per Secret read per reconcile. Turn it off when the cluster holds thousands of Secrets the provider never touches; leave it on if the API server is already busy.

provider-kubernetes and provider-helm: Caching Target-Cluster Clients

provider-kubernetes and provider-helm also build a client per ProviderConfig to talk to the target cluster, and they build it fresh on every reconciliation. Since controller-runtime v0.20.0 (kubernetes-sigs/controller-runtime#2901) a new client's first lookup primes its RESTMapper with full aggregated discovery, which on a cluster with 1,500-2,000 CRDs is a multi-megabyte download per managed resource per reconcile. provider-kubernetes#541 measured the result: ~2.1 GiB peak and ~1,000m of CPU for 300 Objects, against ~0.6 GiB on v1.2.1, with the API server serving 614 discovery requests per round where v1.2.1 made 3.

provider-kubernetes#543 caches the built clients, keyed by a digest of the credential material so that ProviderConfigs sharing a kubeconfig share a client and a rotated credential gets a fresh one. provider-kubernetes#560 sizes the cache automatically: at startup the provider lists its ProviderConfigs, counts the distinct credential sets and bounds the cache at that number plus headroom, so there is no knob to tune. The cache reports provider_kubernetes_client_cache_size, provider_kubernetes_client_cache_entries and provider_kubernetes_client_cache_events_total (hit, miss, evict); a sustained eviction rate is the signal that more credential sets appeared than the cache was sized for and a restart will re-size it. provider-helm imports the same package and inherits all of it on its next dependency bump.

All provider-kubernetes figures come from a reconcile load test: a kind cluster with the provider-upjet-aws CRD set (~2,000 CRDs) and 3,000 Secrets, 300 Objects each managing a ConfigMap and publishing a connection Secret, three update rounds, averaged over the settle phase and the rounds. baseline is v1.3.0 before either change, client cache adds #543, and client cache + runtime v2.4 adds the runtime v2.4.0 defaults (CRD schema strip on, Secret cache off).

Figure 2. Memory and requests per configuration. The client cache removes the discovery traffic and cuts allocation per round tenfold; the runtime v2.4 defaults then take peak heap from 450 MiB to 72 MiB, paid for with ~200 live Secret reads per round.

What You Need to Do

• Operators. Upgrade to a provider release with the runtime v2.4.0 wiring and the CRD saving comes with no configuration. Add --no-enable-secret-cache through your DeploymentRuntimeConfig if the cluster holds many Secrets the provider does not own and API server load is not your constraint. On provider-kubernetes and provider-helm there is nothing to size; watch provider_kubernetes_client_cache_events_total{event="evict"}.
• Provider authors. Follow provider-terraform#320: bump crossplane-runtime/v2, crossplane-tools and crossplane/apis/v2 to v2.4.0, register apiextensionsv1 in the manager scheme, add the ByObject transform and the flag. If your provider builds clients for other clusters on Connect, cache them.

Rolling It Out Across the Ecosystem

Neither change does anything until a provider wires it into its main.go, so the runtime release was followed by the same three-part change across the official and community providers: bump crossplane-runtime/v2, crossplane-tools and crossplane/apis/v2 to v2.4.0, build the scheme explicitly before the manager, and add the ByObject transform plus the the flag. The reference implementation is provider-terraform#320; the release notes point at provider-helm#392. Both templates carry the wiring too, so a provider generated from provider-template or upjet-provider-template starts with it.

Upjet-based Providers

Native Providers

• Release: next major/minor release will contain the performance improvements

Two things bit during the rollout and are worth knowing if you maintain a provider. The pinned crossplane-tools commit raises k8s.io/client-go to v0.36.x through minimal version selection, and controller-runtime v0.23.x does not build against it (cache.ResourceEventHandlerRegistration gained HasSyncedChecker); bump controller-runtime to v0.24.1 in the same change. That bump in turn deprecates sigs.k8s.io/controller-runtime/pkg/scheme.Builder, so hand-written register.go files move to runtime.NewSchemeBuilder + metav1.AddToGroupVersion.

Get Involved

We love contributions from the community in any form: code contributions, issues, questions, feedback on proposals, and many more. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods:

• Crossplane website
• GitHub
• Slack
• LinkedIn
• YouTube
• Reddit
• Email
• Attend a community meeting

Crossplane is a powerful tool for building platforms, but building a platform on top of Crossplane has never been as easy as it should be. First, there's the conceptual complexity: how do composite resource definitions, compositions, composition functions, and managed resources all fit together? Then comes the practical challenge: how should a platform team build, test, package, and distribute all the parts that make up a Crossplane platform? Many platform teams have invented their own solutions to this problem.

Crossplane CLI v2.3 introduced a set of developer experience (DevEx) tools built around a first-class concept called a Project: an opinionated, on-disk layout for everything that makes up a platform API. The CLI provides commands to scaffold, generate, test, and run projects locally. This post explains why we built these tools and walks through building a small platform API from scratch with them.

Why we built this

A minimal Crossplane platform looks like configuration: a set of YAML files that get packaged and deployed into a cluster. However, as a platform grows more complex, its logic often outgrows the configuration paradigm and starts being encapsulated in functions. At this point, a platform starts looking less like configuration and more like software.

Software projects generally have a clear distinction between the development artifact (source code) and the deployment artifact (binaries, packages, or container images). The Crossplane developer experience design started from this question: what should the development artifact for a platform look like? Our answer is the Crossplane Project.

A Project encapsulates configuration (XRDs, Compositions, Operations, etc.) and code (functions) into one development artifact. A single CLI command builds the Project into a set of Crossplane packages: a Configuration containing the configuration resources, and one or more Functions. Dependencies are managed automatically by the tooling, versioning the whole Project as one unit.

The tooling can also generate language bindings for managed resources from Crossplane providers, arbitrary CRDs, and core Kubernetes APIs. This makes building composition functions easier and safer by allowing you to use a wide variety of existing development tools for your preferred language. It also lets AI agents more effectively build, review, and debug Crossplane platforms with their extensive training on the general purpose programming languages we support. Today, those languages are Go, templated YAML (with Go templating syntax), Python, and KCL. TypeScript support is in progress and we welcome contributions of other languages from the community.

Prerequisites

You'll need the Crossplane CLI and a Docker-compatible container runtime. The CLI builds your functions and spins up a local development control plane in a KIND cluster, both of which need Docker. You don't need an existing Kubernetes cluster.

The platform we'll build exposes a WebApp API. A user creates a WebApp:

apiVersion: platform.example.com/v1alpha1
kind: WebApp
metadata:
  name: podinfo
  namespace: default
spec:
  image: docker.io/stefanprodan/podinfo:6.11.0
  replicas: 3
  ports: [9898]

Crossplane will turn the WebApp into a Deployment and a Service. This is a simple platform, but enough to show the power of the tooling.

Create the project

A project is a directory with a standardized layout and some metadata. The init command scaffolds it for you:

crossplane project init example-project-webapp
cd example-project-webapp

That gives you some empty directories and a crossplane-project.yaml metadata file:

example-project-webapp
├── apis
├── crossplane-project.yaml
├── examples
├── functions
├── operations
└── tests

Every project looks essentially the same: apis/ holds your XRDs and compositions, functions/ holds embedded function source, examples/ holds sample resources you render and test against, and crossplane-project.yaml records metadata and dependencies. The directory names are customizable in the metadata file if you don't like the defaults.

Define the API

Crossplane calls a composition-powered custom resource a composite resource (XR), and you describe its schema with a composite resource definition (XRD). Rather than write the XRD by hand, you can describe the API you want and let the CLI generate it. Here we'll write a SimpleSchema document for our WebApp API to apis/webapps/schema.yaml:

apiVersion: platform.example.com/v1alpha1
kind: WebApp
spec:
  image: string | required=true description="OCI image for the webapp"
  replicas: integer | default=1 minimum=1 maximum=100 description="Number of replicas to run"
  ports: "[]integer | default=[80] description=\"Ports to expose from the application container\""

Then generate the XRD:

crossplane xrd generate apis/webapps/schema.yaml --from simpleschema

The CLI writes a complete XRD to apis/webapps/definition.yaml, translating your types, defaults, validation rules, and descriptions into an OpenAPI schema. If you'd rather not learn SimpleSchema, you can point xrd generate at an example XR instead and it will infer the types from the values. Either way, you don't have to write the OpenAPI schema by hand.

Generate the composition

A composition tells Crossplane what to do when someone creates a WebApp. Generate a starting point from the XRD:

crossplane composition generate apis/webapps/definition.yaml

This produces a composition with a single pipeline step running function-auto-ready, which marks the WebApp ready once its composed resources are ready. Notice that the command also added function-auto-ready to your project's dependencies automatically.

Add a dependency

Our function is going to build Deployment and Service objects, so it needs the Kubernetes core API schemas. Add them as a dependency:

crossplane dependency add k8s:v1.35.0

The dependency add command generates language bindings for the dependency and records it in crossplane-project.yaml. These bindings give you autocompletion, hover docs, and type checking when you write your function. If your platform composes cloud resources you can add a Crossplane provider; for our example we just need Kubernetes.

Write the function — in your language

The biggest simplification we've made to platform development with Projects is that the functions live in the project, not a separate codebase. The CLI can scaffold an embedded function and wires it into your composition pipeline.

For this walkthrough we'll use Python.

crossplane function generate compose-webapp apis/webapps/composition.yaml --language python

That scaffolds functions/compose-webapp/ and adds a pipeline step to the composition. Paste the following code into functions/compose-webapp/fn/fn.py:

"""A Crossplane composition function."""

import grpc
from crossplane.function import logging, response, resource
from crossplane.function.proto.v1 import run_function_pb2 as fnv1
from crossplane.function.proto.v1 import run_function_pb2_grpc as grpcv1

from models.com.example.platform.webapp import v1alpha1
from models.io.k8s.api.apps import v1 as appsv1
from models.io.k8s.api.core import v1 as corev1
from models.io.k8s.apimachinery.pkg.apis.core.meta import v1 as metav1
from models.io.k8s.apimachinery.pkg.util import intstr

class FunctionRunner(grpcv1.FunctionRunnerService):
    """A FunctionRunner handles gRPC RunFunctionRequests."""

    def __init__(self):
        """Create a new FunctionRunner."""
        self.log = logging.get_logger()

    async def RunFunction(
        self, req: fnv1.RunFunctionRequest, _: grpc.aio.ServicerContext
    ) -> fnv1.RunFunctionResponse:
        """Run the function."""
        log = self.log.bind(tag=req.meta.tag)
        log.info("Running function")

        rsp = response.to(req)

        xr = v1alpha1.WebApp(**resource.struct_to_dict(req.observed.composite.resource))

        assert xr.metadata is not None
        assert xr.metadata.name is not None
        assert xr.spec.ports is not None

        labels = {"app.kubernetes.io/name": xr.metadata.name}
        ports = xr.spec.ports

        dply = appsv1.Deployment(
            metadata=metav1.ObjectMeta(
                labels=labels,
            ),
            spec=appsv1.DeploymentSpec(
                selector=metav1.LabelSelector(
                    matchLabels=labels,
                ),
                replicas=xr.spec.replicas,
                template=corev1.PodTemplateSpec(
                    metadata=metav1.ObjectMeta(
                        labels=labels,
                    ),
                    spec=corev1.PodSpec(
                        containers=[
                            corev1.Container(
                                name="app",
                                image=xr.spec.image,
                                ports=[corev1.ContainerPort(containerPort=p) for p in ports],
                            )
                        ]
                    ),
                ),
            ),
        )

        resource.update(rsp.desired.resources["deployment"], dply)

        svc = corev1.Service(
            metadata=metav1.ObjectMeta(
                labels=labels,
            ),
            spec=corev1.ServiceSpec(
                ports=[corev1.ServicePort(port=p, targetPort=intstr.IntOrString(p)) for p in ports],
                selector=labels,
            ),
        )

        resource.update(rsp.desired.resources["service"], svc)

        return rsp

The function reads the observed WebApp XR, then builds a Deployment and a Service from its spec. The models packages are the type bindings the CLI generated when you added the Kubernetes dependency, so you build the resources with typed Python classes instead of raw dictionaries. Because the function is a regular Python project, you can use any tools you like from the Python ecosystem, depend on other Python packages, and take advantage of AI tools that excel at writing Python code.

Render before you run

Before spinning anything up, preview what your composition produces:

crossplane composition render examples/webapps/podinfo.yaml apis/webapps/composition.yaml

Inside a project, render discovers and builds your embedded functions automatically, then runs the exact same composition logic a real control plane would. It prints the Deployment and Service your function generates, plus the updates Crossplane would make to the WebApp itself. This is the tightest feedback loop in the workflow: change your function code, re-run render, and see the result. Because the function is embedded in your project, you don't need to build or run it separately.

Run it on a local control plane

When the rendered output looks right, run the whole thing for real:

crossplane project run

The project run command creates a local development control plane in a KIND cluster with a local OCI registry, builds your embedded function into a Function package, builds a Configuration package from your XRD and composition (with dependencies on your embedded function and function-auto-ready), pushes both to the local registry, installs the configuration, and points your kubectl context at the new control plane. The first run takes a little while to create the cluster; subsequent runs will be faster as the cluster gets reused.

Now create a WebApp:

kubectl apply -f examples/webapps/podinfo.yaml

Once the composition runs, you'll see the Deployment and Service show up in the default namespace:

$ kubectl get deployment,service -l app.kubernetes.io/name=podinfo
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/podinfo   3/3     3            3           45s

NAME              TYPE        CLUSTER-IP      PORT(S)    AGE
service/podinfo   ClusterIP   10.96.142.110   9898/TCP   45s

And when they become ready, function-auto-ready will mark the XR as ready, too:

$ kubectl get webapp podinfo
NAME      SYNCED   READY   COMPOSITION                    AGE
podinfo   True     True    webapps.platform.example.com   45s

Edit the WebApp's replicas or image, apply it again, and Crossplane reconciles the Deployment to match. When you're done, delete the WebApp and tear the control plane down:

kubectl delete -f examples/webapps/podinfo.yaml
crossplane project stop

What's next

The initial set of Crossplane DevEx commands covers the basics of building a platform: defining APIs, building compositions, writing functions, and testing them manually. But, platform teams building real things need tools that go beyond day one.

We're working on adding automated testing to the set of tools. This will encompass both render-based tests (see the xprintool for inspiration), and end-to-end tests that build on top of crossplane project run to validate the behavior of a platform in a real cluster.

Work has also started on simulations, giving users the ability to preview changes without actually applying them at both the Crossplane level and the provider level. See the design document for details.

We would love contributions from the Crossplane community on both these efforts, and more. Once you've tried out the current tooling we would love to hear what's missing and what will make it an even better experience for building platforms. Join us in the Crossplane Slack (DevEx is is mostly discusssed in #sig-cli) or file an issue in the crossplane/cli GitHub project.

Try it out

The Crossplane CLI DevEx tools standardize the workflow of building platforms, so that you don't have to reinvent it for your own team. You get a standard project layout, embedded functions in the language of your choice, generated bindings for building functions, a fast local render loop, and a one-command development control plane.

For a more detailed tutorial, including examples in all supported languages, follow the Get Started with Control Plane Projects guide.

I've worked on Crossplane for almost eight years, since the v0.1 release. In that time I've watched a lot of people use it to put cloud infrastructure behind an API. For the last few months I've been using it to put a particular, demanding kind of infrastructure behind an API: a fleet of GPUs running model inference.

The project is called Modelplane. It lets a platform team turn a pile of accelerators (across clouds, neoclouds, and on-premise) into one fleet. It also lets the ML teams they support deploy a model and get a stable, OpenAI-compatible endpoint without thinking about where it runs.

Modelplane exists because open-weight models have moved inference out of the labs and hyperscalers and into everyone else: neoclouds, regulated enterprises keeping models inside their own walls, and companies trying to get their inference bills under control. The open source stack for serving a model on a single cluster is strong now: vLLM, SGLang, KEDA, Gateway API, DRA. But inference almost never stays in one cluster. Capacity is scattered across hardware types, providers, and regions. The hard problems are now above the cluster: placing models across the capacity you have, provisioning more, routing by cost and locality, moving weights around. The labs and the hyperscalers all built systems to do this, but they built them privately. That's the gap Modelplane fills: an open control plane that sits above your clusters and operates them as one inference fleet.

If the inference part interests you, the Modelplane docs and Bassam's introduction are the place to go. This post is for the Crossplane crowd, because the part I think you'll find interesting is that Modelplane is, top to bottom, a Crossplane configuration. It has no bespoke controllers and no custom operators: it's compositions and composition functions. The same primitives you could use to compose an RDS instance, pushed a lot harder.

I want to cover the problem we set out to solve with Crossplane, the parts of the framework we leaned on hardest, and the edges we hit and fixed upstream.

The problem, in Crossplane terms

Strip away the inference vocabulary and Modelplane's job is one Crossplane users will recognize: take a declarative description of what someone wants, and turn it into composed infrastructure spanning cloud accounts, many Kubernetes clusters, and the workloads on them. Provision an EKS or GKE cluster with the right GPUs. Install an inference stack onto it. Decide which cluster each model runs on, and how many copies. Keep it all converged as clusters come and go and people's inference needs change.

Crossplane was built for that shape of problem. Providers gave us reach: we provision clusters and the infrastructure they need across different clouds, and install software onto them, without needing to write new controllers. Functions allowed us to focus on our business logic, the placement and the scheduling. We didn't have to write the controller plumbing by hand: the watches and requeues and finalizers and drift correction that Crossplane core already handles.

Crossplane v2 helped here too. Modelplane has two clear personas. Platform teams describe the fleet. ML teams describe a model. That split maps onto a scope boundary: an InferenceCluster or InferenceClass is cluster-scoped, a ModelDeployment or ModelService is a plain namespaced composite resource the ML team owns. v2 namespaced composites let us express that directly, with no claim-and-XR duality to explain. That's useful, but it isn't what made the project buildable.

What made it buildable: Developer experience

What really unlocked this project was the new Crossplane CLI and the schemas it generates.

Modelplane's functions are all written in Python. We chose Python because it's the lingua franca of the ML world. We hope it might help folks who aren't yet cloud native experts contribute to the project. Writing functions in Python used to mean giving up a lot of the tooling that makes a codebase feel like a proper project. The new crossplane CLI changed that. It scaffolds a project, generates an XRD from an example resource, and generates typed schema bindings for your APIs.

Those generated models changed how we worked. Our functions read and write typed objects instead of poking at untyped dictionaries and hoping the field is spelled the way we remember. A typo or a wrong type now fails at author time. The models also sped up the coding agents we leaned on while building. A generated type tells the agent the exact shape, so it got field names and types right the first time.

There was friction. We outgrew the CLI's built-in function builders early, and we needed schema generation for one language, not all four. Both of those turned into upstream contributions, which I'll come back to.

Designing the API

The hardest part of Modelplane was designing the API.

People come up to me at conferences worried about how they'll make breaking changes to the APIs they build with Crossplane. My answer is usually that you almost never have to, if you really think the API through before you release it. That discipline pays off: reach for arrays and enums before you think you need them, use required fields sparingly, and leave room to grow without a breaking change.

Take the ModelDeployment, arguably Modelplane's most important API. It's how an ML team describes a model to serve: its engines, what their pods need from a node, and how many replicas to run across the fleet.

apiVersion: modelplane.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: qwen3-8b
  namespace: ml-team
spec:
  replicas: 1
  engines:
  - name: qwen3-8b
    members:
    - role: Standalone
      nodeSelector:
        devices:
        - name: gpu
          count: 1
          selectors:
          - cel: device.capacity["gpu.nvidia.com"].memory.compareTo(quantity("20Gi")) >= 0
      template:
        spec:
          containers:
          - name: engine
            image: vllm/vllm-openai:v0.23.0
            args:
            - "--model=Qwen/Qwen3-8B"
            - "--served-model-name=qwen"
            # ... engine flags omitted for brevity

I got the engines array wrong at first, and only caught it about two weeks before we released v0.1. Until then a ModelDeployment had a spec.topology block where you'd write tensor: 8 and pipeline: 2. Modelplane would derive engine-specific flags like --tensor-parallel-size and inject them. The problem was that this coupled Modelplane to engine specifics. We could only run the engines whose flags we knew how to inject. It also couldn't express the data and expert parallelism we knew was coming. I caught it only because I sat down to write worked examples for those topologies and found I couldn't. I replaced it with shape: an engine is an array of Standalone, Leader, or Worker members, and the parallelism lives entirely in the flags the user writes. My takeaway: don't rush API design. Work through it with your users and peers, let it sit if you can, and write enough worked examples to confirm you can model everything they need before you commit.

The functions do the work

One thing I really like about composition functions is that they scale with your problem. At the low end a function is a few Go templates or some KCL inlined in the Composition. At the high end it's a program that can do almost anything a traditional Go operator can. Python rides that whole range too, and Modelplane is what the far end looks like once the problem grows.

Modelplane's business logic is all composition functions. Scheduling is an interesting example. I'll walk through it because it shows how far a composition function can be pushed.

Modelplane runs a fleet scheduler in the control plane. Each cluster in the fleet is an InferenceCluster, and each of its node pools references an InferenceClass that declares the pool's hardware. When an ML team creates a ModelDeployment, the fleet scheduler places each replica against that declared node pool capacity, before any nodes necessarily exist. That's a different job from in-cluster DRA, which is a runtime allocator: its drivers publish ResourceSlices about real hardware on real nodes.

Scheduling happens in two layers. The fleet scheduler picks the cluster and node pool; the cluster's own in-cluster scheduler and DRA then place pods on nodes and bind GPUs. That in-cluster scheduler is good at its layer (a single cluster supports up to 5,000 nodes), but a fleet isn't one cluster. It can't be: a cluster doesn't span clouds, and you wouldn't want one blast radius over your whole accelerator footprint even if it could.

Modelplane borrows DRA's vocabulary and lifts it to the fleet layer. DRA's typed, domain-prefixed attribute model and CEL predicate language are a good fit, and they express cleanly in OpenAPI. A platform team's InferenceClass declares what a pool's hardware offers, the same shape DRA uses to describe a real device:

devices:
- name: gpu
  claim: DRA
  driver: gpu.nvidia.com
  deviceClassName: gpu.nvidia.com
  attributes:
    architecture: { string: Ada Lovelace }
  capacity:
    memory: { value: "23034Mi" } # the L4's real usable VRAM

A deployment's nodeSelector then asks for what it needs with a CEL expression:

selectors:
- cel: device.capacity["gpu.nvidia.com"].memory.compareTo(quantity("20Gi")) >= 0

Our scheduler evaluates that CEL against an InferenceClass's declared attributes, and DRA evaluates the same expression against a real GPU when the pod binds. The same expression runs at both levels.

A composition function is, by design, a pure function of its inputs. Crossplane hands it the observed composite, and the function returns the desired children. Our scheduler is exactly that, a pure function of observed state, which is what makes it safe to run on every reconcile. A scheduler can't decide placement from the deployment alone, though. It has to see the whole fleet: every InferenceCluster and its published capacity, and every ModelReplica that already exists, including those of other deployments, so it can account for capacity they've already consumed.

That's require_resources (what some of you will know as extra resources), and we couldn't have built the scheduler without it. The function asks Crossplane for the resources it needs to reason over, and Crossplane fetches them and calls the function again with them in hand:

# Every InferenceCluster: candidate clusters with their declared capacity.
response.require_resources(
 rsp, name="clusters",
 api_version="modelplane.ai/v1alpha1", kind="InferenceCluster",
 match_labels=clusters_match_labels,
)
# Every ModelReplica across all deployments: capacity already in use.
response.require_resources(
 rsp, name="all-replicas",
 api_version="modelplane.ai/v1alpha1", kind="ModelReplica",
)

That second call, asking for every resource of a kind with no name or label filter, is one of the things we had to fix upstream.

Where Modelplane pushed Crossplane, and we fixed it upstream

Build something this demanding on Crossplane and you find its rough spots. Modelplane found several. Rather than work around them, we sent fixes upstream, so your configurations get the benefit too.

The first thing we hit was the build itself. The CLI's built-in function builders are great until you outgrow them, and we needed to coordinate function builds with Nix, the rest of our build system. We also only consume one of the four schema languages the CLI generates. We added an explicit functions list that loads pre-built image tarballs instead of building them, and a schemas block that restricts generation to the languages you actually use (crossplane/cli #24). Now our functions build with the same tooling as everything else, and we generate Python and nothing else.

The DRA-style attribute model then walked us into a chain of three related bugs, all rooted in the same thing: a DeviceAttribute has fields named exactly int, bool, string, and version. Those are Python keywords and builtins, and they broke at every layer. First, schema generation emitted Python models that referenced undefined type aliases and wouldn't import at all (crossplane/cli #64); the fix was to bump the code generator to a version that sanitizes such names and preserves the wire name with a Pydantic alias. With the models importable, serialization was next: the SDK emitted bool_: true under the Python attribute name instead of bool: true under the wire alias, so the API server rejected it. Passing by_alias=True, and switching from exclude_defaults to exclude_unset so we serialize the fields the caller actually set rather than the fields that differ from a default, put both right (function-sdk-python #208, with helpers in #205).

The fleet scheduler also needed something the framework couldn't yet express. It has to reason over every InferenceCluster and every ModelReplica in the control plane, not one resource by name or a set by a label match. A require_resources selector with no match field is the natural way to say "all resources of this kind," and the protobuf allows it, but Crossplane rejected it on both the wire and the SDK side. We taught the reconciler to treat an empty selector as match-all (crossplane/crossplane #7241) and relaxed the SDK's validation to match (function-sdk-python #213). That match-all call is now the first thing the scheduler does every reconcile.

Testing functions this complex surfaced one more. crossplane render reimplements the XR reconciler's composition pipeline, and that parallel copy drifts from the real one, so a function can pass render and behave differently in a real control plane. We added a hidden render engine that runs the actual reconciler against an in-memory client, so tools can test against the real composition pipeline rather than a copy of it (crossplane/crossplane #7280).

Finally there's a one-pager up for function-controlled deletion (#7242), which aims to make ordered deletion of composed resources within one XR easier and more expressive than using Usages.

None of these are glamorous. But that's how a framework gets better: you build something hard on it and fix the friction.

What I took away

I went into Modelplane wanting to know whether you could build something this demanding entirely on Crossplane, without dropping to a hand-written controller when the going got hard. You can, though we had to fix the framework in a few places to get there. It's still compositions and functions any adopter can read and extend.

Modelplane is Apache 2.0 and developed in the open. If you run accelerators of any kind, take a look at modelplane.ai, and if you build hard things on Crossplane, find its edges and help make them better.

TL;DR

• The vast majority of v1.x control planes can upgrade to Crossplane v2 right away with no changes. Claims, composite resources (XRs), Compositions, and cluster-scoped managed resources all keep working.
• A small set of v1 features were removed or changed in v2. If your control plane uses one, you'll want to find that out before you upgrade.
• crossplane beta upgrade check is a new, read-only CLI command that scans a live v1.x control plane and reports exactly what (if anything) would break, which resource is responsible, and how to fix it.
• It ships in the v1.20 CLI, so install the latest v1.20 CLI and run it against your cluster before you upgrade.

Ready to upgrade?

When we designed Crossplane v2, backward compatibility was a first-class goal: the vast majority of v1.x control planes can upgrade without changing a thing. Your claims, composite resources (XRs), compositions, and cluster-scoped managed resources all keep working in v2.

But "the vast majority" isn't "everyone." Crossplane v2 does remove or change the behavior of a small set of v1 features, and if your control plane uses any of them, you'll want to catch it before you upgrade. Until now, that meant reading the backward compatibility notes and then spelunking through every composition, package, and resource in your cluster for something that's going away - tedious and easy to get wrong on a control plane of any real size. Understandably, that uncertainty has kept some folks from upgrading to v2.

crossplane beta upgrade check is now available to give you a definitive answer. It's a read-only command that connects to a live v1.x control plane and exhaustively scans it for anything v2 removes or changes in a breaking way. Point it at your cluster before you upgrade and it takes you straight to the resources that need attention, with the exact fix for each.

What it checks

The command covers every breaking change called out in the v2 backward compatibility notes. For each one, it reports the specific resources responsible so you can act on them directly:

• Native patch-and-transform Compositions. Composition Functions are how you compose resources in Crossplane v2. Native patch-and-transform (P&T) Composition, where resources and patches are embedded directly in the Composition, is removed. This is the change most likely to affect you, so any Composition still using it needs to move to a function pipeline.
• ControllerConfig usage. The ControllerConfig type is removed, replaced by DeploymentRuntimeConfig. The check finds both ControllerConfigs themselves and the Providers or Functions that still reference one.
• External secret stores. This alpha feature is removed in v2. The check finds where it's enabled and where your Compositions and resources still publish connection details through it.
• Unqualified package sources. Crossplane v2 drops the implicit default registry, so every package reference (Providers, Configurations, Functions, and their dependencies) must be fully qualified with its registry hostname.
• Composite resource connection details. This one is informational. Legacy XRs and Claims keep aggregating connection details in v2, so the upgrade itself needs no action here. The check highlights resources you'd only need to revisit if you later migrate them to v2-style namespaced XRs, which do not support connection details at the XR level.

The command is also careful about false positives. Crossplane's controllers and the API server inject defaults onto every cluster, and the check knows to look past those and flag only the features you actually opted into. A finding means real usage you'll need to address.

Let's see it in action

Running the command is as simple as pointing it at a cluster. By default it uses your current kubeconfig context and sweeps the entire control plane: all of your Crossplane configuration, cluster-scoped resources, and every namespace:

crossplane beta upgrade check

On a control plane that's already clean, you get the answer you're hoping for and a zero exit code:

[✓] 0 issues, 0 informational, 0 incomplete checks.

There we go - ready to upgrade. On a control plane that still leans on some v1-only features, the report tells you precisely what stands in the way. Here's a representative output from a control plane that is still using native patch-and-transform and external secret stores:

[✗] 4 issues, 0 informational, 0 incomplete checks.

[✗] Native patch-and-transform Compositions - 3 issues
    │
    │  Crossplane v2 removes native patch-and-transform (P&T) Composition. Compositions must use mode: Pipeline with Composition Functions.
    │  Fix:   Migrate to Composition Functions (spec.mode: Pipeline with spec.pipeline steps). Run "crossplane beta convert pipeline-composition" to convert existing Compositions.
    │  Docs:  https://docs.crossplane.io/latest/guides/upgrade-to-crossplane-v2/#native-patch-and-transform-composition
    │         https://docs.crossplane.io/v1.20/cli/command-reference/#beta-convert
    │
    │    NAME                                                            FIELD
    │    composition.apiextensions.crossplane.io/nativepnt-composition   .spec.mode
    │    composition.apiextensions.crossplane.io/nativepnt-composition   .spec.patchSets
    │    composition.apiextensions.crossplane.io/nativepnt-composition   .spec.resources
    └──
[✓] ControllerConfig usage
[✗] External secret stores - 1 issue
    │
    │  Crossplane v2 removes support for external secret stores. Publish connection details as Kubernetes Secrets composed by your Compositions, or adopt External Secrets Operator if
    │  you need an external store.
    │  Fix:   Disable --enable-external-secret-stores on the Crossplane Deployment, replace StoreConfig-based publishing with composed Kubernetes Secrets (or adopt External Secrets
    │         Operator), then delete StoreConfig resources. No automated converter exists.
    │  Docs:  https://docs.crossplane.io/latest/guides/upgrade-to-crossplane-v2/#external-secret-stores
    │         https://docs.crossplane.io/latest/guides/connection-details-composition
    │         https://github.com/external-secrets/external-secrets
    │
    │    NAMESPACE           NAME                         FIELD
    │    crossplane-system   deployment.apps/crossplane   .spec.template.spec.containers[0].args
    └──
[✓] Composite resource connection details
[✓] Unqualified package sources

crossplane: error: blockers found

Each finding is self-contained. The top line summarizes the breaking change in plain language, the Fix: line tells you what to do about it - often naming the exact crossplane beta convert command that will migrate the resource for you - and the Docs: line links straight to the relevant section of the upgrade guide. Below that, a table lists every resource and field responsible, so there's no guesswork about which of your Compositions needs attention.

The command exits non-zero when it finds blockers and zero when it doesn't, and -o json emits the same findings as structured data:

crossplane beta upgrade check -o json

The exit code and JSON output make the check easy to automate anywhere it has access to your cluster: a readiness gate in a CD or promotion pipeline, or a scheduled job watching a fleet of control planes. Most often, though, it's simply something you run by hand once, right before you upgrade.

Performance Tuning

Scanning every managed resource for external secret stores usage is the most expensive part of the run on control planes with many provider CRDs, and these flags let you make a trade off for execution time versus the load the command puts on your API server:

• --skip-managed-resources skips the managed-resource scan entirely. The command still inspects StoreConfig resources, the Crossplane Deployment, and composite resources for external secret stores usage; it just doesn't scan managed resources. This gives the biggest reduction in run time, at the cost of not catching external secret stores usage on managed resources.
• --concurrency (default 10) bounds how many resources the command processes in parallel. Lower it, for example --concurrency 2, to put less load on a busy production control plane you don't want to overload; raise it to finish faster at the cost of more load on the API server.

How to get it

Because this tool is only useful before you upgrade, it ships exclusively in the v1.20 Crossplane CLI - a v2 control plane has nothing left to check, and the v1-only types these checks rely on don't even exist there. We made a deliberate exception to v1.20's critical-fixes-only maintenance policy to ship this in the v1.20.9 patch release, precisely so the whole community can check their control planes before they make the jump.

If you're running a v1.x control plane, you likely already have the CLI on hand. You just need a build that includes upgrade check, which means the latest v1.20 patch release. Install it with:

curl -sL "https://raw.githubusercontent.com/crossplane/crossplane/main/install.sh" | XP_VERSION=v1.20.9 sh

You can also download the binary directly from releases.crossplane.io, or see Download other CLI versions in the docs for more options. Then confirm you're ready to roll:

crossplane beta upgrade check --help

When you're ready to dig into the changes themselves, the Crossplane v2 upgrade guide and the backward compatibility notes walk through each breaking change and how to migrate.

Upgrade with confidence

Most of you will run crossplane beta upgrade check, see a clean report, and upgrade to v2 the same day - which is exactly the outcome all that backward compatibility work was for. For everyone else, it trades guesswork for a precise list of what to change before you go. Either way, you upgrade deliberately, knowing exactly where your control plane stands.

This check is just a first step. It tells you what needs to change, and for some breaking changes it can already point you at a crossplane beta convert command that does the rewrite for you, but it stops short of migrating your control plane end to end. Smoothing that path is where the community should head next.

To lead that effort, we've started a new special interest group (SIG) for v2 migration. Drop into #sig-v2-migration to help shape where we invest in tooling, share best practices, and learn from others making the same move - and to tell us how the upgrade check is working for you, including any case it misses or flags incorrectly.

We love to hear from the community, as they are exactly what makes this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods:

• Website
• Slack
• GitHub
• LinkedIn
• YouTube

The combination of YAML+CEL provides a simple experience to define Kubernetes resources — declare what you want, wire dependencies with CEL expressions, and let the system figure out the order. It's an intuitive model, and we've been hearing from our community that they want this authoring experience inside Crossplane, alongside the many existing languages, functions, and experiences that Crossplane already supports.

We're excited to announce function-kro — a Crossplane composition function that brings the YAML+CEL authoring model of kro into Crossplane's pipeline architecture. Your kro-style resource definitions drop straight into a Composition pipeline step — same syntax, same CEL expressions, unchanged and now running inside Crossplane. function-kro embeds kro's graph builder, CEL evaluator, and runtime engine to offer full feature parity with the latest kro release.

function-kro has been donated to the Crossplane community as a community extension project at crossplane-contrib/function-kro. We've shared it with the kro community and we welcome collaboration and contributions from anyone interested in this experience!

The Function Pipeline

Below is a Crossplane Composition that uses function-kro to define a NetworkingStack platform API — a VPC, Subnet, and SecurityGroup, all wired together with CEL expressions:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: networkingstack
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1
    kind: NetworkingStack
  mode: Pipeline
  pipeline:
  - step: kro-run
    functionRef:
      name: function-kro
    input:
      apiVersion: kro.fn.crossplane.io/v1beta1
      kind: ResourceGraph
      status:
        networkingInfo:
          vpcID: ${vpc.status.atProvider.id}
          subnetID: ${subnet.status.atProvider.id}
          securityGroupID: ${securityGroup.status.atProvider.id}
      resources:
      - id: vpc
        template:
          apiVersion: ec2.aws.m.upbound.io/v1beta1
          kind: VPC
          metadata: {}
          spec:
            forProvider:
              region: ${schema.spec.region}
              cidrBlock: 192.168.0.0/16
              enableDnsHostnames: false
              enableDnsSupport: true
      - id: subnet
        template:
          apiVersion: ec2.aws.m.upbound.io/v1beta1
          kind: Subnet
          metadata: {}
          spec:
            forProvider:
              region: ${schema.spec.region}
              cidrBlock: 192.168.0.0/18
              vpcId: ${vpc.status.atProvider.id}
      - id: securityGroup
        template:
          apiVersion: ec2.aws.m.upbound.io/v1beta1
          kind: SecurityGroup
          metadata: {}
          spec:
            forProvider:
              name: my-sg-${schema.metadata.name}
              region: ${schema.spec.region}
              description: Default security group for NetworkingStack
              vpcId: ${vpc.status.atProvider.id}
  - step: auto-ready
    functionRef:
      name: function-auto-ready

This is a two-step pipeline. Step one — function-kro — handles resource composition using kro's YAML+CEL model. Each resource is defined as a template with ${...} CEL expressions that wire dependencies between them. ${schema.spec.region} pulls from the Composite Resource (XR) spec, while ${vpc.status.atProvider.id} creates a dependency on the VPC's output — the subnets and security group won't be created until the VPC is ready and its ID is available. The status block aggregates data from composed resources back to the XR, so consumers of the NetworkingStack API can read the IDs of each resource directly from their XR’s status.

Step two — function-auto-ready — handles automatic readiness detection. But the pipeline could have ten steps. Need to enforce policy on all resources? Add a step. Need to inject cost-allocation tags? Add a step. Need to pull secrets from an external API? Add a step. Each step is independent — you can add the capabilities you need to support your simple resource definitions.

Again, the resources block in that Composition is identical to what you'd write in a standalone kro ResourceGraphDefinition (RGD). If you already have kro resource definitions, they drop into the pipeline input without changes. And when you need more — multi-cloud implementations, safe rollouts, operational controls — you add pipeline steps without rewriting what you already have. Same composition language, with a growth path when you need more.

SimpleSchema for Crossplane

kro bundles schema definition into the RGD using SimpleSchema — a compact shorthand (e.g., region: string | default=us-west-2) that avoids writing full OpenAPIv3. This is a straightforward experience to define a schema, so we’re making it available in Crossplane’s developer tooling as well.

Given a SimpleSchema document that describes the our NetworkingStack API shape:

apiVersion: example.crossplane.io/v1
kind: NetworkingStack
spec:
  region: string
status:
  vpcID: string
  subnetID: string
  securityGroupID: string

You’d be able to generate the full Crossplane XRD with OpenAPIv3 schema automatically:

crossplane xrd generate schema.yaml --from simpleschema

In other words, SimpleSchema becomes just another schema authoring option in Crossplane, similar to how function-kro is just another composition logic option. Write your schema and logic the way you prefer, and Crossplane handles the rest.

Running Inside Crossplane's Architecture

Now that we've seen the syntax, let's look at how these resource definitions fit into the broader Crossplane platform. Because function-kro is a function pipeline step, your YAML+CEL definitions automatically participate in the capabilities that Crossplane provides to all compositions. Crossplane does add some structure beyond a standalone RGD — it separates your API schema from its implementation — but that structure is what makes the following possible.

Safe rollouts. CompositionRevisions let you pin existing instances to a known-good revision, roll forward team-by-team, and roll back without a rewrite. When you update your composition logic, it doesn't immediately affect every running instance. You control the blast radius.

Pipeline composability. Add functions from a vast ecosystem of pipeline steps alongside your YAML+CEL definitions. Need to call an external API, query a cloud provider, connect to a secret store, or enforce policy? Add a pipeline step — your resource definitions stay untouched. For cross-cutting concerns that push beyond what CEL can express, general-purpose languages like Go and Python are available as pipeline steps in the same Composition.

Multi-implementation APIs. Define one API backed by multiple Compositions. The same NetworkingStack API can be backed by AWS, GCP, and Azure. One schema, many implementations. Platform teams define the API once and provide region or cloud-specific implementations behind it.

Operational controls. Pausing reconciliation, management policies, and resource lifecycle control. When something goes wrong at 2 AM, you can stop reconciling a specific instance with a single annotation without touching the Composition or affecting other instances.

Developer experience. Shift-left your testing and validation with crossplane render, unit tests, and integration tests. Run a Composition locally against a sample XR before deploying and see the same outputs that would surface at runtime. Diff your logic against a live environment with crossplane-diff. Develop your logic in a modular fashion with Crossplane’s DevEx tooling.

Extensibility Offers Choice

Crossplane's composition function architecture was designed around a simple idea: the platform team should pick the authoring model that fits their needs and skills. function-kro adds YAML+CEL to that already expansive menu.

Today, Crossplane supports KCL for streamlined expressiveness, Go and Python for teams that want full programming languages, YAML templating for those familiar with Helm, HCL for those coming from Terraform, and many others. Now YAML+CEL can be added to that list for teams that like declarative resource definitions with simple expressions to wire them all up.

You don't have to choose one approach for everything - different Compositions can use different functions, and a single pipeline can mix multiple languages across steps.

Get Involved

We're continuing to track upstream kro releases and will keep function-kro current as the kro project evolves. We're also expanding our library of working examples — YAML+CEL has been added to the getting started with composition guide in the Crossplane docs and the function repo includes multiple examples for basic composition, conditionals, readiness checks, collections, and external references to help you get started.

We welcome the kro community to start using and contributing to this new function! We love to hear from all users, as they are exactly what makes this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to reach out via any of the following:

• GitHub — function-kro
• Crossplane website
• Crossplane docs
• Slack
• LinkedIn
• YouTube
• Community meetings

KubeCon + CloudNativeCon Europe is back in Amsterdam, and so are we. This year's edition is shaping up to be one of the most exciting yet for the Crossplane community, featuring end-user stories from major financial institutions and global tech companies, from API-driven infrastructure to self-healing platforms.

Whether you're just getting started with Crossplane, running it in production, or curious about where the project is headed, there's a session (or five) in this schedule with your name on it.

Find Us at the Project Pavilion

Stop by the Crossplane kiosk in the CNCF Project Pavilion (Kiosk #P-16) during Solutions Showcase hours:

• Tuesday, March 24: 10:45–19:00 CET
• Wednesday, March 25: 10:30–17:00 CET
• Thursday, March 26: 10:30–14:00 CET

Come ask questions, see demos of the latest work from the Crossplane community, and connect directly with Crossplane maintainers and community members. If you've been curious about where the project is headed, this is the place to find out.

Crossplane Sessions at KubeCon EU 2026

Start Here: The Crossplane Project Intro and Deep Dive

Crossplane: The Cloud Native Framework for Platform Engineering
Tuesday, March 24 | 15:15 – 15:45 CET | Amtrium 1+2
Jared Watts & Adam Wolfe Gordon, Upbound

New to Crossplane or want to catch up on the latest? This is the session to start with. Jared and Adam, two of Crossplane's core maintainers, will introduce the project, walk through key features, and share what's coming next. Expect demos and walkthroughs of current functionality, real-world use cases, and an interactive opportunity to engage directly with the maintainers. Whether you're at KubeCon to evaluate Crossplane for the first time or you've been building with it for years, this session covers something for everyone.

End-User Stories: Crossplane in Production

API-Driven Infrastructure as Code: Kubernetes APIs as the Contract Bridge Between Teams
Tuesday, March 24 | 11:15 – 11:45 CET | Hall 7, Room C
Florian Hopfensperger, Allianz Technology & Yury Tsarev, Upbound

How does one of the largest technology companies in insurance manage infrastructure across teams at scale? Florian and Yury will share how Allianz Technology is using Kubernetes APIs as the contract layer between platform and application teams, and what it looks like to treat your infrastructure as an API in a large enterprise. If you're navigating the organizational and technical challenges of API-driven infrastructure as code, or just getting started with it, this talk is for you.

From Alert Fatigue to Self-Healing: Building AI-Enabled Control Planes in Banking
Tuesday, March 24 | 16:15 – 16:45 CET | Hall 8, Room D
Nuno Guedes, Millennium bcp & Yury Tsarev, Upbound

Nuno and Yury will walk through how Millennium bcp is moving from reactive alert management toward proactive, AI-enabled, self-healing infrastructure. This is a real-world look at what it takes to build AI-enabled control planes in a highly regulated industry, and what Crossplane makes possible at the foundation.

From GitOps to AIOps: Evolving RBI's Kubernetes Platform with Crossplane and Sharded Kargo
Wednesday, March 25 | 11:00 – 11:30 CET | Forum
Gabor Horvath & Ewald Überall, Raiffeisen Bank International

Another major European financial institution sharing their Crossplane journey at KubeCon. Gabor and Ewald will share how Raiffeisen Bank International is evolving their Kubernetes platform from GitOps patterns toward AIOps, with Crossplane as the control plane backbone, and what Sharded Kargo brings to that picture.

From Projects to Products: The Sociotechnical Journey Behind Sony's Internal Cloud Platform
Wednesday, March 25 | 11:45 – 12:15 CET | Forum
Eugenia Bergman & Hagen Tonnies, Sony Interactive Entertainment

Platform engineering is as much a people and organizational challenge as a technical one. Eugenia and Hagen from Sony Interactive Entertainment will walk through how they transformed their internal cloud platform from a collection of projects into a real product, covering the cultural, structural, and technical shifts that made it possible. If you're on a platform team navigating the "projects to products" transition, this one is worth blocking out time for at KubeCon.

Invisible Guardrails: Enabling Developer Velocity With a Secure Platform
Wednesday, March 25 | 15:00 – 15:30 CET | Hall 8, Room F
James Elías Sigurðarson & Vignir Hafsteinsson, Asana

The best platform guardrails are the ones developers don't notice, until they realize they've been shipping fast without breaking things. The Asana team will show how they've built security and compliance into their platform in a way that accelerates developer velocity rather than blocking it. A must-see for anyone at KubeCon trying to balance governance with a great developer experience.

AI Meets Platform Engineering

Choose Your Own Adventure: AI Meets Internal Developer Platform Tuesday, March 24 | 15:15 – 15:45 CET | Hall 7, Room A
Whitney Lee, Datadog & Viktor Farcic, Upbound

Whitney and Viktor are back at KubeCon with an interactive format exploring what happens when AI becomes an interface layer for your internal developer platform. Expect real demos and a conversation about how AI agents can interact with and through control planes built on Crossplane.

More Crossplane Across the Week

Before KubeCon: Cloud Native Rejekts (Saturday, March 21)

Achieving Platform Engineering Multi-Tenancy with kcp and Crossplane
Cloud Native Rejekts | Saturday, March 21 | 17:00 CET | Room 2
Simon Bein, Kubermatic & Lovro Sviben, Upbound

The unofficial start to KubeCon week happens at Cloud Native Rejekts. Lovro Sviben, a Crossplane contributor, will be presenting on multi-tenancy in platform engineering using kcp and Crossplane. If you're arriving early for the weekend, this is a great way to kick off the conversations that will continue all week.

Project Lightning Talk (Monday, March 23)

Project Lightning Talk: What Can Crossplane Actually Do? A Real World Field Guide
Monday, March 23 | 12:27 - 12:32 CET | Elicium 2
Jonasz Łasut-Balcerzak, Baloise
Five minutes, one practitioner, real-world use cases. Jonasz will give a field guide to what Crossplane can actually do.

Platform Engineering Day (Monday, March 23)

The GitOps Paradox: Why Your Devs Need an API You Don't Want to Build
Monday, March 23 | 15:55 - 16:20 CET | Hall 8 | Room F
Simon Koudijs, ConfigButler

Simon makes the case for why developer-friendly APIs are the missing piece in most GitOps setups, and what it actually takes to build one that your teams will use.

Backend-First IDP: A Production Roadmap With Argo CD, Crossplane, and OPA
Monday, March 23 | 15:20 – 15:45 CET  | Hall 8 | Room G
Abhinav Sharma, KodeKloud & Aditya Soni, Sailpoint

A practical production roadmap for building an internal developer platform using Argo CD, Crossplane, and OPA together, with real lessons from teams who've done it.

Party with the Crossplane Community

After a full day at KubeCon, come unwind with the Crossplane and Kubernetes community on Wednesday, March 25, at 6:00 PM. Upbound is hosting an evening for practitioners, contributors, and cloud-native builders at The Upside Down Amsterdam, an immersive museum just a 3-minute walk from RAI Amsterdam, featuring 25+ interactive rooms, optical illusions, and an LED light ball pit.

Free for KubeCon attendees, but spots are limited; come early. Register here.

See You in Amsterdam

KubeCon + CloudNativeCon Europe runs March 23 – 26. The Crossplane community will be present before day zero, starting at Cloud Native Rejekts on the weekend, through Platform Engineering Day on Monday, all the way to the main conference and our community party on Wednesday evening.

Whether you're stopping by the Project Pavilion kiosk, catching talks throughout the week, or joining us at The Upside Down on Wednesday night, we can’t wait to meet you.

Come find us, ask hard questions, share what you're building, and help us figure out together what platform engineering looks like in 2026 and beyond.

We are excited to announce that Crossplane v2.2.0 has been released and is now available for installation into your control planes. This release is a regular quarterly release focused on maturing key areas of functionality across the project, as Crossplane continues to become more capable, more reliable, and more performant for your production workloads. In this post, we'll dive into the highlights.

Debugging with the Pipeline Inspector (Alpha)

Composition functions give platform builders the power to express arbitrarily complex resource generation logic, but debugging that logic inside a running control plane has historically been difficult. You can write tests and use crossplane render locally, but viewing the requests are going into each function step and the responses coming out required updating the pipeline or the functions themselves.

Crossplane v2.2 introduces a new alpha feature to address this: the pipeline inspector. When enabled, the inspector intercepts every function request and response in a composition pipeline and forwards them over gRPC to a user-configured socket path. This opens the door to a range of debugging and observability use cases — from a local observer that dumps requests to stdout during development, to a production-grade system that stores pipeline traces for auditing or troubleshooting.

The pipeline inspector is disabled by default. To enable it, pass the --enable-pipeline-inspector flag to Crossplane. The default socket address is /var/run/pipeline-inspector/socket, which can be overridden by passing the --pipeline-inspector-socket flag. The full Helm values required to enable the inspector and run a basic pipeline logging sidecar are:

# Enable the pipeline inspector feature flag
args:
  - --enable-pipeline-inspector
  - --pipeline-inspector-socket=/var/run/pipeline-inspector/socket

# Inject the pipeline inspector sidecar
sidecarsCrossplane:
  - name: pipeline-inspector
    image: xpkg.crossplane.io/crossplane/inspector-sidecar:v0.0.3
    args:
      - --socket-path=/var/run/pipeline-inspector/socket
      - --max-recv-msg-size=8388608  # 8MB
    volumeMounts:
      - name: pipeline-inspector-socket
        mountPath: /var/run/pipeline-inspector
    resources:
      requests:
        cpu: 10m
        memory: 64Mi
      limits:
        cpu: 100m
        memory: 128Mi

# Add the shared volume for Unix socket communication
extraVolumesCrossplane:
  - name: pipeline-inspector-socket
    emptyDir: {}

extraVolumeMountsCrossplane:
  - name: pipeline-inspector-socket
    mountPath: /var/run/pipeline-inspector

With this configured, Crossplane will forward a copy of each RunFunctionRequest and RunFunctionResponse to the inspector sidecar as the pipeline runs, giving you full visibility into the data flowing through your function pipeline. This feature remains Alpha in v2.2 and will continue to be matured in future releases.

Runtime Configuration for Package Dependencies

Previously, if you needed to configure a package's runtime you had to set a runtimeConfigRef directly on each package resource. This worked for explicitly installed packages, but had no effect on packages installed as dependencies. This made it difficult to leverage Configuration packages as a top-level construct.

ImageConfig, introduced in Crossplane v1.18, has grown into a powerful centralized mechanism for configuring package installation. New in v2.2, ImageConfig can configure the DeploymentRuntimeConfig used by packages matching a given image prefix, including packages installed as dependencies. The new spec.runtime.configRef field lets you specify a runtime configuration and it applies to all matching packages regardless of how they are installed.

Consider a scenario where you're running on Azure and need your provider pods to use Azure Workload Identity. Previously, you might have disabled dependency resolution to install the provider manually. Now you can express this intent once, declaratively:

---
apiVersion: pkg.crossplane.io/v1beta1
kind: ImageConfig
metadata:
  name: azure-workload-identity
spec:
  matchImages:
    - prefix: xpkg.crossplane.io/crossplane-contrib/provider-azure-
    - prefix: xpkg.crossplane.io/crossplane-contrib/provider-family-azure
  runtime:
    configRef:
      name: azure-workload-identity

---
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: azure-workload-identity
spec:
  serviceAccountTemplate:
    metadata:
      annotations:
        azure.workload.identity/client-id: "12345678-1234-1234-1234-123456789012"
  deploymentTemplate:
    metadata:
      labels:
        azure.workload.identity/use: "true"
    spec:
      selector: {}
      template:
        metadata:
          labels:
            azure.workload.identity/use: "true"
        spec:
          containers:
            - name: package-runtime
              args: []

With this ImageConfig in place, all Azure family providers — whether installed explicitly or as dependencies — will get the azure-workload-identity runtime configuration applied, and therefore have Workload Identity configured on their service accounts. Note that a matching ImageConfig runtime takes precedence over any runtimeConfigRef specified directly on a package's spec.

To confirm the configuration was applied, check the appliedImageConfigRefs field in the package revision's status:

$ kubectl get providerrevision crossplane-contrib-provider-family-azure-e57d1e7b1ce7 -o yaml
# ...
status:
  appliedImageConfigRefs:
  - name: azure-workload-identity
    reason: ConfigureRuntime
# ...

XRD Validation Beyond the Spec

Crossplane XRDs let you define rich OpenAPI schemas for the resources your platform users create. Until now, x-kubernetes-validations (CEL validation rules) could only be used to validate fields within the spec of a composite resource. This left a real gap for platform builders who wanted to enforce naming conventions, require specific labels, or validate other metadata properties at admission time.

Crossplane v2.2 closes this gap by allowing XRDs to configure x-kubernetes-validations outside of spec. This lets you write CEL rules that validate things like the resource's name format or required label values and provide a more complete API contract for your composite resources.

For example, suppose your platform requires all XR names to follow a specific pattern. You can now enforce this directly in the XRD:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: databases.platform.example.org
spec:
  group: platform.example.org
  names:
    kind: Database
    plural: databases
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          x-kubernetes-validations:
            - rule: "self.metadata.name.startsWith('db-')"
              message: "Database names must start with 'db-'"
          properties:
            spec:
              type: object
              properties:
                region:
                  type: string

Any attempt to create an Database with a name that doesn't start with db- will now be rejected at the Kubernetes API server level, giving your users clear, immediate feedback rather than a confusing reconciliation error later.

Using OpenAPI Schemas in Functions

Composition and operation functions may wish to know the OpenAPI schemas for the resources they're working with, for example to make schema-aware decisions, validate resources, or dynamically generate resources. Previously a function could indirectly get the OpenAPI schema for a CRD by requesting the CRD as a RequiredResource (or ExtraResource in Crossplane 1.x). However, schemas were not available for built-in Kubernetes kinds and some manipulation was necessary to extract the OpenAPI schema from the CRD.

Crossplane v2.2 introduces RequiredSchemas in the RunFunctionResponse, allowing functions to request schemas. Analogous to RequiredResources, a function can populate the new field with the API group/version/kind of any resources whose OpenAPI schema it needs and on the next reconciliation Crossplane will include the requested schemas in the function request.

With this expansion of the function gRPC API, Crossplane v2.2 also introduces capability advertisement. The RunFunctionRequest now includes a Capabilities field that tells the function what the running version of Crossplane supports. For example, a function that can optionally use OpenAPI schemas may check whether Capabilities contains CAPABILITY_REQUIRED_SCHEMAS to determine whether schemas are available.

Improvements to crossplane beta trace

Many platform engineers use the crossplane beta trace command is a to understand the state of resources in a Crossplane control plane. v2.2 ships two significant enhancements to make it even more useful.

Trace all resources of a given kind. Previously, crossplane beta trace could take only a single resource instance. Now you can pass a kind (and optionally a namespace) to trace all resources of that type at once. This is helpful when you want to quickly understand the health of every Database or every Provider in your cluster:

# Trace all Database resources in the platform namespace
crossplane beta trace databases.platform.example.org -n platform

# Trace all Providers
crossplane beta trace providers

Watch mode. The trace command now supports a --watch flag (shorthand -w), which keeps the trace output live and refreshes it as resources change, similar to kubectl get --watch. This is especially useful during debugging sessions when you're waiting for a resource to reconcile and want to see the dependency tree update in real time:

crossplane beta trace databases.platform.example.org db-production --watch

Notable Breaking Changes

Before upgrading, be aware of two breaking changes in v2.2:

Function input CRDs are no longer installed. Input CRDs included in Function packages are no longer installed by the package manager, following the xpkg specification. Additionally, unknown or disallowed resource types within a package are now silently ignored instead of causing package installation to fail.

Package cache on-disk structure has changed. The internal structure of the package cache has been updated. This breaks an undocumented behavior that allowed packages to be "side-loaded" directly into Crossplane, which was sometimes used for testing. If you relied on this behavior, please see #6981 and #7147 for details on the necessary changes.

Get Involved

We love contributions from the community in any form: code contributions, issues, questions, feedback on proposals, and many more. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods:

• Crossplane website
• GitHub
• Slack
• LinkedIn
• YouTube
• Reddit
• Email
• Attend a community meeting

AI-assisted development has changed the way engineers create and commit code. But writing code is no longer the bottleneck. The bottleneck is everything that happens after git push.

From infrastructure provisioning, policy enforcement, day-two operations, drift, compliance, to cross-team coordination. That still requires multiple steps, and no new tool will fix it. This is an architecture problem. AI needs APIs, not UIs, and most platforms still aren't built that way.

Current Platforms

Talk to almost any organization, and you'll hear that the desired state lives in Git, while the actual state lives in cloud providers. Policies are buried in pipeline configs. Organizational knowledge exists in wikis no one reads and in engineers who eventually leave.

This has worked up to now because humans worked with humans to navigate the context switching and informal coordination required to get the job done. People fill in the gaps, ask the questions, and translate intent across systems.

But in a world where AI agents are embedded into our organizations, this workflow breaks down. The agent hits a wall, not because it lacks capability, but because the platform wasn't built for programmatic access. It was built for humans who can compensate for inconsistency.

Agents require a unified, structured, machine-readable interface. They need explicit governance rules, readable historical patterns, and discoverable dependencies. Without that structure, autonomy stalls.

Platforms Built on Declarative Control

Kubernetes introduced a simple but powerful control pattern that changes this entirely. Every resource follows a consistent schema:

apiVersion: example.crossplane.io/v1
kind: Database
metadata:
  name: user-db
spec:
  engine: postgres
  storage: 100Gi

Desired state lives in spec, actual state is reflected in status, and controllers observe the difference and reconcile continuously. That reconciliation is consistent and automatic; no human is required to coordinate convergence.

Crossplane extends this model beyond containers to all infrastructure and applications: cloud databases, object storage, networking, SaaS systems, clusters, and custom platform APIs. The result isn't just infrastructure-as-code. It's your entire platform, infrastructure, and applications as a single API. That difference matters.

The three core elements that make this work in practice:

• Desired State: the declarative specification of what we think the world should be. (Example: The frontend service should have 3 replicas with 2 GB of memory each.)
• Actual State: the operational reality of what exists in the infrastructure. (Example: The frontend service has 2 healthy replicas, 1 pending.)
• Policy: the rules and governance that constrain operations. (Example: Production changes require approval between 9 AM and 5 PM PST.)

Controllers continuously reconcile desired state with actual state, and policy is enforced at execution rather than left to manual review. Context becomes part of the system, not something external to it.

Why This Model Works for Agents

An AI agent interacting with a Crossplane-managed platform doesn't need to orchestrate workflows across multiple systems. It interacts with a single API surface.

It can discover resource types via the Kubernetes API, inspect status fields for real-time operational state, watch resources for change events, and submit declarative intent. Since reconciliation handles mechanical execution, agents don't need to coordinate step-by-step logic; they just declare intent and let controllers handle convergence.

This separation of concerns is critical. Controllers handle mechanics, while agents focus on higher-level reasoning. Without a control plane, agents become fragile orchestrators. With one, they become declarative participants.

When the entire platform is accessible through a single, consistent API, the agent has everything it needs. No Slack messages and no tribal knowledge required.

Policy at the Point of Execution

In fragmented platforms, governance follows lots of procedures: reviews, tickets, Slack threads. In a Kubernetes-native control plane, governance is architectural.

RBAC controls who can act. Admission controllers validate changes before they're persisted. Policy engines such as OPA and Kyverno enforce constraints at runtime. Crossplane compositions encode organizational patterns directly into APIs. Every change flows through the same enforcement path, no hidden approval steps, no undocumented exception paths.

This removes ambiguity for agents entirely. The system defines what is allowed. Agents operate within clearly defined boundaries, and the platform enforces them automatically.

Crossplane 2.0: Full-Stack Control

With Crossplane 2.0, compositions can include any Kubernetes resource, not just managed infrastructure. That means a single composite API can provision infrastructure, deploy applications, configure networking, set up observability, and define operational workflows,  all in one place.

apiVersion: platform.acme.io/v1
kind: Microservice
metadata:
  namespace: team-api
  name: user-service
spec:
  image: acme/user-service:v1.2.3
  database:
    engine: postgres
    size: medium
  ingress:
    subdomain: users

Behind that abstraction may live RDS instances, security groups, deployments, services, ingress rules, and monitoring resources. To a human developer or an AI agent, it's a single API. That consistency is what enables automation to scale safely.

Day-two operations follow the same pattern. Crossplane's Operation types bring declarative control to scheduled upgrades, backups, maintenance, and event-driven automation:

apiVersion: ops.crossplane.io/v1alpha1
kind: CronOperation
metadata:
  name: weekly-db-maintenance
spec:
  schedule: "0 2 * * 0"
  operationTemplate:
    spec:
      pipeline:
        - step: upgrade
          functionRef:
            name: function-database-upgrade

Operational workflows are now first-class API objects. Agents can inspect them, trigger them, observe their status, and propose modifications. No need for hidden runbooks.

Where to Start

This doesn't require a start-from-scratch migration. Bring core infrastructure under declarative control first. Your existing resources don't need to be replaced; they just need to be unified behind a consistent API.

For teams using AI-assisted development, engineers express intent and iterate quickly as tools accelerate implementation. As deployment decouples from release, with changes shipping behind feature flags and systems reconciling toward the desired state, the platform must be deterministic and self-correcting, not reliant on someone catching drift or running the right command at the right time.

That is what a declarative control plane provides. Crossplane ensures that intent has somewhere safe, structured, and deterministic to land. Without it, AI will always be bolted onto human-centric workflows. With it, agents become first-class participants in infrastructure operations.

And that starts with a consistent API. Get started by checking out the Crossplane Docs, attending a community meeting, or watching to CNCF's Cloud Native Live on Crossplane 2.0 - AI-Driven Control Loops for Platform Engineering.

We are absolutely thrilled today to announce the graduation of Crossplane within the CNCF! This is a huge milestone for the project as it joins the ranks of other mature, production-ready CNCF projects like Kubernetes, Prometheus, and Helm.

Graduation represents the highest level of maturity in the CNCF and signals that Crossplane is stable, reliable, and broadly deployed in many production environments. Originally created and open-sourced by Upbound, it’s been amazing to see how far Crossplane has come since we donated the project to the CNCF at the Sandbox level in 2020!

Built by the Community

This graduation accomplishment is a testament to an incredible community that has rallied around the vision of making cloud native control planes accessible to everyone. The growth of the Crossplane community has been remarkable. We now have over 3,000 people from more than 480 companies that have contributed to the project - that is a ton of people getting involved!

Compared to other projects, Crossplane ranks #13 out of 231 CNCF projects for number of PR authors, placing us in the top 10% of the CNCF. These aren't just faceless numbers—behind every contribution is a real person who decided to help make Crossplane better. Whether you contributed code, documentation, opened issues, or supported another user, you are a key part of this achievement.

Production-Ready and Battle-Tested

Crossplane is built on community engagement and that combined effort has enabled the project to power real production workloads.

We have over 70 public adopters, including organizations like Nike, Nokia, Grafana, NASA Science Cloud, Elastic, Akamai, SAP, and IBM. These organizations, and many others, trust Crossplane to manage their critical services at scale in production.

The project is also operated according to a robust vendor-neutral governance, completed two comprehensive security audits, achieved an OpenSSF Best Practices badge, and follows a mature release process with a well-defined LTS policy for over 100 total releases to date. Crossplane is definitely built as a project you can bet your cloud native software on.

Building for Long-Term Sustainability

Throughout the rigorous due diligence process for our graduation application, our sponsors on the CNCF Technical Oversight Committee provided thoughtful feedback on how we could strengthen the project's long-term sustainability. We took this guidance to heart and implemented several improvements to the project based on their recommendations.

We established a vendor-neutral community registry to host Crossplane packages at xpkg.crossplane.io and migrated critical release infrastructure to CNCF-owned resources, ensuring the project can continue thriving independently. We also rolled out clear policies for Community Extension Projects to help the community navigate the broad ecosystem of Crossplane extensions.

The result of this effort is that Crossplane is now a stronger and more resilient project. We have the infrastructure and governance to sustain the project for years to come. We are truly grateful for the expertise our TOC sponsors brought to the process—their guidance has made Crossplane better for the cloud native ecosystem.

Beyond Graduation

While graduation is a huge milestone, it isn’t the finish line. It's a recognition of how far we've come on this journey, but also a foundation for the future yet to come.

Thank you to every person who has contributed to Crossplane in any way and any team that has adopted the project—we truly appreciate our amazing community. We are also grateful to the CNCF for all the support they have provided the project over the years.

We're excited to continue building the future of cloud native control planes together. Join us in the Crossplane Slack community, star the project on GitHub, and follow us on Bluesky, X, or LinkedIn.

Here's to the next chapter of Crossplane! 🎉

When we started working on Crossplane back in 2018, we had a simple but ambitious goal: bring the elegance of Kubernetes APIs to infrastructure management. Seven years, 3,000+ contributors, and over 100 releases later, we're thrilled to announce Crossplane 2.0—a release that doesn't just represent new features, but rethinks how platform teams approach control planes.

Crossplane 2.0 is the culmination of everything we've learned from watching real-world platform teams scale from managing a handful of cloud resources to orchestrating entire application ecosystems. It's the result of helping folks debug real production issues, iterating on APIs with hundreds of community members, and constantly asking ourselves: "How can we make this simpler?"

What's New in Crossplane 2.0

• Application support - Crossplane 2.0 extends beyond infrastructure to manage applications alongside your cloud resources
• Broader composition capabilities - Compositions can now include any Kubernetes resource, not just Crossplane-defined resources, enabling full-stack abstractions
• Namespaced by default - Composite Resources (XRs) and Managed Resources (MRs) are now namespaced, providing better isolation and aligning with Kubernetes conventions
• Declarative day two operations - The new Operation type supports one-off, scheduled, and event-driven workflows for upgrades, backups, and maintenance
• Managed resource filtering - Install only the managed resources you need instead of getting everything from a provider

The Problem We Set Out to Solve

When we first built Crossplane, we focused on what seemed like the biggest pain point: giving developers self-service access to cloud infrastructure without the operational overhead. The early versions did this well—platform teams could wrap cloud resources like AWS RDS instances in their own APIs, hide the complexity of security groups and subnet configurations, and let developers get infrastructure with simple YAML declarations.

But as we watched teams adopt Crossplane in production, a pattern emerged. Infrastructure was just the beginning. Crossplane v1 let teams move their infrastructure management into Kubernetes with declarative APIs, but applications and infrastructure remained separate concerns. Platform teams could offer self-service APIs for databases and storage, but those APIs couldn't include applications. Teams still had to manage their apps and infrastructure as distinct, disconnected pieces.

The problem was that Crossplane v1's architecture, while powerful, had become overly opinionated. Claims, cluster-scoped resources, and infrastructure-only compositions had also grown complex and created unnecessary friction. We realized we needed to step back and ask: what would Crossplane look like if we designed it today, with everything we've learned?

What We Built

Crossplane 2.0 represents our answer to that question. Here's what's fundamentally different:

Applications Are First-Class Citizens

The biggest change in Crossplane 2.0 is that compositions can now include any Kubernetes resource, not just Crossplane-managed infrastructure. This means you can define a composite resource that provisions a database, configures networking, deploys an application, and sets up monitoring, all in one cohesive abstraction.

For example, a platform team could now offer their development teams a simple "microservice" API that provisions everything needed for a new service:

apiVersion: platform.acme.io/v1
kind: Microservice
metadata:
  namespace: team-api
  name: user-service
spec:
  image: acme/user-service:v1.2.3
  database:
    engine: postgres
    size: medium
  ingress:
    subdomain: users

Behind the scenes, this creates an RDS instance, configures security groups, deploys the application via a Kubernetes Deployment, creates a Service and Ingress, and even sets up monitoring dashboards. The platform team handles all the complexity; the development team gets exactly what they need.

Namespaces by Default

We've moved away from the cluster-scoped-by-default model that often confused new users. In Crossplane 2.0, both composite resources and managed resources are namespaced by default. This aligns with Kubernetes conventions and makes multi-tenancy much more intuitive.

The old claim/XR duality is gone. Instead, you simply create resources in the namespace where they belong:

apiVersion: example.crossplane.io/v1
kind: Database
metadata:
  namespace: frontend-team
  name: user-db
spec:
  engine: postgres
  storage: 100Gi

For the rare cases where you need cluster-scoped resources like shared networking infrastructure, you can still create them, but now it's an explicit choice rather than the default.

Day Two Operations, Declaratively

One of the most exciting additions is the new Operation type. After years of watching teams struggle with operational tasks like backups, upgrades, and maintenance windows, we realized these shouldn't require custom controllers or external cron jobs.

Operations bring the same declarative approach you know from Crossplane to operational workflows:

apiVersion: ops.crossplane.io/v1alpha1
kind: CronOperation
metadata:
  name: weekly-db-maintenance
spec:
  schedule: "0 2 * * 0"  # Sundays at 2 AM
  operationTemplate:
    spec:
      pipeline:
      - step: upgrade
        functionRef:
          name: function-database-upgrade
        input:
          apiVersion: fn.crossplane.io/v1beta1
          kind: UpgradeInput
          maxDowntime: 5m

We've also introduced WatchOperation for event-driven automation. Want to automatically trigger a backup when someone deploys to production? Or run maintenance tasks when specific resource conditions change? It's all declarative now.

Install Only What You Need

Installing an AWS provider used to mean getting every single AWS resource type, often 100+ CRDs hitting your API server at once. We knew this was a problem, but finding the right solution proved more complex than it initially appeared. Every approach we considered had significant trade-offs.

Crossplane 2.0 introduces ManagedResourceDefinitions (MRDs) and activation policies. We think they're an elegant solution to this longstanding challenge:

apiVersion: apiextensions.crossplane.io/v1alpha1
kind: ManagedResourceActivationPolicy
metadata:
  name: aws-essentials
spec:
  activate:
  - instances.rds.m.aws.crossplane.io
  - buckets.s3.m.aws.crossplane.io
  - "*.ec2.m.aws.crossplane.io"

This not only reduces API server load but also makes it clear to your team exactly which cloud resources are available in your platform.

The Journey Here

Building Crossplane 2.0 required some difficult decisions. We deprecated features that seemed important at the time but added unnecessary complexity in practice. We redesigned core APIs based on thousands of hours of community feedback. Most importantly, we focused on making Crossplane feel more like native Kubernetes—because that's what platform teams consistently told us they wanted.

The engineering effort was substantial. We refactored core controllers, redesigned the composition engine to handle arbitrary Kubernetes resources, and built entirely new systems for operations and provider management. But more than the code, this release represents a shift in philosophy: Crossplane isn't just about infrastructure anymore. It's about giving platform teams the tools to build comprehensive developer experiences.

Backward Compatibility and Migration

We know many of you are running Crossplane in production with business-critical workloads. Crossplane 2.0 maintains backward compatibility with the vast majority of v1.x configurations. Your existing compositions, providers, and composite resources will continue to work.

For new projects, we recommend starting with v2 patterns immediately. For existing workloads, you can migrate gradually—there's no rush. The v1 APIs aren't going anywhere, and we'll continue supporting them as the community needs.

What's Next

Crossplane 2.0 is just the beginning. We're already working on enhanced observability features, improved composition debugging tools, and even tighter integration with the broader cloud-native ecosystem. The community roadmap includes exciting work on composition testing frameworks and advanced policy integration.

But perhaps most importantly, we're continuing to listen. The best features in Crossplane 2.0 came directly from community feedback, production war stories, and late-night Slack conversations about what wasn't working. Keep that feedback coming.

Try It Today

Crossplane 2.0 is available now. You can get started with our updated quick start guide, and the full feature documentation is live at docs.crossplane.io.

We're curious to see how you use these new capabilities. After seven years of working on this project, there's nothing more satisfying than watching the community take our tools and create things we never imagined. Whether you're building internal platforms at a Fortune 500 company or experimenting with control planes in your homelab, Crossplane 2.0 enables the kind of full-stack self-service APIs that platform teams have been asking for.

Join us in the Crossplane Slack community, star us on GitHub, and tell us what works and what doesn't. There's more work ahead.

We are excited to announce the first release of Function Python, a powerful new tool that enables you to write Crossplane compositions using Python. This release opens up new possibilities for platform engineers and developers who prefer Python's simplicity and readability when configuring Crossplane.

Function Python was developed by Upbound and has now been contributed to the Crossplane project. This addition strengthens the growing set of tools available for defining compositions with Crossplane.

What Are Crossplane Composition Functions?

In Crossplane, composition functions are custom programs that template Crossplane resources. They allow you to define how composite resources (XRs) should be composed from other Kubernetes resources. When you create a composite resource, Crossplane invokes these functions to determine the resources to create, update, or delete. This approach provides flexibility and enables advanced logic, such as loops and conditionals, in your infrastructure definitions.

Why Use Python for Composition Functions?

Python is renowned for its ease of use and extensive ecosystem. By writing composition functions in Python, you can leverage:

• Readability: Python's clean syntax makes your composition logic easy to understand and maintain.
• Rich Libraries: Access to a vast array of libraries and frameworks to enhance your compositions.
• Rapid Development: Quickly prototype and iterate on your infrastructure definitions.

With Function Python, you can now harness these benefits within the Crossplane ecosystem.

Getting Started with Function Python

To start using Function Python in your Crossplane compositions:

Install the Function: Apply the following manifest to install Function Python:

apiVersion: pkg.crossplane.io/v1
kind: Function
metadata:
  name: function-python
spec:
  package: xpkg.crossplane.io/crossplane-contrib/function-python:v0.1.0

This will deploy the Function Python package into your Crossplane environment.

Define Your Composition: Create a Composition that references the function-python function. Here's an example snippet:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example-composition
spec:
  compositeTypeRef:
    apiVersion: example.org/v1alpha1
    kind: XExample
  mode: Pipeline
  pipeline:
    - step: python
      functionRef:
        name: function-python
      input:
        apiVersion: fn.crossplane.io/v1alpha1
        kind: Script
        source: |
          from crossplane.function import v1alpha1 as fn
          def compose(request: fn.RunFunctionRequest) -> fn.RunFunctionResponse:
            rsp.desired.resources["bucket"].resource.update({
                "apiVersion": "s3.aws.upbound.io/v1beta2",
                "kind": "Bucket",
                "spec": {
                    "forProvider": {
                        "region": req.observed.composite.resource["spec"]["region"]
                    }
                },
            })
            rsp.desired.resources["bucket"].ready = True

Your script has access to function-sdk-python. For example you can import crossplane.function.resource. It also has access to the full Python standard library - use it with care.

The RunFunctionRequest and RunFunctionResponse types provided by the SDK are generated from a Protocol Buffers schema. Their fields behave similarly to built-in Python types like lists and dictionaries, but there are some differences. Read the generated code documentation to familiarize yourself with the the differences.

Create Composite Resources: With the composition in place, you can now create composite resources that utilize your Python-based composition logic.

For a comprehensive guide on using composition functions, refer to the Crossplane documentation.

Join the Community

Function Python is an open-source project under the Crossplane Contrib organization. We welcome contributions, feedback, and suggestions from the community. Visit the GitHub repository to report issues, submit pull requests, or explore the source code.

We look forward to seeing how the community leverages Function Python to build their control planes!

The Crossplane community has always thrived on collaboration, innovation, and openness. Today, we're excited to announce key updates aimed at fostering the growth of community-contributed Crossplane extensions (providers and functions) while reinforcing our commitment to vendor neutrality—just like all CNCF projects.

We recognize the importance of ensuring that all community-driven contributions remain accessible, well-maintained, and free from vendor lock-in. These updates provide clear expectations and a neutral foundation to support the entire Crossplane ecosystem.

What’s Changing?

Clear Expectations for Community-Contributed Extensions

We are introducing operational guidelines for maintaining extensions contributed under the github.com/crossplane-contrib organization. These policies ensure that community extensions are properly maintained, fostering long-term sustainability and reliability.

A Vendor-Neutral, Community-Driven Home for Extensions

To enhance accessibility and neutrality, all community-contributed extensions will now be published, at a minimum, to ghcr.io/crossplane-contrib. Project documentation will reference this location, providing a consistent and independent source that the entire community can trust—regardless of vendor affiliations.

By reinforcing vendor neutrality, we align with the broader CNCF ecosystem’s principles, ensuring that Crossplane remains an open, community-first project. These changes support our shared goal of making Crossplane the best choice for building cloud-native control planes while improving transparency, accessibility, and long-term adoption.

Your contributions and feedback fuel Crossplane’s success. We appreciate your continued support and look forward to building a strong, open, and vendor-neutral ecosystem together!

The Crossplane Ecosystem

Crossplane has a large and thriving ecosystem of providers and functions—collectively referred to as extensions. These extensions can be public or private and maintained by individuals, end-user organizations, or vendors.

A subset of these extensions has been donated to the Crossplane project and resides in the crossplane-contrib GitHub organization. With today's governance updates, these projects will follow the same requirements as the Crossplane core project.

The Goals of crossplane-contrib

From its inception, crossplane-contrib was created to provide a collaborative home for community-driven Crossplane extensions. With today’s changes, we are introducing a clear health criteria for these projects, aligned with the CNCF’s standards. These criteria include:

• Regular builds
• A vendor-neutral home for released OCI images

However, it is not a requirement for all Crossplane extensions to reside in crossplane-contrib. To ensure the ecosystem thrives, there must be opportunities for differentiation through commercial offerings and services around these extensions.

Regular Builds for Community Extensions

A key governance update requires that all extensions donated to github.com/crossplane-contrib provide regular builds, ensuring the community has reliable access to up-to-date extensions. This change will help maintain a curated and well-maintained catalog of community extensions.

Introducing xpkg.crossplane.io

All Crossplane packages—including Crossplane itself and community-contributed extensions—will be hosted on GitHub Container Registry (GHCR) at ghcr.io/crossplane-contrib and proxied via xpkg.crossplane.io.

Previously, much of the community content was served from xpkg.upbound.io. To ensure vendor neutrality, Crossplane and community content will now be published to xpkg.crossplane.io. Additionally, content can be published to other registries such as Upbound and Docker Hub.

API Groups for Extensions

Crossplane providers define managed resources using Kubernetes API conventions. Moving forward:

• All new extension projects in crossplane-contrib must use a crossplane.io API group.
• Existing projects that are later donated to Crossplane will not be required to change their API group, as this would introduce breaking changes for existing compositions.

Updates to Crossplane Documentation and Website

To maintain vendor neutrality, the Crossplane documentation and website will only reference community extensions. Commercial offerings related to Crossplane will be listed separately on a dedicated Enterprise page.

Implementing the New Policies

Over the coming weeks, we will work with community maintainers to bring their Crossplane extensions into compliance with these new policies. Additionally, project documentation will be updated to refer to images from GHCR.

Looking Ahead

We’re excited about the future of Crossplane and the continued growth of community contributions. We look forward to discussing these updates and other new features at this year’s KubeCon in London—be sure to stop by our booth!

The Crossplane community is at the heart of this project’s success. Whether you're a developer, user, or just interested in what we're building, we’d love to hear from you! Connect with us through the following channels:

• Crossplane Website
• GitHub
• Slack

Let’s continue building a strong, open, and vendor-neutral ecosystem together!

Since its launch in 2018, Crossplane has grown into a powerful tool for building Kubernetes-native control planes. Organizations around the world use it to manage cloud infrastructure through declarative APIs, bringing a familiar Kubernetes experience to platform engineering.

With Crossplane v2, we’re introducing a set of thoughtful refinements that make Crossplane more intuitive, flexible, and well-suited for managing applications—all while maintaining compatibility with existing workflows.

What’s Changing in Crossplane v2?

Crossplane v2 simplifies how composite resources (XRs) and managed resources (MRs) are structured:  

• Namespaces by default – Most composite resources (XRs) and all managed resources (MRs) are now namespaced.
• Broader composition capabilities – XRs can now compose any Kubernetes resource, not just MRs.
• Claims are removed – This reduces complexity and aligns Crossplane more closely with Kubernetes conventions.  

These changes make it easier to manage applications and infrastructure in a unified way, without the need for extra cluster-scoped components.

A More Intuitive Model

Many teams use Crossplane to define opinionated self-service APIs for cloud infrastructure, but applications often remain a separate concern. Crossplane v2 makes application composition a first-class use case.  

By moving to a namespaced model, applications and their dependencies can be managed together in a way that feels natural within Kubernetes. The result is a cleaner, more consistent architecture that works equally well for infrastructure and applications.

Backward Compatibility & Migration

While Crossplane v2 introduces new APIs, existing v1 APIs will remain supported. That means:  

• No breaking changes for most users
• Legacy features will continue to work, though they won’t be the default
• Adoption of v2 features is opt-in, allowing teams to migrate at their own pace

The goal is to make Crossplane more intuitive for new users, while ensuring that existing users can transition smoothly when they’re ready.

Get Involved

These changes are designed to make Crossplane more flexible and approachable while staying true to its core strengths. As development progresses, we’ll share more details and provide resources to help teams explore and adopt these improvements.  

We’ve created a draft proposal on GitHub. You can read it here. Please give us your feedback!

We look forward to continuing the conversation with the Crossplane community.

As someone passionate about open-source software, being a mentee in the LFX Mentorship Program, backed by the Linux Foundation, has been one of the most enriching experiences of my career. Through this program, I had the opportunity to work on the Crossplane project, a Kubernetes-based framework that empowers developers to manage and provision infrastructure through Kubernetes' API. This mentorship gave me hands-on experience and invaluable insights into contributing to large-scale, real-world projects.

Application Process

My application process to LFX Mentorships has a very long history to be honest. As far as I know I applied to the program 5 times (first one was to the Crossplane project) and got rejected from 4 of them. Each time I got rejected I learned things about different projects and tried to understand why I was not the selected mentee. These rejections made me learn the open source philosophy and created a mindset that always tries to be “useful” for the team while creating or improving a project.

While I may have been rejected many times, the only number that counts is how I eventually did get accepted. Even before mentorship projects were given to the LFX program by CNCF projects, I started looking for an issue that I thought would be useful, in my skill set, and achievable for a new contributor. This issue was Switch to the Renovate GitHub Action. I started on this issue and got it merged before the mentorship projects were given to the LFX. This greatly improved my chance of being selected to mentorship since I proved that I can be “useful” for the community and the project. I think this shows to the mentors that you are worth their time and they will be very happy to help you learn things and improve the project together.

Contributions to Crossplane

Here are some of the key PRs I worked on during my time in the program:

1. Run Renovate as a Github Action:
   My first PR was about transferring our Renovate tooling from a hosted environment to a self-hosted Github Action so that we would be able to run make generate after Renovate generates PRs that bump dependencies
2. Add unknown field check to beta validate
   This PR focused on adding a frequently requested feature to the validate command. This enables validate to inform developers about the typos and indentation errors at their schemas in addition to already existing validation logics, which have plagued Crossplane users for a very long time.
3. download/cache full graph of dependencies at beta validate
   This PR enabled validate to download all of the deep dependencies of configurations so that we have a closer experience to real world usage and have all resources available while doing validations.
4. Add configuration.meta/crossplane.yaml support for dependencies to beta validate
   This PR enabled validate to have support for meta files so that configuration writers can do validation before sending their configurations to the remote and move the validation experience even further to the left.
5. Use more global cache directory at validate
   This PR is transporting the cache directory of validate from a workspace centered approach to a global cache approach.
6. Download dependencies of providers and functions at beta validate
   This PR is actually very interesting. It started as a PR for adding support for downloading dependencies of providers and functions to a performance improvement PR which improved the speed of validate by 150x. Yes, you read that number correctly. I really enjoyed doing this.

Challenges and Learning

Like any new experience, there were challenges along the way. Familiarizing myself with such a large and complex codebase was hard at first. At times I was feeling like ‘I don't even understand how the things work how am I going to contribute??’ But I was eventually successful with guidance from my precious mentors Ezgi, Jared and others in the Crossplane community (especially Nic, Philippe and Jean), so much on both golang and Crossplane.

One of the biggest lessons I learned was the importance of clear communication. In a very short time I learned how to look at the software not just as a functional product but as a direct source of user experience, shaping how people feel and interact with technology. This realization greatly improved my perspective on software development, as I started to focus more on the user’s journey, ensuring every aspect of the software contributed positively to their overall experience.

The Importance of Mentorship in Open Source

Being a mentee in the LFX Mentorship program has been transformative. It has not only enhanced my technical skills but also introduced me to the collaborative and community-driven nature of open-source projects. I gained first-hand experience in contributing to a project that is widely used by organizations around the world. The mentorship also provided me with a network of passionate contributors from whom I learned a great deal.

Looking Ahead

As I conclude my time in the LFX Mentorship program, I feel prepared to continue contributing to Crossplane and other open-source projects. As a freshly graduated student from college, skills and knowledge I gained during this experience will no doubt influence my future contributions, and I’m excited to remain a part of the open-source community.

I encourage anyone interested in open-source development to apply for the LFX Mentorship program. Whether you are just starting or have prior experience, the mentorship will help you grow as a developer and contribute meaningfully to important projects.

We are excited to announce today that Crossplane v1.16.0 has been released and is now available for installation into your control planes. This latest release of Crossplane focused on maturing a number of key areas of functionality across the project, as Crossplane continues to become more capable, more reliable, and more performant for your production workloads. In this blog post, we will dive into the details for the highlights from all of these major investment areas.

But first, let’s talk about the inspiring progress the community has made in getting involved with the project and keeping Crossplane advancing on an amazing upward trajectory…

Community engagement is 📈🔥

While preparing the release notes for v1.16, one statistic stood out immediately that was quite informative on how folks are getting more directly involved in the project. The v1.16 release has more first time contributors to the project than any release before!

24 different community members decided they wanted to leave their mark on the project for the first time and roll up their sleeves to commit their first Crossplane code change ever in this release. That’s amazing, and is absolutely the way that this project will remain healthy and sustainable far into the future (in addition to our greatly appreciated returning contributors of course!). Thank you everyone that made their first (or second, or 100th) contribution in this release.

Additionally, we saw some community members step up in a big way and start to drive impressive design changes for core functional areas in Crossplane. While these will take some time to get to a design that the entire team feels is the right experience we can support long term, we are truly grateful to see the depth of the investments that @dalton-hill-0 and @TerjeLafton are taking on for the benefit of the entire project. Looking forward to seeing these contributions land in the next release friends!

Finally, we expect to see even more folks continue to get involved as we make deeper investments in contributor enablement in the near future. Contributing to Crossplane should be easy and with the right support in place, we’ll grow an even bigger set of awesome engineers contributing to a healthy project for years to come.

What’s going on with my Managed Resources?

While Crossplane has provided some internal metrics for a long time, these metrics originated from the Kubernetes controller-runtime and were generally applicable to all Kubernetes controllers as opposed to providing insight specific to Crossplane. Starting with v1.16.0, providers in the Crossplane ecosystem are now able to export fine grained data about the operations they are performing on Crossplane managed resources.

These metrics have been incorporated into the upjet framework that most of the community is using to generate new providers for Crossplane, and the providers for the major clouds have also been updated to include these metrics (for example provider-upjet-aws:v1.4.0).

Let’s look at a couple examples of brand new insight that you can now get in your control planes:

• How many resources is this control plane managing? How many of them are ready and synced?
  • crossplane_managed_resource_exists
  • crossplane_managed_resource_ready
  • crossplane_managed_resource_synced
• How long is it taking for each type of resource to be reconciled and to become ready for the first time?
  • crossplane_managed_resource_first_time_to_reconcile_seconds
  • crossplane_managed_resource_first_time_to_readiness_seconds
• How long is it taking for various managed resource types to be deleted?
  • crossplane_managed_resource_deletion_seconds
• How long is it taking to discover that a resource is out of sync and needs to be updated?
  • crossplane_managed_resource_drift_seconds

These new metrics will provide useful new information into the performance and health of your control planes, so make sure to try them out soon.

Clean up of resources in use gets much faster

The Usage API of Crossplane has been very useful for protecting resources from being deleted while they are still in use (depended on) by other resources. However, a frustrating aspect of this experience has been that the eventual clean up of protected resources can take a very long time. People tend to notice this mostly in ephemeral environments that are purposefully torn down and recreated on a regular basis. In some cases, the deletion of the protected resource would take 10 extra minutes, and in extreme cases even more.

The reason for this major slowdown is essentially the exponential back-off Kubernetes uses when retrying operations that fail. While the resource is still in use, the Usage webhook will block attempts to delete it (e.g. by the garbage collector), thus causing the wait time for the next retry to exponentially increase. By the time the resource can finally be cleaned up, it will likely be multiple minutes before Kubernetes will even attempt the deletion again.

This slowdown has been addressed in v1.16 by the addition of a replayDeletion field on the Usage object. When this field is set to true, the controller will replay the deletion attempt of the used object as soon as the Usage object is gone, which bypasses the long back-off waiting period.

The example below captures the dependency on an EKS cluster by a Helm release. Note that replayDeletion: true is set, so the deletion of the EKS cluster will be replayed immediately after the Usage is gone, which drastically speeds up the clean-up process.

apiVersion: apiextensions.crossplane.io/v1alpha1
kind: Usage
metadata:
  name: release-uses-cluster
spec:
  replayDeletion: true
  of:
    apiVersion: eks.upbound.io/v1beta1
    kind: Cluster
    resourceRef:
      name: my-cluster
  by:
    apiVersion: helm.crossplane.io/v1beta1
    kind: Release
    resourceRef:
      name: my-prometheus-chart

One Crossplane user even reported that the clean up of their scenario was now 56 minutes faster while testing this fix! Note that while this is a great improvement to the usability of the Usages API, it remains in Alpha for this v1.16 release and will continue to be matured in future releases.

Providing credentials to Composition Functions

Composition Functions have significantly enhanced the Crossplane platform builder’s experience with capabilities that were not previously possible before their maturity to beta in v1.14. An entire ecosystem of useful functions have bloomed and Crossplane adopters are building complex resource generation pipelines with logic to accomplish basically anything they can dream up.

Functions can even make network calls to external services to discover information relevant to their pipeline logic. However, a missing piece of this end to end flow has been the ability to supply the function with a set of credentials to use when making these remote API calls. The v1.16 release overcomes this limitation and now provides a secure way for a Composition Function to authenticate to external systems.

In the example snippet below, credentials to access an external service are stored in a Kubernetes secret and then passed to the new credentials field of a function pipeline step. The composition machinery will retrieve the credentials from this secret location and then securely pass them over gRPC with mutual TLS to the given composition function so it can then successfully make its external call.

pipeline:
- step: compose
  functionRef:
    name: function-needs-a-secret
  credentials:
  - name: creds-secret
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: ext-svc-creds

Developer Experience gets more 💜

Continuing the trend of the last handful of Crossplane releases, v1.16 also received numerous investments to improve the developer experience and make Crossplane easier to use. The crossplane CLI in particular received the most fixes, mostly in response to feedback from adopters utilizing its functionality to manage their control planes.

This particular functional area had a good amount of first time contributors, so we’re especially grateful for all the passion being shown by the community here. There’s a ton more opportunities to continue contributing in this area in future releases of Crossplane, so we recommend taking a look at the DevEx tracking epic and seeing if anything catches your eye, and then come chat with us more in #SIG-DevEx.

Next stop: v1.17

As we take our first steps towards the next Crossplane release of v1.17, we continue to invite your contributions in many forms: from giving a ⭐ star to the project on GitHub, reporting or commenting on issues, recounting your journey in our adoption stories, to offering feedback on design proposals and pull requests.

We love to hear from the community, as you make this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods:

• Crossplane website
• GitHub
• Slack
• LinkedIn
• Twitter
• YouTube
• Reddit
• Email
• Sign up for the community newsletter