Crossplane v1.16 - a Community Driven Success

Jared Watts -

We are excited to announce today that Crossplane v1.16.0 has been released and is now available for installation into your control planes. This latest release of Crossplane focused on maturing a number of key areas of functionality across the project, as Crossplane continues to become more capable, more reliable, and more performant for your production workloads. In this blog post, we will dive into the details for the highlights from all of these major investment areas.

But first, let’s talk about the inspiring progress the community has made in getting involved with the project and keeping Crossplane advancing on an amazing upward trajectory…

Community engagement is 📈🔥

While preparing the release notes for v1.16, one statistic stood out immediately that was quite informative on how folks are getting more directly involved in the project. The v1.16 release has more first time contributors to the project than any release before!

24 different community members decided they wanted to leave their mark on the project for the first time and roll up their sleeves to commit their first Crossplane code change ever in this release. That’s amazing, and is absolutely the way that this project will remain healthy and sustainable far into the future (in addition to our greatly appreciated returning contributors of course!). Thank you everyone that made their first (or second, or 100th) contribution in this release.

Additionally, we saw some community members step up in a big way and start to drive impressive design changes for core functional areas in Crossplane. While these will take some time to get to a design that the entire team feels is the right experience we can support long term, we are truly grateful to see the depth of the investments that @dalton-hill-0 and @TerjeLafton are taking on for the benefit of the entire project. Looking forward to seeing these contributions land in the next release friends!

Finally, we expect to see even more folks continue to get involved as we make deeper investments in contributor enablement in the near future. Contributing to Crossplane should be easy and with the right support in place, we’ll grow an even bigger set of awesome engineers contributing to a healthy project for years to come.

What’s going on with my Managed Resources?

While Crossplane has provided some internal metrics for a long time, these metrics originated from the Kubernetes controller-runtime and were generally applicable to all Kubernetes controllers as opposed to providing insight specific to Crossplane. Starting with v1.16.0, providers in the Crossplane ecosystem are now able to export fine grained data about the operations they are performing on Crossplane managed resources.

These metrics have been incorporated into the upjet framework that most of the community is using to generate new providers for Crossplane, and the providers for the major clouds have also been updated to include these metrics (for example provider-upjet-aws:v1.4.0).

Let’s look at a couple examples of brand new insight that you can now get in your control planes:

  • How many resources is this control plane managing? How many of them are ready and synced?
    • crossplane_managed_resource_exists
    • crossplane_managed_resource_ready
    • crossplane_managed_resource_synced
  • How long is it taking for each type of resource to be reconciled and to become ready for the first time?
    • crossplane_managed_resource_first_time_to_reconcile_seconds
    • crossplane_managed_resource_first_time_to_readiness_seconds
  • How long is it taking for various managed resource types to be deleted?
    • crossplane_managed_resource_deletion_seconds
  • How long is it taking to discover that a resource is out of sync and needs to be updated?
    • crossplane_managed_resource_drift_seconds

These new metrics will provide useful new information into the performance and health of your control planes, so make sure to try them out soon.

Clean up of resources in use gets much faster

The Usage API of Crossplane has been very useful for protecting resources from being deleted while they are still in use (depended on) by other resources. However, a frustrating aspect of this experience has been that the eventual clean up of protected resources can take a very long time. People tend to notice this mostly in ephemeral environments that are purposefully torn down and recreated on a regular basis. In some cases, the deletion of the protected resource would take 10 extra minutes, and in extreme cases even more.

The reason for this major slowdown is essentially the exponential back-off Kubernetes uses when retrying operations that fail. While the resource is still in use, the Usage webhook will block attempts to delete it (e.g. by the garbage collector), thus causing the wait time for the next retry to exponentially increase. By the time the resource can finally be cleaned up, it will likely be multiple minutes before Kubernetes will even attempt the deletion again.

This slowdown has been addressed in v1.16 by the addition of a replayDeletion field on the Usage object. When this field is set to true, the controller will replay the deletion attempt of the used object as soon as the Usage object is gone, which bypasses the long back-off waiting period.

The example below captures the dependency on an EKS cluster by a Helm release. Note that replayDeletion: true is set, so the deletion of the EKS cluster will be replayed immediately after the Usage is gone, which drastically speeds up the clean-up process.

apiVersion: apiextensions.crossplane.io/v1alpha1
kind: Usage
metadata:
  name: release-uses-cluster
spec:
  replayDeletion: true
  of:
    apiVersion: eks.upbound.io/v1beta1
    kind: Cluster
    resourceRef:
      name: my-cluster
  by:
    apiVersion: helm.crossplane.io/v1beta1
    kind: Release
    resourceRef:
      name: my-prometheus-chart

One Crossplane user even reported that the clean up of their scenario was now 56 minutes faster while testing this fix! Note that while this is a great improvement to the usability of the Usages API, it remains in Alpha for this v1.16 release and will continue to be matured in future releases.

Providing credentials to Composition Functions

Composition Functions have significantly enhanced the Crossplane platform builder’s experience with capabilities that were not previously possible before their maturity to beta in v1.14. An entire ecosystem of useful functions have bloomed and Crossplane adopters are building complex resource generation pipelines with logic to accomplish basically anything they can dream up.

Functions can even make network calls to external services to discover information relevant to their pipeline logic. However, a missing piece of this end to end flow has been the ability to supply the function with a set of credentials to use when making these remote API calls. The v1.16 release overcomes this limitation and now provides a secure way for a Composition Function to authenticate to external systems.

In the example snippet below, credentials to access an external service are stored in a Kubernetes secret and then passed to the new credentials field of a function pipeline step. The composition machinery will retrieve the credentials from this secret location and then securely pass them over gRPC with mutual TLS to the given composition function so it can then successfully make its external call.

pipeline:
- step: compose
  functionRef:
    name: function-needs-a-secret
  credentials:
  - name: creds-secret
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: ext-svc-creds

Developer Experience gets more 💜

Continuing the trend of the last handful of Crossplane releases, v1.16 also received numerous investments to improve the developer experience and make Crossplane easier to use. The crossplane CLI in particular received the most fixes, mostly in response to feedback from adopters utilizing its functionality to manage their control planes.

This particular functional area had a good amount of first time contributors, so we’re especially grateful for all the passion being shown by the community here. There’s a ton more opportunities to continue contributing in this area in future releases of Crossplane, so we recommend taking a look at the DevEx tracking epic and seeing if anything catches your eye, and then come chat with us more in #SIG-DevEx.

Next stop: v1.17

As we take our first steps towards the next Crossplane release of v1.17, we continue to invite your contributions in many forms: from giving a ⭐ star to the project on GitHub, reporting or commenting on issues, recounting your journey in our adoption stories, to offering feedback on design proposals and pull requests.

We love to hear from the community, as you make this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: