Announcing Crossplane v1.14 - Power to the Platform Engineer

Jared Watts -

Crossplane v1.14 was released today, and it is packed full of big features that will make Platform Engineers more productive and effective in building control planes to power their infrastructure. This is the biggest release we’ve done to date, with over 700 commits, so the velocity has only continued to increase as the project matures. Additionally, the community has recently reached an exciting milestone by surpassing over 10,000 members on Slack, so everything about the project continues to grow. Crossplane isn’t slowing down one bit, so let’s jump into a tour of all the exciting progress in v1.14!

Composition Functions mature to Beta

Composition Functions were first introduced in v1.11 in an Alpha state, intending to provide an expressive and flexible way of writing code to compose resources in your control plane. Using your language of choice, you can craft custom logic for the exact behavior needed by your organization or environment.

We intentionally put it into the hands of the community early to get valuable feedback that would help us refine a high-quality experience for this critical feature. In today’s release, we have incorporated the many months of feedback into a reimagined architecture and experience, resulting in Functions being promoted to the Beta level of maturity.

At a high level, the new workflow to create a Function with custom logic of your choosing and deploy it to your control plane looks like the following:

  • Initialize everything you’ll need for your new Function project with crossplane beta xpkg init
  • Write your custom function logic in the generated fn.go file (more languages will be supported soon!)
  • Test your logic locally and iterate until it has correct behavior with crossplane beta render
  • Package your new function and push it to a registry so it can be distributed with crossplane xpkg build and crossplane xpkg push
  • Install the function into your control plane with crossplane xpkg install

You may have noticed a new Crossplane CLI and new developer tooling to help streamline your efforts of building and deploying your Function - we’ll share more on those in the next section.

It’s important to remember that you don’t have to write code yourself to take advantage of Functions. As part of the v1.14 release, we’re including a few generic Functions with general applicability for the community. Here’s a composition snippet that makes use of a Function to compose resources using templates similar to Helm, from the crossplane-contrib/function-go-templating repo:

apiVersion: gotemplate.fn.crossplane.io/v1beta1
kind: GoTemplate
source: Inline
inline: |
  {{- range $i := until ( .observed.composite.resource.spec.count | int ) }}
  ---
  apiVersion: iam.aws.upbound.io/v1beta1
  kind: User
  metadata:
    name: test-user-{{ $i }}
    labels:
      testing.upbound.io/example-name: test-user-{{ $i }}
    {{ if eq $.observed.resources nil }}
      dummy: {{ randomChoice "foo" "bar" "baz" }}
    {{ else }}
      dummy: {{ ( index $.observed.resources ( print "test-user-" $i ) ).resource.metadata.labels.dummy }}
    {{ end }}
  {{-end}}

Can you see now why Functions enable a significantly more flexible and expressive experience? An entire ecosystem of reusable Functions will be available in the Upbound Marketplace that will address common scenarios not previously possible with traditional composition based on patch and transform abilities. This flexibility of writing your custom logic in a language of your choice or reusing general Functions from the ecosystem will unlock a wealth of new scenarios for people building control planes with Crossplane!

Read the updated Composition Function documentation for more details on how to build your own functions.

Improving the Developer Experience

We mentioned in a previous post that the charter of the Crossplane project was expanded to include investments in developer experience. This is incredibly important, and we’ve already started delivering on this expanded focus in v1.14 with a series of new experiences and tooling to make Crossplane easier to use.

The crossplane CLI will be getting many new features starting today with v1.14. This tool aims to provide an experience that helps accelerate Platform Engineers to create and manage their control planes. Here are some of the main commands shipping in the release today:

  • init to initialize a new Crossplane project
  • build and push to package and distribute to a registry
  • install to deploy the package into a control plane
  • render to execute a composition’s logic and test its functionality
  • trace to examine live resources and find the root cause of issues quickly

The two most impactful of this new set of commands are render and trace, so let’s examine them in more detail.

render is absolutely a game changer for the composition authoring and testing process. Before this release, there was little support for testing your compositions before deploying them to a live cluster. Now, you can render the work-in-progress compositions on your local laptop during development and visually see their output. This will drastically reduce the iteration cycle time of making a change and verifying its correctness as well as make the whole process safer because it doesn’t require a live control plane. This theme of improving the “authoring time” experience is very important for making Crossplane easier to use, and you can be sure we will continue to make investments here in the next Crossplane milestone.

trace will make the troubleshooting process much easier, for example when there is an unhealthy resource you need to investigate. Previously, we have written blog posts, documentation, and even given talks to educate people on troubleshooting Crossplane resources. To make Platform Engineers more productive, we’re now providing an automated experience to take care of the previously manual troubleshooting steps.

As a visual example, look at the output below, where trace is given an unhealthy claim. It traces through the reference chain to find the root cause database instance resource that cannot be created because its input credentials secret does not exist. This streamlined experience is much easier to use than the previous manual approach!

❯ crossplane beta trace acmedatabases acme-db-prod
NAME                                              SYNCED   READY   STATUS
AcmeDatabase/acme-db-prod (default)               True     False   Waiting: ...claim is waiting for composite resource to become Ready
└─ XAcmeDatabase/acme-db-prod-tx6f7               True     False   Creating
   └─ DatabaseInstance/acme-db-prod-tx6f7-sgnfm   False    -       ReconcileError: ...ls: cannot get credentials secret: Secret "gcp-secret" not found

Read the CLI docs to see all the commands in the new Crossplane CLI.

Ordered Deletion

One feature of this release that will enhance the reliability of Crossplane is the new Usage API.  There are currently some scenarios where Crossplane isn’t always able to clean up all of your resources while they are being deleted, potentially leaving orphaned resources within your cloud provider.

This behavior is due to the eventual consistency of the Kubernetes API. When multiple resources are being created simultaneously, for example from a composition, the dependencies between them will block some resources from being created on the first try. That’s fine though, because the Crossplane controllers will keep retrying until everything is created successfully. In other words, the actual state of the real world becomes “eventually consistent” with the desired state expressed by your composition.

This behavior, however, does not always work well on the deletion path. Occasionally, if the resource that is depended on by another is deleted first, then that dependent resource cannot be deleted successfully by the control plane, leaving an orphaned resource in the real world.

The new Usage type aims to solve this problem by allowing you to declare dependency relationships between your resources as shown in the below example for a Helm chart that uses an EKS cluster:

  apiVersion: apiextensions.crossplane.io/v1alpha1
  kind: Usage
  metadata:
    name: release-uses-cluster
  spec:
    reason: "Release uses Cluster"
    of:
      apiVersion: eks.upbound.io/v1beta1
      kind: Cluster
      resourceRef:
        name: my-cluster
    by:
      apiVersion: helm.crossplane.io/v1beta1
      kind: Release
      resourceRef:
        name: my-prometheus-chart

With these dependency relationships now modeled, a new admission webhook provided by Crossplane will reason over them and block the deletion of any resource that is still depended on by others. Through this deletion ordering enforcement, the new Usage API improves the reliability of life cycle management for both your control planes and the cloud resources they manage.

The Usage docs have more examples and details.

Tons of Other Improvements

We mentioned that we had over 700 commits in the release, so there are lots of other enhancements and improvements beyond the major features we’ve already covered. Here’s a lightning tour of some of the highlights!

  • RuntimeConfig is the long-awaited replacement for ControllerConfig, which enables more flexibility in how you choose to deploy Crossplane Providers into your control plane
  • ManagementPolicies are declared Beta now, further stabilizing the API and giving more adopters the confidence to deploy them to their production environments
  • Composition Validation is declared Beta now as well and is on by default, which will help verify the correctness of your compositions when they are deployed
  • Realtime Compositions is a new Alpha feature that speeds up the composition machinery to propagate changes in your compositions to the real world
  • The longstanding and often misleading “object has been modified” warning errors have been greatly reduced, which will minimize confusion and streamline troubleshooting to find the true root cause more easily
  • The crossplane.io/paused annotation was extended to more of the core types, such as Configurations, Functions, and Providers, giving more control over the reconciliation of resources while performing operational tasks

See you at Kubecon Chicago

Now that v1.14 is out the door, there’s not much time to rest before we turn our sights to the upcoming v1.15 release scheduled for late January ‘24. There will be further major investments in the developer experience and making it easier to build and manage cloud native control planes, as well as more maturity and production readiness in core APIs such as Composition Functions, ManagementPolicies, and EnvironmentConfig.

But before that, we hope to see many of you at KubeCon NA 2023 in Chicago next week and listen to your great feedback on the project. We’ll be hanging out at the Crossplane kiosk (F14) in the CNCF Project Pavilion, running a Contribfest event, and giving multiple talks about Crossplane. Make sure to register for all the Crossplane sessions on the schedule!

As always, we welcome you to contribute through various other opportunities, such as simply ⭐ starring the project on GitHub, opening and commenting on issues, joining and sharing your adoption story, and providing feedback on design docs and pull requests.

We love to hear from the community, as they are exactly what makes this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: