Enhancing Developer Experience with Crossplane 1.15: CLI, Composition Functions Python SDK, and More

What's New in Crossplane 1.15

The Crossplane 1.15 release introduces a range of improvements and features to enhance the developer experience for platform engineers. This version emphasizes practical enhancements in command line functionality and usability, making it easier to manage cloud infrastructure.

The main themes of the release are:

  • Refining the command line interface (CLI) and introducing functionality that streamline the development of cloud infrastructure within the control plane paradigm.
  • Composition functions, a new Python SDK and other improvements.
  • Adopting xpkg.upbound.io as the default registry for Crossplane’s package manager.

Outside of the 1.15 release, we are highlighting two big milestones for the project as a whole.

  • We’ve applied for CNCF graduation! Crossplane is in the process of going from Incubating to becoming a CNCF Graduated project. While Crossplane has been an Incubating project since 2021 and has had significant growth and production usage, the next and final stage of Graduation is a formalized signal of what sorts of enterprises should adopt the project. If you are a fan of Crossplane, you can show your support by reacting to the proposal (like 👍, ❤️, or 🚀), and adding your company to the adopters list.
  • We have expanded the steering committee! Welcome Brian Lindblom from Apple and Bob Haddleton from Nokia. They join Upbound at the highest level of leadership in the project, responsible for overseeing its overall health and success.

Let’s dive in, explore Crossplane 1.15 release in more detail, and see how it empowers platform builders.

Default Registry Change

The default value for the registry that Crossplane's package manager will install packages from changes from index.docker.io to xpkg.upbound.io. Crossplane command xpkg push already defaults to the new registry, and now crossplane will pull by default from the new registry as well.

The Upbound marketplace hosted at xpkg.upbound.io is the only OCI-compliant registry in the ecosystem that understands the internals of Crossplane packages, and it will be able to provide the best possible experience for the Crossplane community. More details can be found about this change in a separate blog post.

Composition Functions Improvements

Composition Functions adoption in the Crossplane community has been increasing, and we are excited to see all the use cases you are building. In this release, we are improving the developer experience and expanding the functionality.

Composition Functions in Python

Python is the second most popular language for creating composition functions in the Crossplane community. With the introduction of the Python SDK, you can use the function-template-python and start developing functions in AI’s favorite language. You can learn more about developing composition functions in Python in the Crossplane docs.

Prometheus metrics for functions

Crossplane emits basic functions metrics for number of requests sent, number of responses received and histogram of function run duration. This is a very useful feature, helping profile functions performance and stability in a production environment.

Functions can use extra observed resources

Composition functions can now request additional cluster-scoped resources that Crossplane has access to. This feature enables scenarios where we want to access resources from within the function that are not part of the XR. For example, given a VPC provisioned by a separate composition, now we could request this resource, extract the VPCId and use it within the function.

Server-Side Apply based Claim syncing

Alpha support for Server-Side Apply (ssa) based syncing between Composite Resources and Claims was introduced in this release and can be enabled by passing the --enable-ssa-claims flag to Crossplane.

This change improves stability and correctness of the syncing process. Please note that this is an alpha feature, make sure to test in non-production environments. Read more about the change in this Crossplane issue.

Please note that this is an alpha feature, try it out in a non-production environment first.

New and Improved CLI commands

The Crossplane 1.15 release introduces 3 new beta command line subcommands as well as enhancements to existing ones. The Crossplane CLI significantly improves the control plane development experience. Below is an overview of these commands with practical examples.

These enhancements to the CLI commands in Crossplane 1.15 not only introduce new capabilities but also make existing workflows more efficient and user-friendly.

💡Interactive Workshop: Hands-on with Crossplane 1.15
Explore Crossplane 1.15's capabilities through our free interactive workshop. This hands-on experience is designed to familiarize you with the new features and improvements. Check out the open source workshop at Crossplane Interactive Workshop.

The validate command

crossplane beta validate: Validates resources against their schemas using Kubernetes API server's validation library that is useful for offline schema validation while you are designing and building your platform.

Crossplane resources are represented as YAML files, often with complex structures. It's easy to make an indentation error or misspell a resource name. The validate command is an invaluable tool in ensuring correctness and making errors visible early in the development cycle. It enables shift-left where composition authors can detect errors in a very early development stage, without the need to connect to a running cluster.

The validate command enables various validation scenarios:

  • Validate any Crossplane resource against any schema (provider, XRD).
  • Use the output of the render command to validate function based compositions.
  • When validating resources against provider schemas, the command will download and cache the providers’ CRDs to make future runs much more performant.
  • Work with Kubernetes CEL validation rules to enable validation of complex business rules expressed in the composition resources definitions (XRDs).
  • We can pass the entire folder with schema files and the command is going to validate our resources against all of those.
  • Finally, since the CLI is a single binary, it’s very easy to set up validation steps in a CI pipeline.

The command requires two parameters; extensions and resources. These parameters can be provided to the command via a file, folder or standard input.

Example: crossplane beta validate schemas.yaml resources.yaml

crossplane beta validate schemas.yaml resources.yaml
[✓] example.crossplane.io/v1beta1, Kind=XR, example validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=AccessKey, sample-access-key-0 validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=AccessKey, sample-access-key-1 validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=User, test-user-0 validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=User, test-user-1 validated successfully
Total 5 resources: 0 missing schemas, 5 success cases, 0 failure cases

The xpkg init command

crossplane beta init: Initiates a new project, including running an init.sh script if available in the template repository, simplifying project setup. If the template repository contains a NOTES.txt file, it will be printed out explaining what the init script is going to do. It can also be used by the template authors to provide additional instructions or information to the user.

Crossplane uses the concept of providers and recently added composition functions as building blocks empowering compositions to render virtually any infrastructure.

Both providers and functions build upon other abstractions and it can be challenging to get a new code base started. Thankfully the crossplane beta xpkg init comes to the rescue!

Example: crossplane beta xpkg init function-example function-template-go results in the following folder structure:

tree
.
|-- Dockerfile
|-- LICENSE
|-- README.md
|-- example
|   |-- README.md
|   |-- composition.yaml
|   |-- functions.yaml
|   `-- xr.yaml
|-- fn.go
|-- fn_test.go
|-- go.mod
|-- go.sum
|-- input
|   |-- generate.go
|   `-- v1beta1
|       |-- input.go
|       `-- zz_generated.deepcopy.go
|-- main.go
|-- package
|   |-- crossplane.yaml
|   `-- input
|       `-- template.fn.crossplane.io_inputs.yaml
`-- renovate.json

5 directories, 18 files

The addition to the command is the ability to run an init script when a template repository has one. The function-template-go contains an init script and when the command is run, we can see a helpful dialog from the NOTES.txt file in the template repository.

To get started:

1. Replace `function-template-go` with your function in `go.mod`,
   `package/crossplane.yaml`, and any Go imports. (You can also do this
   automatically by running the `./init.sh <function-name>` script.)
2. Update `input/v1beta1/` to reflect your desired input (and run `go generate`)
3. Add your logic to `RunFunction` in `fn.go`
4. Add tests for your logic in `fn_test.go`
5. Update `README.md`, to be about your function!

Found init.sh script!
Do you want to run it? [y]es/[n]o/[v]iew:

This helps us with the initial changes required to the repository.

The top command

crossplane beta top: Provides quick resource utilization checks of Crossplane pods, similar to kubectl top pods.

The top command acts similarly to the kubectl top pods giving insight into the resources utilization by the Crossplane pod.

By default, the top subcommand will search for Crossplane pods in the crossplane-system namespace, however, it's possible to change it with the -n flag.

It's also possible to print a short summary by using the -s or --summary.

Example: crossplane beta top -s

crossplane beta top --summary
Nr of Crossplane pods: 10
Crossplane: 2
Function: 3
Provider: 5
Memory: 304Mi
CPU(cores): 3m

TYPE         NAMESPACE           NAME                                                              CPU(cores)   MEMORY
crossplane   crossplane-system   crossplane-b96ddf55d-p465z                                        1m           30Mi
crossplane   crossplane-system   crossplane-rbac-manager-86f6976998-rdrwg                          1m           15Mi
function     crossplane-system   function-auto-ready-ad9454a37aa7-7d8c7f84f4-hbqhb                 0m           5Mi
function     crossplane-system   function-go-templating-eff9a0400879-65fb9cbf98-cftxf              0m           5Mi
function     crossplane-system   function-patch-and-transform-6072b6a09072-79f9b9596f-42pcn        0m           5Mi
provider     crossplane-system   provider-aws-account-1b29d2bc79dc-5fcfd68cdf-c274z                1m           107Mi
provider     crossplane-system   provider-azure-certificateregistration-7c75572c5480-68c98chb4kr   1m           44Mi
provider     crossplane-system   provider-nop-ecc25c121431-5b59b5f678-prfx4                        1m           7Mi
provider     crossplane-system   upbound-provider-family-aws-fec919bd2218-6c4bdb9bdd-j9c5w         1m           35Mi
provider     crossplane-system   upbound-provider-family-azure-dde405d96fb8-98468c5c7-nfqc9        1m           46Mi

The trace command

crossplane beta trace: The trace command now includes the ability to trace Crossplane packages, offering deeper insight into installation processes and dependencies. We’ve often heard feedback from the community that when a package is unhealthy, it’s difficult and unclear how to find the root cause. This new trace functionality for packages aims to make that much more easy.

The trace subcommand was introduced with the Crossplane v1.14 release, however, in this release, it was expanded by adding the capability to trace all types of Crossplane packages. This is very helpful as Crossplane configurations can bring a lot of additional resources such as providers and functions.

Example: crossplane beta trace configuration/configuration-caas

crossplane beta trace configuration/configuration-observability-oss
NAME                                                                     	VERSION   INSTALLED   HEALTHY   STATE	STATUS
Configuration/configuration-observability-oss                            	v0.2.0	True    	Unknown   -    	ActivePackageRevision
├─ ConfigurationRevision/configuration-observability-oss-a51529457ad7    	v0.2.0	-       	Unknown   Active   UnknownPackageRevisionHealth: ...ndencies: [xpkg.upbound.io/upbound/function-patch-and-transform]
├─ Provider/crossplane-contrib-provider-helm                             	v0.16.0   True    	True  	-    	HealthyPackageRevision
│  └─ ProviderRevision/crossplane-contrib-provider-helm-b4cc4c2c8db3     	v0.16.0   -       	True  	Active   HealthyPackageRevision
├─ Provider/crossplane-contrib-provider-kubernetes                       	v0.10.0   True    	True  	-    	HealthyPackageRevision
│  └─ ProviderRevision/crossplane-contrib-provider-kubernetes-63506a3443e0   v0.10.0   -       	True  	Active   HealthyPackageRevision
└─ Function/upbound-function-patch-and-transform                         	v0.2.1	True    	Unknown   -    	ActivePackageRevision
   └─ FunctionRevision/upbound-function-patch-and-transform-a2f88f8d8715 	v0.2.1	-       	-     	Active

In this example, we can see providers being provisioned as well as transient errors and other helpful messages. The reconciliation process is reflected in real-time in the output of the command. The health of the configuration is reported as Unknown until the remaining dependencies are reconciled.

Running the command with a watch will provide constant updates on how the installation progresses and will allow for detecting errors early on.

The convert command

crossplane beta convert pipeline-composition and crossplane beta convert deployment-runtime: Convert classic compositions to new function pipeline compositions and ControllerConfig to DeploymentRuntimeConfig.

Crossplane is evolving at a very fast pace, therefore some features supersede others and we need to migrate to newer APIs. Manual migrations are often prone to errors and frustrating. The command can also be used in a CI pipeline, helping with automated conversion scenarios.

With the introduction of the convert command, we can now easily migrate to a new API in a reliable and repeatable way. In the example below, we are migrating from the deprecated ControllerConfig to the new DeploymentRuntimeConfig using a simple one-liner .

Example: crossplane beta convert deployment-runtime controllerconfig.yaml

Getting Started with Crossplane 1.15

Ready to dive into Crossplane 1.15? Here's how to get started:

But wait there's more

Platform ref configurations are now modular

The Crossplane ecosystem grows faster and faster every day bringing more exciting features. Worth highlighting is the growing number of configuration packages and the recent redesign of platform-ref-* configurations which are now modular and much more flexible.

As an example check out the platform-ref-aws configuration. It now builds on top of other modules enabling you to combine your infrastructure as complex or simple as required.

See you at KubeCon Paris

This release deepens our commitment to enhancing the developer experience even further, simplifying the creation and management of cloud native control planes by adding new Crossplane CLI commands validate, top, and convert, as well as improving the existing trace and init commands. With the release of v1.15 now behind us, our team can hardly pause as we focus on the next iteration, v1.16, anticipated for release in early May ‘24.

Meanwhile, we're excited about the opportunity to connect with many of you at KubeCon EU 2024 in Paris in March. We're eager to hear your invaluable insights on Crossplane. Meet the Crossplane maintainers at the Crossplane kiosk in the CNCF project pavilion, and chat with the experts at Upbound in booth G14 to experience a live demo and take home the coolest new socks at KubeCon. Be sure to check out all the exciting Crossplane talks scheduled for the event and co-located events.

We continue to invite your contributions in various forms: from giving a ⭐ star to the project on GitHub, reporting or commenting on issues, recounting your journey in our adoption stories, to offering feedback on design proposals and pull requests.

We love to hear from the community, as you make this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: