Enhancing Developer Experience with Crossplane 1.15: CLI, Composition Functions Python SDK, and More
What's New in Crossplane 1.15
The Crossplane 1.15 release introduces a range of improvements and features to enhance the developer experience for platform engineers. This version emphasizes practical enhancements in command line functionality and usability, making it easier to manage cloud infrastructure.
The main themes of the release are:
- Refining the command line interface (CLI) and introducing functionality that streamline the development of cloud infrastructure within the control plane paradigm.
- Composition functions, a new Python SDK and other improvements.
- Adopting xpkg.upbound.io as the default registry for Crossplane’s package manager.
Outside of the 1.15 release, we are highlighting two big milestones for the project as a whole.
- We’ve applied for CNCF graduation! Crossplane is in the process of going from Incubating to becoming a CNCF Graduated project. While Crossplane has been an Incubating project since 2021 and has had significant growth and production usage, the next and final stage of Graduation is a formalized signal of what sorts of enterprises should adopt the project. If you are a fan of Crossplane, you can show your support by reacting to the proposal (like 👍, ❤️, or 🚀), and adding your company to the adopters list.
- We have expanded the steering committee! Welcome Brian Lindblom from Apple and Bob Haddleton from Nokia. They join Upbound at the highest level of leadership in the project, responsible for overseeing its overall health and success.
Let’s dive in, explore Crossplane 1.15 release in more detail, and see how it empowers platform builders.
Default Registry Change
The default value for the registry that Crossplane's package manager will install packages from changes from index.docker.io to xpkg.upbound.io. Crossplane command xpkg push already defaults to the new registry, and now crossplane will pull by default from the new registry as well.
The Upbound marketplace hosted at xpkg.upbound.io is the only OCI-compliant registry in the ecosystem that understands the internals of Crossplane packages, and it will be able to provide the best possible experience for the Crossplane community. More details can be found about this change in a separate blog post.
Composition Functions Improvements
Composition Functions adoption in the Crossplane community has been increasing, and we are excited to see all the use cases you are building. In this release, we are improving the developer experience and expanding the functionality.
Composition Functions in Python
Python is the second most popular language for creating composition functions in the Crossplane community. With the introduction of the Python SDK, you can use the function-template-python and start developing functions in AI’s favorite language. You can learn more about developing composition functions in Python in the Crossplane docs.
Prometheus metrics for functions
Crossplane emits basic functions metrics for number of requests sent, number of responses received and histogram of function run duration. This is a very useful feature, helping profile functions performance and stability in a production environment.
Functions can use extra observed resources
Composition functions can now request additional cluster-scoped resources that Crossplane has access to. This feature enables scenarios where we want to access resources from within the function that are not part of the XR. For example, given a VPC provisioned by a separate composition, now we could request this resource, extract the VPCId and use it within the function.
Server-Side Apply based Claim syncing
Alpha support for Server-Side Apply (ssa) based syncing between Composite Resources and Claims was introduced in this release and can be enabled by passing the --enable-ssa-claims flag to Crossplane.
This change improves stability and correctness of the syncing process. Please note that this is an alpha feature, make sure to test in non-production environments. Read more about the change in this Crossplane issue.
Please note that this is an alpha feature, try it out in a non-production environment first.
New and Improved CLI commands
The Crossplane 1.15 release introduces 3 new beta command line subcommands as well as enhancements to existing ones. The Crossplane CLI significantly improves the control plane development experience. Below is an overview of these commands with practical examples.
These enhancements to the CLI commands in Crossplane 1.15 not only introduce new capabilities but also make existing workflows more efficient and user-friendly.
💡Interactive Workshop: Hands-on with Crossplane 1.15
Explore Crossplane 1.15's capabilities through our free interactive workshop. This hands-on experience is designed to familiarize you with the new features and improvements. Check out the open source workshop at Crossplane Interactive Workshop.
The validate command
crossplane beta validate: Validates resources against their schemas using Kubernetes API server's validation library that is useful for offline schema validation while you are designing and building your platform.
Crossplane resources are represented as YAML files, often with complex structures. It's easy to make an indentation error or misspell a resource name. The validate command is an invaluable tool in ensuring correctness and making errors visible early in the development cycle. It enables shift-left where composition authors can detect errors in a very early development stage, without the need to connect to a running cluster.
The validate command enables various validation scenarios:
- Validate any Crossplane resource against any schema (provider, XRD).
- Use the output of the
rendercommand to validate function based compositions. - When validating resources against provider schemas, the command will download and cache the providers’ CRDs to make future runs much more performant.
- Work with Kubernetes CEL validation rules to enable validation of complex business rules expressed in the composition resources definitions (XRDs).
- We can pass the entire folder with schema files and the command is going to validate our resources against all of those.
- Finally, since the CLI is a single binary, it’s very easy to set up validation steps in a CI pipeline.
The command requires two parameters; extensions and resources. These parameters can be provided to the command via a file, folder or standard input.
Example: crossplane beta validate schemas.yaml resources.yaml
crossplane beta validate schemas.yaml resources.yaml
[✓] example.crossplane.io/v1beta1, Kind=XR, example validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=AccessKey, sample-access-key-0 validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=AccessKey, sample-access-key-1 validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=User, test-user-0 validated successfully
[✓] iam.aws.upbound.io/v1beta1, Kind=User, test-user-1 validated successfully
Total 5 resources: 0 missing schemas, 5 success cases, 0 failure casesThe xpkg init command
crossplane beta init: Initiates a new project, including running an init.sh script if available in the template repository, simplifying project setup. If the template repository contains a NOTES.txt file, it will be printed out explaining what the init script is going to do. It can also be used by the template authors to provide additional instructions or information to the user.
Crossplane uses the concept of providers and recently added composition functions as building blocks empowering compositions to render virtually any infrastructure.
Both providers and functions build upon other abstractions and it can be challenging to get a new code base started. Thankfully the crossplane beta xpkg init comes to the rescue!
Example: crossplane beta xpkg init function-example function-template-go results in the following folder structure:
tree
.
|-- Dockerfile
|-- LICENSE
|-- README.md
|-- example
| |-- README.md
| |-- composition.yaml
| |-- functions.yaml
| `-- xr.yaml
|-- fn.go
|-- fn_test.go
|-- go.mod
|-- go.sum
|-- input
| |-- generate.go
| `-- v1beta1
| |-- input.go
| `-- zz_generated.deepcopy.go
|-- main.go
|-- package
| |-- crossplane.yaml
| `-- input
| `-- template.fn.crossplane.io_inputs.yaml
`-- renovate.json
5 directories, 18 filesThe addition to the command is the ability to run an init script when a template repository has one. The function-template-go contains an init script and when the command is run, we can see a helpful dialog from the NOTES.txt file in the template repository.
To get started:
1. Replace `function-template-go` with your function in `go.mod`,
`package/crossplane.yaml`, and any Go imports. (You can also do this
automatically by running the `./init.sh <function-name>` script.)
2. Update `input/v1beta1/` to reflect your desired input (and run `go generate`)
3. Add your logic to `RunFunction` in `fn.go`
4. Add tests for your logic in `fn_test.go`
5. Update `README.md`, to be about your function!
Found init.sh script!
Do you want to run it? [y]es/[n]o/[v]iew:
This helps us with the initial changes required to the repository.
The top command
crossplane beta top: Provides quick resource utilization checks of Crossplane pods, similar to kubectl top pods.
The top command acts similarly to the kubectl top pods giving insight into the resources utilization by the Crossplane pod.
By default, the top subcommand will search for Crossplane pods in the crossplane-system namespace, however, it's possible to change it with the -n flag.
It's also possible to print a short summary by using the -s or --summary.
Example: crossplane beta top -s
crossplane beta top --summary
Nr of Crossplane pods: 10
Crossplane: 2
Function: 3
Provider: 5
Memory: 304Mi
CPU(cores): 3m
TYPE NAMESPACE NAME CPU(cores) MEMORY
crossplane crossplane-system crossplane-b96ddf55d-p465z 1m 30Mi
crossplane crossplane-system crossplane-rbac-manager-86f6976998-rdrwg 1m 15Mi
function crossplane-system function-auto-ready-ad9454a37aa7-7d8c7f84f4-hbqhb 0m 5Mi
function crossplane-system function-go-templating-eff9a0400879-65fb9cbf98-cftxf 0m 5Mi
function crossplane-system function-patch-and-transform-6072b6a09072-79f9b9596f-42pcn 0m 5Mi
provider crossplane-system provider-aws-account-1b29d2bc79dc-5fcfd68cdf-c274z 1m 107Mi
provider crossplane-system provider-azure-certificateregistration-7c75572c5480-68c98chb4kr 1m 44Mi
provider crossplane-system provider-nop-ecc25c121431-5b59b5f678-prfx4 1m 7Mi
provider crossplane-system upbound-provider-family-aws-fec919bd2218-6c4bdb9bdd-j9c5w 1m 35Mi
provider crossplane-system upbound-provider-family-azure-dde405d96fb8-98468c5c7-nfqc9 1m 46MiThe trace command
crossplane beta trace: The trace command now includes the ability to trace Crossplane packages, offering deeper insight into installation processes and dependencies. We’ve often heard feedback from the community that when a package is unhealthy, it’s difficult and unclear how to find the root cause. This new trace functionality for packages aims to make that much more easy.
The trace subcommand was introduced with the Crossplane v1.14 release, however, in this release, it was expanded by adding the capability to trace all types of Crossplane packages. This is very helpful as Crossplane configurations can bring a lot of additional resources such as providers and functions.
Example: crossplane beta trace configuration/configuration-caas
crossplane beta trace configuration/configuration-observability-oss
NAME VERSION INSTALLED HEALTHY STATE STATUS
Configuration/configuration-observability-oss v0.2.0 True Unknown - ActivePackageRevision
├─ ConfigurationRevision/configuration-observability-oss-a51529457ad7 v0.2.0 - Unknown Active UnknownPackageRevisionHealth: ...ndencies: [xpkg.upbound.io/upbound/function-patch-and-transform]
├─ Provider/crossplane-contrib-provider-helm v0.16.0 True True - HealthyPackageRevision
│ └─ ProviderRevision/crossplane-contrib-provider-helm-b4cc4c2c8db3 v0.16.0 - True Active HealthyPackageRevision
├─ Provider/crossplane-contrib-provider-kubernetes v0.10.0 True True - HealthyPackageRevision
│ └─ ProviderRevision/crossplane-contrib-provider-kubernetes-63506a3443e0 v0.10.0 - True Active HealthyPackageRevision
└─ Function/upbound-function-patch-and-transform v0.2.1 True Unknown - ActivePackageRevision
└─ FunctionRevision/upbound-function-patch-and-transform-a2f88f8d8715 v0.2.1 - - ActiveIn this example, we can see providers being provisioned as well as transient errors and other helpful messages. The reconciliation process is reflected in real-time in the output of the command. The health of the configuration is reported as Unknown until the remaining dependencies are reconciled.
Running the command with a watch will provide constant updates on how the installation progresses and will allow for detecting errors early on.
The convert command
crossplane beta convert pipeline-composition and crossplane beta convert deployment-runtime: Convert classic compositions to new function pipeline compositions and ControllerConfig to DeploymentRuntimeConfig.
Crossplane is evolving at a very fast pace, therefore some features supersede others and we need to migrate to newer APIs. Manual migrations are often prone to errors and frustrating. The command can also be used in a CI pipeline, helping with automated conversion scenarios.
With the introduction of the convert command, we can now easily migrate to a new API in a reliable and repeatable way. In the example below, we are migrating from the deprecated ControllerConfig to the new DeploymentRuntimeConfig using a simple one-liner .
Example: crossplane beta convert deployment-runtime controllerconfig.yaml
Getting Started with Crossplane 1.15
Ready to dive into Crossplane 1.15? Here's how to get started:
- Download and install the latest version of Crossplane.
- Follow the documentation for a smooth setup and initial configuration.
- Explore the interactive workshop for hands-on experience with the new features.
But wait there's more
Platform ref configurations are now modular
The Crossplane ecosystem grows faster and faster every day bringing more exciting features. Worth highlighting is the growing number of configuration packages and the recent redesign of platform-ref-* configurations which are now modular and much more flexible.
As an example check out the platform-ref-aws configuration. It now builds on top of other modules enabling you to combine your infrastructure as complex or simple as required.
See you at KubeCon Paris
This release deepens our commitment to enhancing the developer experience even further, simplifying the creation and management of cloud native control planes by adding new Crossplane CLI commands validate, top, and convert, as well as improving the existing trace and init commands. With the release of v1.15 now behind us, our team can hardly pause as we focus on the next iteration, v1.16, anticipated for release in early May ‘24.
Meanwhile, we're excited about the opportunity to connect with many of you at KubeCon EU 2024 in Paris in March. We're eager to hear your invaluable insights on Crossplane. Meet the Crossplane maintainers at the Crossplane kiosk in the CNCF project pavilion, and chat with the experts at Upbound in booth G14 to experience a live demo and take home the coolest new socks at KubeCon. Be sure to check out all the exciting Crossplane talks scheduled for the event and co-located events.
We continue to invite your contributions in various forms: from giving a ⭐ star to the project on GitHub, reporting or commenting on issues, recounting your journey in our adoption stories, to offering feedback on design proposals and pull requests.
We love to hear from the community, as you make this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: