Using KCL to Write Crossplane Composition Functions

KCL is a constraint-based record and functional language hosted by the Cloud Native Computing Foundation (CNCF) that enhances the writing of complex configurations, including those for cloud-native scenarios. With its advanced programming language technology and practices, KCL is dedicated to promoting better modularity, scalability, and stability for configurations. It simplifies logic writing and offers ease of automation APIs and integration with homegrown systems.

Crossplane and its Composition model allow developers to create higher-level abstractions that can encapsulate and compose multiple types of cloud resources across different providers and services. Using Composition Functions to render these abstractions can effectively enhance template capabilities for various provider resources while reducing the amount of YAML code needed.

Combining KCL with Composition Functions offers several benefits:

Simplification of Complex Configurations: KCL provides a more concise syntax and structure as a DSL, reducing the complexity of configurations. When combined with Crossplane’s composite resources, you can create more intuitive and easy-to-understand configuration templates with loop and condition features, simplifying the definition and maintenance of resources instead of duplicate YAMLs.
Reusability and Modularity: KCL supports modularity and code reuse through OCI Registry, which means you can create reusable configuration components. Combined with Crossplane, this promotes the modularity of composite resources, increases the reuse of configurations, and reduces errors.
Automation and Policy-Driven: You can use KCL’s powerful features to write policies and constraints that, combined with Crossplane’s declarative resource management, can be automatically enforced, ensuring compliance within the cloud environment.

Additionally, you can refer to the KCL docs to learn about the differences between KCL and other configuration formats or DSLs.

KCL Donates function-kcl to the Crossplane Community

The surface area of possible experiences to build your cloud native platform with Crossplane has been expanding rapidly since the beta release of Composition Functions in Crossplane v1.14. The KCL team took the initiative to build a reusable function that the entire Crossplane ecosystem can now take advantage of to build their own cloud native platforms with the high level experience and capabilities offered by KCL.

Today, we announce this function’s release to the Upbound marketplace at https://marketplace.upbound.io/providers/crossplane-contrib/function-kcl. You can get started in your Crossplane control plane by installing function-kcl with the command below:

crossplane xpkg install function xpkg.upbound.io/crossplane-contrib/function-kcl:v0.2.0

The Crossplane team and community thank the KCL team for this awesome donation and great addition to the growing ecosystem of Functions for Crossplane! The source code can be found at https://github.com/crossplane-contrib/function-kcl, where contributions and feedback are welcome.

Walkthrough: Building a Composition with KCL

Prerequisites

Prepare a Kubernetes cluster
Install Kubectl
Install Crossplane and Crossplane CLI 1.14+
Install Go 1.21+

Quick Start

Let’s write a KCL function abstraction which generates managed resources VPC and InternetGateway from an input resource Network. This essentially enables your developers to have a simplified Network abstraction to create network infrastructure in a self-service manner.

1. Install the Crossplane KCL Function

Installing a function creates a function pod. The function logic is processed as a pipeline step in a composition that may create managed resources when triggered through specified parameters.

Install a function with a Crossplane function object setting the spec.package value to the location of the function package.

kubectl apply -f- << EOF
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
 name: kcl-function
spec:
 package: xpkg.upbound.io/crossplane-contrib/function-kcl:v0.2.0
EOF

2. Install and configure the AWS provider

In order for our KCL based composition to create resources in AWS, we also need to install the AWS provider. The network resources included in our composition are managed by EC2, so we need to specifically install xpkg.upbound.io/upbound/provider-aws-ec2:v1.1.0.

Simply follow these instructions in the Crossplane docs to install and configure the provider, but make sure to install the above provider-aws-ec2 instead of the S3 provider used in the steps in the docs, like so:

kubectl apply -f- << EOF
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-ec2
spec:
  package: xpkg.upbound.io/upbound/provider-aws-ec2:v1.1.0
EOF

Then simply follow the instructions to:

3. Create Crossplane XRD

We define a schema using the crossplane XRD for the input resource Network, it has a field named id which denotes the network id.

kubectl apply -f- << EOF
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
 name: xnetworks.fn-demo.crossplane.io
spec:
 group: fn-demo.crossplane.io
 names:
   kind: XNetwork
   plural: xnetworks
 claimNames:
   kind: Network
   plural: networks
 versions:
   - name: v1alpha1
     served: true
     referenceable: true
     schema:
       openAPIV3Schema:
         type: object
         properties:
           spec:
             type: object
             properties:
               id:
                 type: string
                 description: ID of this Network that other objects will use to refer to it.
             required:
               - id
EOF

4. Apply the Composition Resource

You can apply the composition resource with the inline KCL code into the cluster.

kubectl apply -f- << EOF
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
 name: xlabels.fn-demo.crossplane.io
 labels:
   provider: aws
spec:
 writeConnectionSecretsToNamespace: crossplane-system
 compositeTypeRef:
   apiVersion: fn-demo.crossplane.io/v1alpha1
   kind: XNetwork
 mode: Pipeline
 pipeline:
 - step: normal
   functionRef:
     name: kcl-function
   input:
     apiVersion: krm.kcl.dev/v1alpha1
     kind: KCLRun
     metadata:
       name: basic
     spec:
       # Generate new resources
       target: Resources
       # OCI, Git or inline source
       # source: oci://ghcr.io/kcl-lang/crossplane-xnetwork-kcl-function
       # source: github.com/kcl-lang/modules/crossplane-xnetwork-kcl-function
       source: |
         # Get the XR spec fields
         id = option("params")?.oxr?.spec.id or ""
         # Render XR to crossplane managed resources
         network_id_labels = {"networks.meta.fn.crossplane.io/network-id" = id} if id else {}
         vpc = {
             apiVersion = "ec2.aws.upbound.io/v1beta1"
             kind = "VPC"
             metadata.name = "vpc"
             metadata.labels = network_id_labels
             spec.forProvider = {
                 region = "eu-west-1"
                 cidrBlock = "192.168.0.0/16"
                 enableDnsSupport = True
                 enableDnsHostnames = True
             }
         }
         gateway = {
             apiVersion = "ec2.aws.upbound.io/v1beta1"
             kind = "InternetGateway"
             metadata.name = "gateway"
             metadata.labels = network_id_labels
             spec.forProvider = {
                 region = "eu-west-1"
                 vpcIdSelector.matchControllerRef = True
             }
         }
         items = [vpc, gateway]
EOF

5. Apply the Crossplane Claim

Now we can initiate the creation of our resources by creating a Network claim:

kubectl apply -f- << EOF
apiVersion: fn-demo.crossplane.io/v1alpha1
kind: Network
metadata:
 name: network-test-functions
 namespace: default
spec:
 id: network-test-functions
EOF

6. Verify the Generated Managed Resources

VPC

kubectl get VPC -o yaml | grep network-id
     networks.meta.fn.crossplane.io/network-id: network-test-functions

InternetGateway

kubectl get InternetGateway -o yaml | grep network-id
     networks.meta.fn.crossplane.io/network-id: network-test-functions

It can be seen that we have indeed successfully generated VPC and InternetGateway resources, and their fields meet expectations.

We can also watch these resources progress and become ready in the real world (AWS), using the following crossplane trace CLI commmand:

❯ crossplane beta trace network.fn-demo.crossplane.io/network-test-functions
NAME                                       SYNCED   READY   STATUS
Network/network-test-functions (default)   True     False   Waiting: Claim is waiting for composite resource to become Ready
└─ XNetwork/network-test-functions-mrfnv   True     False   Creating: Unready resources: basic-gateway, basic-vpc
   ├─ InternetGateway/gateway              True     False   Creating
   └─ VPC/vpc                              True     True    Available

7. Debugging KCL Functions Locally

See the function-kcl repository on GitHub for more information and examples on how to use function-kcl and provide inspiration on more use cases for your own platform.

Client Enhancements

It can be seen that the above abstract code often requires a crossplane as a control plane intermediary, and you can still complete the abstraction in a fully client-side manner and directly generate crossplane managed resources to reduce the burden on the cluster.

On the client side, there are two methods to render managed resources. One method is to use the crossplane beta render command, and the other is to render directly using the kcl run command. The usage for the former can be found here. For the latter, the usage is as follows.

kcl run oci://ghcr.io/kcl-lang/crossplane-xnetwork-kcl-function -S items -D params='{"oxr": {"spec": {"id": "network-test-functions"}}}'

The output is

apiVersion: ec2.aws.upbound.io/v1beta1
kind: VPC
metadata:
 name: vpc
 labels:
   networks.meta.fn.crossplane.io/network-id: network-test-functions
spec:
 forProvider:
   region: eu-west-1
   cidrBlock: 192.168.0.0/16
   enableDnsSupport: true
   enableDnsHostnames: true
---
apiVersion: ec2.aws.upbound.io/v1beta1
kind: InternetGateway
metadata:
 name: gateway
 labels:
   networks.meta.fn.crossplane.io/network-id: network-test-functions
spec:
 forProvider:
   region: eu-west-1
   vpcIdSelector:
     matchControllerRef: true

Both methods require a registry to assist in completing the work. The ultimate choice between them may depend on your operational habits and environmental costs. Regardless of the method chosen, we recommend maintaining your KCL code in Git to better implement GitOps and obtain a better IDE experience and reusable modules such as the Crossplane AWS Provider Modules.

Get started building your own Crossplane Functions with KCL

Now that function-kcl has been donated to the Crossplane project, we encourage the entire community to take it for a test spin and try building their cloud native control planes with KCL, the newest high level language experience offered by Crossplane Functions. Contributions and feedback are more than welcome from the community in the https://github.com/crossplane-contrib/function-kcl repository on GitHub. Let us know what you think!

And of course, we love to hear from the community about Crossplane in general, as you make this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: