Crossplane v0.13 paves the way for v1.0 with platform configuration support to create a universal cloud API for your app teams

Phil Prasek -

v0.13 is a major release with new support for platform Configurations and Providers in an overhauled v2 package manager with upgrade and rollback support plus faster deploys. Define your own cloud APIs with CompositeResourceDefinitions (XRDs) with multiple Compositions to offer classes-of-service, and enable your app teams to consume them with Composite Resource Claims (XRCs). More AWS cloud service primitives, a new Helm Provider, and Open Application Model (OAM) enhancements round out this release!

The momentum keeps rolling forward with a huge v0.13 release that establishes near final core APIs and major upgrades that pave the way for a v1.0 release later this year!

Composition naming has been finalized with InfrastructureDefinition and InfrastructurePublication types merged into a new CompositeResourceDefinition (XRD). Requirements are now known as Composite Resource Claims (XRCs), and no longer require any specific kind suffix.

The v2 package manager is a major overhaul that adds support to install and manage new platform Configurations and Providersthat enable you to define and compose a univeral cloud API for your app teams -- without writing code. An overhauled kubectl plugin allows you to build, push, and install Configuration and Providerpackages and a new crossplane.yaml identifies each package and includes dependency info. The v2 package manager supports upgrade and rollback plus everything is much faster!

The's been a ton of community engagement and contributions adding support for more clouds and cloud services -- and we'd like to give a special shout out for some notable provider-aws contributions to @enderv for adding ECR support and @krishchow and @RedHatOfficial! for:

  • S3 Buckets upgrade to v1beta1 quality
  • S3 Bucket Policy support
  • ElasticCache and SNS Topic enhancements

RedHat and @krishchow also published an excellent blog post on how to provision cloud services in OpenShift clusters using Crossplane that expands on the previous work announced back in May 2020!

If there are additional cloud service primitives you'd like to see, please drop us a note in the Crossplane Provider repos on GitHub: AWS, GCP, Azure, or Alibaba!

The community has also made a ton of progress on code generating Crossplane Providers. We'd like to give a shout out to @jaypipes, @Dave_Fellows, @matthchr, @babbageclunk, and @theunrepentantgeek. Follow along with our progress with the links below or drop us a note in the Crossplane #providers channel on Slack to learn more:

We've also been collaborating with @emeshbi from cdk8s on enabling cdk8s users to offer constructs as native CRDs via Crossplane Composition, to run cdk8s behind the Kubernetes API line as part of a longer journey to provide first-class multi-language support for authoring Crossplane platform Configurations and Compositions.

Packet (now Equinix Metal) has a Crossplane Provider for Equinix Metalwith contributions from @displague @rainleander and @jasmingacic plus an awesome Crossplane intro by David McKay, Marques Johansson, and Crossplane's Daniel Mangum, so be sure to check it out!

We'd love to see you at any of our upcoming virtual KubeCon NA talks November 17-20, so register now if you haven't already and checkout the schedule below!

TheNewStack (TNS) coverage has also been great, thanks for having us on the show @alexwilliams, @Joab_Jackson, @LibbyMClark, and @ricmac!

For the KubeCon Pre-event Livestream the folks at Upbound put together an AWS Reference Platform for Kubernetes and Data Services(Apache 2.0) that shows how to use all the new features of Crossplane v0.13 to define your own cloud platform and universal cloud API. The AWS Reference Platform works with both upstream Crossplane and Upbound Cloud -- which is powered by Crossplane.

To get a better idea of using Crossplane v0.13 in action, let's walk through some key aspects of the KubeCon Lightning demo and the AWS Reference Platform for Kubernetes and Data Services.

Note: see APIs in this Configuration for an overview of all APIs provided in this Configuration and full resource definitions.

Define your own cloud APIs with XRDs

CompositeResourceDefinitions aka XRDs let you define your own cloud APIs with a simple API schema that generates two CRDs:

  • composite resource (XR) kind - e.g. CompositePostgreSQLInstance
  • composite resource claim (XRC) kind - e.g. PostgreSQLInstance
apiVersion: apiextensions.crossplane.io/v1alpha1
kind: CompositeResourceDefinition
metadata:
  name: compositepostgresqlinstances.aws.platformref.crossplane.io
spec:
  claimNames:
    kind: PostgreSQLInstance
    plural: postgresqlinstances
  group: aws.platformref.crossplane.io
  names:
    kind: CompositePostgreSQLInstance
    plural: compositepostgresqlinstances
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              parameters:
                type: object
                properties:
                  storageGB:
                    type: integer
                required:
                  - storageGB
            required:
              - parameters

The API schema is the same for the XR and XRC kinds and is passed down from the XRC to the XR and into the Composition where it is mapped onto the composed resources using a patch overlay, in this case onto a DBSubnetGroup and an RDSInstance.

apiVersion: apiextensions.crossplane.io/v1alpha1
kind: Composition
metadata:
  name: compositepostgresqlinstances.aws.platformref.crossplane.io
spec:
  compositeTypeRef:
    apiVersion: aws.platformref.crossplane.io/v1alpha1
    kind: CompositePostgreSQLInstance
  resources:
    - base:
        apiVersion: database.aws.crossplane.io/v1beta1
        kind: DBSubnetGroup
        spec:
          forProvider:
            region: us-west-2
            description: An excellent formation of subnetworks.
          reclaimPolicy: Delete
      patches:
...
    - base:
        apiVersion: database.aws.crossplane.io/v1beta1
        kind: RDSInstance
        spec:
          forProvider:
            region: us-west-2
            dbInstanceClass: db.t2.small
            engine: postgres
            engineVersion: "9.6"
            skipFinalSnapshotBeforeDeletion: true
          reclaimPolicy: Delete
      patches:
        - fromFieldPath: "metadata.uid"
          toFieldPath: "spec.writeConnectionSecretToRef.name"
          transforms:
            - type: string
              string:
                fmt: "%s-postgresql"
        - fromFieldPath: "spec.parameters.storageGB"
          toFieldPath: "spec.forProvider.allocatedStorage"

The XRD defines a custom cloud API for a PostgreSQLInstance that your teams can provision with kubectl that will be satisfied with a Composition that composes a DBSubnetGroup and RDSInstance.

Multiple Compositions can be used to offer classes-of-service for each API, while providing a consistent API across environments.

Include Helm Releases in your Compositions

The AWS Reference Platform for Kubernetes and Data Services also supports a K8s Cluster API and a Composition of EKSCluster, NodeGroup, IAMRole, IAMPolicyAttachment, and cluster Services that compose Helm Releases to provision a fully-configured EKS cluster with the exact configuration you're looking for.

apiVersion: apiextensions.crossplane.io/v1alpha1
kind: Composition
metadata:
  name: services.aws.platformref.crossplane.io
spec:
  compositeTypeRef:
    apiVersion: aws.platformref.crossplane.io/v1alpha1
    kind: Services
  resources:
    - base:
        apiVersion: helm.crossplane.io/v1alpha1
        kind: Release
        spec:
          forProvider:
            namespace: operators
            chart:
              # from https://github.com/prometheus-community/helm-charts
              # default values are overridden by the patches below.
              name: kube-prometheus-stack
              repository: https://prometheus-community.github.io/helm-charts
              version: "10.1.0"
            values: {}
      patches:
        # All Helm releases derive their labels and annotations from the XR.
        - fromFieldPath: metadata.labels
          toFieldPath: metadata.labels
        - fromFieldPath: metadata.annotations
          toFieldPath: metadata.annotations
        # All Helm releases derive the ProviderConfig to use from the XR.
        - fromFieldPath: spec.providerConfigRef.name
          toFieldPath: spec.providerConfigRef.name
        # Derive the Prometheus operator image and tag from the XR.
        - fromFieldPath: spec.operators.prometheus.version
          toFieldPath: spec.forProvider.chart.version

The composed Helm Releases are automatically installed into the target EKS cluster as part of the Services composition in the top-level Cluster composition, showing how Compositions can compose both Managed Resources and other Compositions.

Checkout provider-helm being incubated in https://github.com/crossplane-contrib/provider-helm for more info.

Build and push your platform Configuration

Once you have your platform Configuration defined, you can build and push it to any OCI registry using the new Crossplane CLI kubectl plugin:

kubectl crossplane build configuration --name package.xpkg

kubectl crossplane push configuration acme/platform-aws:v1.0.0 -f package.xpkg

To build a Configuration package, you'll need to define a crossplane.yaml that identifies the package as a Configurationand lists its dependencies:

apiVersion: meta.pkg.crossplane.io/v1alpha1
kind: Configuration
metadata:
  name: platform-ref-aws
  annotations:
    company: Acme
    maintainer: Joe <joe@acme.io>
    keywords: aws, cloud-native, kubernetes, example, platform, reference
    source: github.com/acme/platform-aws
    description: |
      Acme cloud platform k8s applications running in AWS.
spec:
  crossplane:
    version: ">=v0.13.0-0"
  dependsOn:
    - provider: crossplane/provider-aws
      version: v0.12.0

To learn more about building and pushing your platform Configurations checkout the Package Infrastructure section of the https://crossplane.io/docs!

Install and use your platform APIs

The new Crossplane v2 Package Manager supports installing and upgrading Configurations that bundle all the XRDs and Compositions that define the cloud APIs in your platform and the Provider packages they depend on.

1) Install your platform Configuration:

kubectl crossplane install configuration acme/platform-aws:v1.0.0

2) Connect to your cloud provider

Create the ProviderConfig and Secret:

kubectl create secret generic aws-creds --from-file=key=./creds.conf
kubectl apply -f examples/aws-default-provider.yaml

3) Use the cloud APIs in your platform

Now you can consume the cloud APIs in your platform using kubectl, deployment pipelines and GitOps workflows, or anything that works with the Kubernetes API:

kubectl apply -f examples/network.yaml
kubectl apply -f examples/cluster.yaml
kubectl apply -f examples/postgres-claim.yaml

The PostgreSQLInstance claim used above can be customized to have the exact shape and size of the abstraction you want in the cloud APIs for your teams, by customizing the platform ConfigurationXRDs to your liking:

apiVersion: aws.platformref.crossplane.io/v1alpha1
kind: PostgreSQLInstance
metadata:
  name: my-db
spec:
  parameters:
    storageGB: 20
    networkRef:
      id: network-fabric-1
  writeConnectionSecretToRef:
    name: my-db-conn

Crossplane provides kubectl categories to see the resources created:

kubectl get claim
kubectl get composite
kubectl get managed

Checkout the AWS Reference Platform for Kubernetes and Data Services to learn more and customize it to match the exact needs of your organization!

Other Notable v0.13 Items

  • A new RBAC manager automatically manages the RBAC roles and bindings required by providers and composite resources. It is an optional deployment that uses RBAC privilege escalation - Crossplane no longer requires cluster-admin privileges.
  • Crossplane provider’s Provider resource has been replaced by a similar type named ProviderConfig.
  • Workload APIs (e.g. KubernetesApplication) are deprecated and slated for removal in v0.14. Take a look at provider-helm for an alternative way to run workloads from within a Compositionusing a Helm Release resource.
  • Crossplane’s original resource claims (e.g. MySQLInstance) were deprecated in v0.12 and have been removed in v0.13, since you can now define your own claim kinds in a CompositeResourceDefinition (XRD).
  • Helm Provider: install a Helm Release from a Crossplane Composition
  • Open Application Model (OAM)
  • can now be installed using: helm install crossplane with the --set alpha.oam.enabled=true flag
  • Enhanced health scopes with informative health conditions
  • Health check support for containerized.standard.oam.dev in Health
  • Run with fewer privileges
  • Hardening and robustness enhancements towards v1beta1 quality

TBS#23: Sourcegraph

In TBS#23 Dan is joined by Beyang Liu, the CTO and co-founder of Sourcegraph. Sourcegraph is a fast, open-source, fully-featured code search and navigation engine.

Dan and Beyang will explore why universal code search is valuable for developers, how Sourcegraph works, and why Kubernetes is a useful platform for deploying a code search platform. Then they will demonstrate how Crossplane can enhance the experience of deploying Sourcegraph on Kubernetes by providing access to cloud provider managed services from within the cluster.

To get the latest content subscribe to the Crossplane YouTube channeland join the TBS livestreams which are announced by @hasheddan and @crossplane_io on Twitter and in the #general channel of Crossplane Slack!

Beyond v0.13

v0.14.0 - Hardening, robustness, v1beta1 APIs in core (Nov 2020)

  • XRDs can support defining multiple versions of an XR
  • Surface XR claim binding and secret publishing errors
  • ControllerConfig can override default values for a Provider
  • Crossplane version constraints in Provider and Configuration packages
  • AWS Provider:
  • S3 Bucket Policy to v1beta1
  • IAM User Access Key v1alpha1
  • Open Application Model (OAM)
  • HealthScope support for PodSpecWorkload
  • Allow OAM controller to create events
  • CRD discovery mechanism
  • Remove deprecated Workload API types:
  • KubernetesApplication, KubernetesTarget, KubernetesCluster
  • replaced by Composition and provider-helm
  • All core APIs to v1/v1beta1
  • Code Generation of Providers (work-in-progress)
  • AWS ACK Code Generation of the Crossplane provider-aws
  • Azure SO Code Generation of the Crossplane provider-azure
  • Clouds that don't have code gen pipelines - wrap stateless Terraform Providers #262

v1.0.0 Release Candidate (Dec 2020)

  • Leader election for all controllers
  • Prometheus metrics for all binaries
  • Claim update propagation to its underlying composite resource
  • Bi-directional Composition patching for status
  • Revision support for incremental Composition upgrades
  • Use Composition members to fill a connection secret
  • Composition validation webhooks
  • Auto dependency resolution for packages - install providers
  • All core APIs to v1/v1beta1
  • Code Generation of Providers (initial set of generated resources)
  • Native AWS, Azure provider resources.
  • Terraform-based provider resources.

Next / Under Consideration

  • Code Generation of Providers (100% coverage)
  • First-class multi-language support for Compositions and Configurations
  • Managed resources can accept an array of resource references for enhanced cross-resource reference (CRR) / dependencysupport.
  • Per-namespace mapping of IRSA and workload identity for finer grained infra permissions in multi-tenant clusters
  • Additional conversion strategies for XRDs with multiple versions of an XR
  • Conversion webhooks to support installing multiple API versions
  • CustomComposition support for use with cdk8s sidecar, TYY, and others

Checkout the following to learn more:

KubeCon NA 2020 - November 17-20th

We'd love to see you at any of our upcoming virtual KubeCon NA talks, so register now if you haven't already and note all times are in Eastern Time (UTC–05:00)!

Wednesday Nov. 18th
Crossplane Project Office Hours - Join Here (Zoom)
Crossplane Maintainers
03:00 PM EST

Managing Apps and Cloud Resources with a Unified Approach in Kubernetes
Jianbo Sun, Alibaba & Jared Watts, Upbound
4:55 pm EST

Friday Nov. 20th
Building an Enterprise Infrastructure Control Plane on Kubernetes
Daniel Mangum, Upbound & Steven Borrelli, Mastercard
5:05 pm EST

Get involved!

We're excited to see the continual growth of the Crossplane community and would love for you to get involved. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: