Crossplane v0.8 - Spin up complete cloud environments using kubectl for GCP, AWS, and Azure

Phil Prasek -

Provision complete environments (dev, staging, prod) from kubectl with a few lines of yaml using new minimal environment stacks for GCP, AWS, and Azure. Infrastructure operators can package turn-key environment templates and quickly spin up environments for app teams, so apps can be built and released faster with a streamlined workflow using familiar Kubernetes tooling. PostgreSQL 11 support, EKS support for IAM Roles for Service Accounts, enhanced logging and eventing, and a new Observability Developer Guide round out the release.

There's been a ton of activity on the project this past month with community contributions and collaboration coming in on multiple fronts. With more Crossplane deployments sprouting up we've been focused on enabling real-world scenarios for the community including Postgres 11 support (thanks @vasartori!), EKS support for IAM Roles for Service Accounts, versioning and upgrade, logging and eventing enhancements in crossplane-runtime, a new Observability Developer Guide, and updated providers for GCP, AWS, and Azure!

Minimal environment stacks for GCP, AWS, and Azure

v0.8 adds alpha support for minimial environment stacks (GCP, AWS, and Azure) that enable provisioning complete environments from kubectl with a few lines of yaml.

For example stack-minimal-gcp packages a turn-key environment template, so you can quickly spin up a dev or test environment using kubectl.

Step 1: Install Crossplane and the GCP provider
Install Crossplane and the GCP provider.

Step 2: Install and configure the Minimal GCP stack
Install the Minimal GCP stack and apply the yaml below to spin up a dev/test environment that applications can be deployed into, including their cloud service dependencies.

apiVersion: gcp.resourcepacks.crossplane.io/v1alpha1
kind: MinimalGCP
metadata:
  name: dev-uswest-demo
spec:
  region: us-west2
  projectID: crossplane-playground
  credentialsSecretRef:
    name: gcp-account-creds
    namespace: crossplane-system
    key: credentials

The MinimalGCP CRD is provided by the Minimal GCP stack which uses the templating-controller to observe new MinimalGCP instances and render the following child resources using kustomize templates.

Environment core configuration:

Service catalog (classes of service):

Step 3: Deploy applications with self-service cloud infrastructure
With a dev/test environment configured in step #2, app teams can deploy applications and consume cloud infrastructure on-demand from the service catalog.

For example the Wordpress application can be deployed via Crossplane using the following:

apiVersion: wordpress.samples.stacks.crossplane.io/v1alpha1
kind: WordpressInstance
metadata:
  name: wordpress-demo
spec:
  provisionPolicy: ProvisionNewCluster

The WordpressInstance CRD is provided by the Wordpress application which uses the templating-controller to observe new WordpressInstances and render the following child resources using helm3 templates.

Done!
Our WordpressInstance has successfully deployed into our MinimalGCP environment with only a few lines of yaml!

New versions of stack-minimal-gcp and sample-stack-wordpress have been published to DockerHub, use the new templating-controller(kustomize, helm3, etc.), and provide a great starting point for building custom stacks of your own!

IAM Roles for Service Accounts on EKS

We also enabled stack-aws to authenticate to the AWS API using IAM Roles for Service Accounts in v0.8, based on community feedback that some people were unable to create RDS instances using Crossplane due to security restrictions in their environments -- they needed to use IAM roles associated with Kubernetes ServiceAccounts, rather than providing credentials as a Kubernetes Secret.

Checkout the details here!

Logging and Eventing Upgrade

Observability is crucial to Crossplane users; both those operating Crossplane and those using Crossplane to operate their infrastructure. Crossplane currently approaches observability via Kubernetes events and structured logs with enhancements made to the crossplane-runtime in the v0.8 release. Providers for GCP, AWS, and Azure were also updated to emit logs and events as outlined in the Observability Developer Guide.

In short, a non-admin user and an admin user should both be able to debug any issues only by inspecting logs and events. There should be no need to rebuild the Crossplane binary or to reach out to a Crossplane developer, but we have great community on Slack so please reach out with any questions.

Checkout the Observability Developer Guide for logging and eventing best-practices when developing Crossplane controllers!

Additional noteworthy items in v0.8

  • More v1beta1 APIs:
  • GCP networking resources
  • Host-aware Stack Manager
  • Enables deploying multiple Crossplane instances watching different Kubernetes API servers on a single Host Kubernetes cluster.
  • RBAC group and role refinements
  • Default Crossplane roles (admin, edit, view)
  • Checkout the v0.8 release notes for more info.

Beyond v0.8

  • Versioning and upgrade support for all aspects of Crossplane
  • Stacks Manager support for private repos and robot account credentials
  • Rename GitHub org from crossplaneio to crossplane

Minimal Environment Stacks (alpha)

More v1beta1 APIs:

  • remaining GCP API types (storage buckets)
  • AWS and Azure to follow
  • Enable Workload Identity for stack-gcp when running on GKE
  • Enhanced automated integration tests for GCP, AWS, Azure
  • Additional real-world App Stacks
  • Expanded Rook support for additional in-cluster stateful storage types
  • Policy-based secure connectivity strategies
  • Enhanced application scheduling (cost, weight)

Get involved!

We are excited to see the continual growth of the Crossplane community and would love for you to get involved. Whether you are a developer, user, or just interested in what we are up to, feel free to join us via one of the following methods:

Join the open cloud movement to help level the playing field for everyone!