Crossplane is often compared to HashiCorp’s Terraform. It’s common for enterprise platform teams to find Crossplane as they outgrow Terraform and look for alternatives. There are parallels between the two projects:
- Both allow engineers to model their infrastructure as declarative configuration
- Both support managing a myriad of diverse infrastructure using "provider" plugins
- Both are open source tools with strong communities
The key difference is that Crossplane is a control plane, where Terraform is a command-line tool - an interface to control planes. This post touches on a handful of pain points that enterprises commonly face as they scale Terraform, and highlights how Crossplane addresses these issues.
Collaboration
Enterprises often adopt Terraform via their operations team. It’s a great way for a small team of engineers to begin to wrangle their organisation’s infrastructure. Representing infrastructure as declarative configuration allows an operations team to benefit from software engineering best practices - keeping the configuration in revision control where changes can be peer reviewed and reverted when necessary.
Where Terraform can fall apart is when more engineers need to collaborate to manage their organisation’s infrastructure. Terraform relies on a monolithic state file to map desired configuration to actual, running infrastructure. A lock must be held on this state file while configuration is being applied, and applying a Terraform configuration is a blocking process that can take minutes to complete. During this time no other entity - no other engineer - can apply changes to the configuration. Similarly, Terraform uses a monolithic ‘apply’ process - there’s no recommended way to modify only one piece of infrastructure within a configuration. If you use the same configuration to manage your caches and your databases you must always update both - you can’t update only your caches.
Terraform recommends breaking a monolithic configuration up into increasingly more granular configurations. So while the operations team might start with a Terraform configuration that represents ‘production’, they’re encouraged to factor that out into scoped configurations like ‘production billing’ and ‘production auth’. It’s hard to get this right up front, so it can require a lot of refactoring over time, and often results in a complex mesh of Terraform configurations coupled by their inputs and outputs.
Collaboration scales in Crossplane because the Crossplane Resource Model (XRM) promotes loose coupling and eventual consistency. In Crossplane every piece of infrastructure is an API endpoint that supports create, read, update, and delete operations. Crossplane does not need to calculate a graph of dependencies to make a change, so you can easily operate on a single database, even if you manage your entire production environment with Crossplane.
Self Service
Modern organisations are evolving from centralised management of infrastructure to a self service model in which an operations team - often called a platform team - defines opinionated infrastructure abstractions that the development teams they support can consume on demand. Terraform has evolved to support this model through the use of modules. A module is not unlike a software library. Like Crossplane, Terraform resources are high-fidelity representations of an external API resource. A module provides a simplified abstraction atop a broader configuration of these resources - for example the RDS module abstracts eight distinct Terraform resources into a single “RDS instance” concept.
Treating application teams as consumers of a “library” of Terraform configuration means they are subject to Terraform’s collaboration constraints. Application developers are invited to collaborate on their organisation’s infrastructure as if they were an operations team with a narrower focus. The platform team invites application development teams to share their workflow, rather than offering them a service. This means application teams must learn a new, special purpose toolset and language - Terraform and the HashiCorp Configuration Language (HCL). It also raises the level of configuration abstraction for application developers without raising the level of access control abstraction. While the platform team can publish a module that allows an application team to manage “RDS instances”, access control remains down at the cloud provider API level and is thus framed around “database subnet groups” and “database parameter groups”.
The Crossplane equivalent of a Terraform module is a Composite Resource - an XR. Each XR is exposed as an API endpoint. A platform team can define and document the OpenAPI schema of each XR - each API - and enforce Role Based Access Control (RBAC) at the API level. This means that if a platform team decides to frame the abstraction they offer to their development teams as “an AcmeCo PostgreSQL database” they can grant RBAC access to create, read, update, or delete an AcmeCo PostgreSQL database, rather than having to manage access to various underlying cloud concepts like RDS instances or subnet groups. Because Crossplane builds on the battle hardened Kubernetes RBAC system a platform team can easily support many teams of application developers within a single control plane. Each team can be granted access only to the abstractions they need - some might be able to manage only storage buckets, while others may be allowed to manage caches and databases.
Self service scales even further in Crossplane because any one XR may offer several classes of service. Crossplane decouples an XR’s inputs and outputs - its spec and status in Kubernetes parlance - from its implementation, which is described by a Composition. If an application developer has been granted access to create AcmeCo PostgreSQL databases, they may easily select from any of the classes of service - any of the Compositions - their platform team has declared to be compatible with said databases. These classes of service could represent production, staging, and development; AWS, Azure, and GCP; fast and slow; or any combination thereof.
Integration and Automation
Terraform sits in front of many APIs, but it does not offer its own. This leads many teams automate it by committing their Terraform configuration(s) to revision control - git - and executing Terraform as part of their CI/CD pipeline. This is an improvement relative to a team running Terraform from their laptops, but it exposes a key issue that organisations face when attempting to scale their use of Terraform. Terraform is a command line tool - not a control plane. Because it is a short lived, one-shot process it will only attempt to reconcile your desired configuration with actual infrastructure when it is invoked. Whether run from a CI/CD pipeline or a laptop Terraform is typically invoked only when an engineer expects that infrastructure needs updating.
Terraform’s conservative, ‘on-demand’ approach to reconciling desired with actual infrastructure state can lead to a novel deadlock. Recall that the process of applying a Terraform configuration is all-or-nothing - if you describe your caches and your databases in the same configuration you must always update both to update either. This means that if anyone in your organisation circumvents Terraform the next person to trigger a Terraform run will be faced with a surprising plan as it attempts to undo the change. Consider for example a scenario in which an engineer is paged in the middle of the night to handle an incident, makes some quick edits to the production cache configuration via the AWS console, and forgets to reflect those changes in Terraform. It’s not unheard of for infrastructure to drift so much that applying a Terraform configuration becomes a risky, intimidating proposition.
Crossplane, on the other hand, is built as a series of long lived, always-on control loops. It constantly observes and corrects an organisation’s infrastructure to match its desired configuration whether changes are expected or not. This disincentivizes teams from circumventing Crossplane. When Crossplane has been asked to manage a piece of infrastructure any change made outside it will automatically and persistently be reverted.
A continued theme among the pain points organisations face with Terraform is that it does not offer an API. Integrating with Terraform is challenging because it is configured using a Domain Specific Language (DSL) - HCL - and invoked by calling a command line tool. Crossplane exposes a REST API - the lingua franca of automation. Whether a team writes mostly shell scripts, mostly Python, or mostly Erlang common patterns and libraries will exist for integrating with REST APIs and thus for integrating with Crossplane.
Crossplane doesn’t expose any old REST API. Building on the Kubernetes API means that teams can orchestrate all of their infrastructure - cloud and otherwise - using tools like kubectl. The same tools they use to orchestrate their containerised applications. Crossplane can even expose the details an application needs to connect to infrastructure as a Kubernetes secret to ease integration. It can be paired with projects like ArgoCD, Gatekeeper, or Velero to enable GitOps, advanced policy, and backups. Need bespoke automation? Leverage one of many well documented frameworks to build your own Kubernetes operator that integrates with Crossplane.
Why Not Both?
Crossplane and Terraform can both orchestrate an organisation’s infrastructure. There are similarities between the two, but each project approaches orchestration differently. Terraform offers a command-line interface to control plane APIs, while Crossplane is itself a control plane that can be used to build abstractions atop other control planes. Because Crossplane enables a platform team to offer their own control plane it avoids many of the challenges platform teams face when scaling Terraform.
Savvy readers might notice that the two projects can complement each other - Terraform is an interface to control planes, and its Kubernetes provider allows to orchestrate the Kubernetes control plane! This means it’s possible to pair Terraform with Crossplane, for example if your organisation prefers HCL to YAML its possible for your platform team to use Terraform to define XRs and Compositions, and for your application teams use Terraform to plan and and apply changes to Crossplane’s desired state!
We think Crossplane is a great way for platform teams to empower the developers they support to self-service their infrastructure needs. Check out our guide to getting started with Crossplane if you’d like to try it for yourself, and reach out to us on Slack if you have any questions or feedback about Crossplane.