Design Compositions For Organizational Needs: Crossplane

This article is for curious individuals with specific requirements for building and operating cloud infrastructure efficiently along corporate guidelines, governance, security, and the breadth of systems that can be configured and resource lifecycle managed with Crossplane.

Custom Cloud Platform Abstraction Layer

Custom cloud platforms are purpose built to satisfy specific organizational requirements as a meaningful abstraction layer. They allow creators to innovate behind their API that reduces the cognitive load required from their users. Custom cloud creators infer and seamlessly update regulatory and organizational mandates behind an evolving stable custom cloud platform user interface.

When platform teams offer well understood features behind their own supported API versions, this yields a higher chance of business success compared to passing through native provider capabilities and features directly to end users. Some of the native features may break aspects of the proprietary environment out of the box or accidentally reduce compliance. Others may simply not be known to the platform team at the time when their users already use them and now run into issues where the users seek platform team support.

Crossplane Zero To Hero

Crossplane is the open source framework created by Upbound that supports building the above mentioned control plane based custom cloud platforms. It enables flexible resource compositions to satisfy your organizational needs. Upbounders build platforms with the Crossplane framework, and we have great respect for cloud platform and service team users.

Use the Crossplane getting started guide and the 'What Is Crossplane?’ O’Reilly report to get familiar with control plane based platform building.

When communicating your platform performance through solid service level objectives (SLOs), you bring clarity about your platform value to your users. Practically, SLOs may describe requested resource availability within a deterministic timeframe, e.g. a compute cluster with all of its network, storage, security, and other dependencies is ready for use in under 30 minutes 99 percent of the time. Crossplane can help establish that with x-metrics while satisfying the various needs of your customers and stakeholders including finance, governance, security, IT, legal, procurement, operations, support and internal and external partners.

The Crossplane framework runs in Kubernetes and extends its resource model. It enables teams to create APIs through composite resource definitions (XRDs). Resource claims that conform to the APIs are issued against the Kubernetes API server. Platform teams create compositions that map the XRDs to managed resources (MRs). The MRs represent external resources. The external resources are created by Crossplane providers. The providers perform frequent reconciliation between MRs and external resources.

Organizations exist to fulfill a purpose including their business units. Verticals share commonalities and they have their own requirements, tooling and functions. Different groups may have competing needs that will benefit from an architecture that allows enough autonomy and flexibility to address new API requirements concurrently with API and composition revisions, and upgrades of packages and distributions. Crossplane is conveniently agnostic about the tools and technologies that it can be integrated with.

Visit the Crossplane architecture documentation to learn about creating your own cloud API, composing the functionality from multiple cloud resources, packaging everything in a configuration and distributing it to offer the platform interface yourself or enable a customer or partner team. Learn about single and multi control plane architectures, GitOps workflows and integrations, secrets management, policy engines, observability and platform continuity with disaster recovery.

Design Principles

Tagging: Rich automated resource tagging is highly recommended for easier audits, meeting compliance requirements, and answering ad-hoc questions from stakeholders such as where to escalate an issue related to a set of resources for a particular incident.

Multi-Tenancy: Multi-tenant access segregation for resource claims is an excellent pattern for proper cloud governance and security.

Security: External secret stores can keep credentials and private information safe and secure. Cloud provider agnostic stores offer a solution that can transcend specific clouds.

GitOps: Integration into powerful workflows including GitOps will reduce your own configuration package distribution overhead and simplifies how your platform accepts resource claims from your users.

Iterate: Maintain short feedback loops with your customers and stakeholders to learn which features and aspects are important to them. The Crossplane framework enables you to iterate with confidence.

Composition Iteration

Use the following approach to increase your composition writing and validation velocity.

Start with creating Managed Resources for quick rewards seeing external resources.
Reformulate the Managed Resources into Kuttl tests to surface potential issues systematically.
Merge Managed Resources into a singular assert file in preparation for compositions.
Adjust Managed Resource names to simplify tracing.
Establish resource claims as part of the testing phase that will invoke the custom platform API.
Initialize the composition and the composite resource definition (XRD) which is your API.
Use test-driven composition development with tools like Kind, Skaffold, Yamllint and Kuttl.
Receive quick Kuttl test loop feedback highlighting discrepancies between the rendered MRs and their expected values.
Each Kuttl test takes 2-5 minutes and simplifies pinpointing errors.
Benefit from Kuttl test logs and cluster event data for debugging.
Execute future-proof testing for seamless scalability and evolution of the composition.

For more information and real time help, feel free to join the Crossplane slack channel and give Crossplane a try with one of the reference architectures on the Upbound marketplace. Happy crossplaning!