TL;DR - Summary
- We’re starting a new blog post series for FAQs in Crossplane
- Today’s first post explores “Why is my composition not working”?
- First, look at the status and events on your top level objects for relevant information
- Then, follow the chain of references to composed objects to get more details
- A detailed walkthrough can be found in this Crossplane docs section
- Let us know what other questions you want answered in this new series!
A New Blog Series to Answer Your Crossplane FAQs
This post is the first in a new series that we’re introducing on the Crossplane blog today. The Crossplane project has a considerably large surface area of functionality due to its ability to manage essentially anything that has an API. The core Crossplane machinery, its resource model, extensibility, compositions, patching/transforming, functions, package manager, and others have a lot of capability. Crossplane can do a lot.
We’ve heard feedback multiple times that due to this large amount of functionality, the learning curve for Crossplane can be quite steep. As we continue to address this with both developer experience and documentation improvements, we thought we’d also tackle some of these challenges in this brand new blog series of Frequently Asked Questions (FAQs) from the Crossplane community. You can expect us to regularly publish new posts that each dive into a common confusion or obstacle for Crossplane users, so keep coming back for more entries in the series! And of course, we will be continually incorporating this new content back into the docs as well to keep making them better.
Why is my composition not working?
The first question we’ll tackle today is certainly a common one. Imagine you have used Crossplane’s composition feature to offer a custom infrastructure API to your developers called AcmeDatabase. This database resource captures all the important config and policy from your platform team and exposes it as a simple abstraction to your developers with just a few configuration options you’ve chosen to make available to them.
To review, this scenario is done by creating 2 main entities in Crossplane:
CompositeResourceDefinition(XRD) that defines the schema (shape) of theAcmeDatabaseAPI and its config options that will be exposed to the devsCompositionthat defines the specific resources and their configuration for Crossplane to compose together at runtime when your devs request a newAcmeDatabase
With these pieces in place, your developer can create a simple AcmeDatabase via GitOps or kubectl and the Crossplane machinery will start creating and connecting all the resources the platform team specified in the corresponding Composition. Here's an example of using your custom AcmeDatabase API.
apiVersion: acme.com/v1
kind: AcmeDatabase
metadata:
name: db-prod
spec:
storageGB: 15
Using this API the developer has now created an AcmeDatabase, but after waiting patiently for it to come online, it still isn’t ready. Where do we start looking to solve this?
Status and Conditions
Every object in the Crossplane resource model has consistent status fields, where the latest state of your AcmeDatabase will be written by the Crossplane controllers to its .status section. Of particular note in the status will be the .conditions:
❯ kubectl get AcmeDatabase db-prod -o jsonpath='{.status}' | jq .
{
"conditions": [
{
"lastTransitionTime": "2023-06-22T10:35:06Z",
"reason": "ReconcileSuccess",
"status": "True",
"type": "Synced"
},
{
"lastTransitionTime": "2023-06-22T10:35:06Z",
"reason": "Composite resource claim is waiting for composite resource to become Ready",
"status": "False",
"type": "Ready"
}
]
}
These status conditions tell us that our AcmeDatabase claim is waiting for its underlying composite resource (XR) to become ready. Remember that a claim is basically a namespaced representation of its underlying XR, so the XR would be the next logical place to look for more details. We’ll see how to find this XR and get more information from it in just a bit.
Events Tell a Story
Crossplane is also very consistent about recording the actions it takes and outcomes it sees into standard Kubernetes Events. These Events tell the history of what happened to the object and where it may be stuck. Let’s look at the events for the AcmeDatabase (with some details omitted for readability):
❯ kubectl describe AcmeDatabase db-prod
Name: db-prod
Namespace: default
…
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
…
Normal BindCompositeResource 9m56s (x5 over 9m57s) offered/xrd Composite resource is not yet ready
In this case, there’s a recurring (5 times so far) event that is reinforcing what the status/conditions indicated: this AcmeDatabase claim is waiting for its underlying XR to become ready. Let’s dig deeper and look at the Claim's references.
Follow the References
In Crossplane’s resource model, a namespaced claim will point to its underlying composite resource (XR). In turn, that XR will point to all of its child resources that it is bringing together into a single abstraction. Let’s follow the chain of references from claim → composite resource (XR) → child composed resources, as visualized below, and find available details at each level.
AcmeDatabase claim points to its underlying XAcmeDatabase composite resource from its .spec.resourceRef field:
❯ kubectl get AcmeDatabase db-prod -o jsonpath='{.spec.resourceRef}' | jq .
{
"apiVersion": "acme.com/v1",
"kind": "XAcmeDatabase",
"name": "db-prod-zrg9f"
}
The XAcmeDatabase points to its child composed resources from its spec.resourceRefs field (note the plural there). In this case, it’s pointing to a single GCP SQL database, but in other cases, it could be multiple infrastructure resources:
❯ kubectl get XAcmeDatabase db-prod-zrg9f -o jsonpath='{.spec.resourceRefs}' | jq .
[
{
"apiVersion": "sql.gcp.upbound.io/v1beta1",
"kind": "DatabaseInstance",
"name": "db-prod-zrg9f-ztvsg"
}
]
Now that we’ve found the leaf managed resource(s), we can examine their status conditions and events to find a root cause:
❯ kubectl get DatabaseInstance db-prod-zrg9f-ztvsg -o jsonpath='{.status.conditions}' | jq .
[
{
"lastTransitionTime": "2023-06-22T10:42:08Z",
"message": "...cannot get credentials secret: Secret \"gcp-secret\" not found",
"reason": "ReconcileError",
"status": "False",
"type": "Synced"
}
]
There we go, our SQL database isn’t being created in GCP because the credentials secret it is supposed to use to authenticate with GCP is missing. Now we know exactly what to fix to get our AcmeDatabase (backed by a GCP SQL database) up and running!
More Details in the Docs
This specific example was somewhat basic, but this general type of scenario commonly occurs while using Crossplane, so we hope you’ve found it useful.
Troubleshooting to find the root cause of why a composite resource is not working can be explored in even more details in this official Crossplane documentation section.
Answers for all your Questions
Now that we’ve answered one Crossplane FAQ, we’re going to keep the momentum going!
We have some pretty good ideas of popular questions from all the ways that we get to interact with our amazing community, but we’d still love to hear more from you all. What Crossplane questions do you have on your mind? For any questions or problems with Crossplane that you’d like to see featured in this new FAQ series, just post about it on the Crossplane Slack #documentation channel, and we’ll try to include them in an upcoming post!
Crossplane is a community driven project and we welcome you to join the community and contribute through a variety of opportunities, such as opening and commenting on issues, joining the community meetings, sharing your adoption story, and providing feedback on design docs and pull requests.
We love to hear from the community, as they are exactly what makes this project great. Whether you are a developer, user, or just interested in what we're up to, feel free to join us via one of the following methods: