Azure Database instances can now be securely consumed from app deployments in an AKS cluster using kubectl with new Crossplane resources in the Azure Stack including: Virtual Networks (VNets), Subnets, VNet Rules, and more!
Continuing our series on cloud networking & security we’ll explore new Crossplane support for provisioning Azure networking and security resources from
kubectl, so pods in an AKS cluster can securely connect to Azure DB instances (MySQL, PostgreSQL) using VNet Service Endpoints and VNet Rules.
Previously in Crossplane, all networking for AKS clusters and Azure DB instances had to be configured manually or with Infrastructure as Code tools separate from Kubernetes-native tools like
kubectl, and then stitched together to obtain a complete deployment.
With Crossplane v0.3,
kubectl can now be used to provision networking and security resources, the AKS cluster itself, and Kubernetes apps that dynamically consume Azure DB instances using Crossplane claims and classes.
Considerations for Azure secure connectivity
In general, Azure supports multiple options for securely connecting Azure resources:
- Private IP within a Virtual Network (VNet)
- Azure resources that can be deployed into a VNet are able to communicate freely using private IP addresses with other resources in the same VNet.
- Virtual Network Service Endpoint
- Service resources (Azure DB for MySQL and PostgreSQL) can be made available to other resources in a VNet using a VNet Service Endpoint. This is done by adding a shared service tag like Microsoft.Sql to the cluster subnet’s service endpoints.
- Virtual Network Peering
- VNet peering enables communication between Azure resources deployed into different VNets, as if they were on the same VNet.
AKS Clusters are a managed service, but the individual nodes are deployed into a VNet. Azure DB for MySQL and PostgreSQL, on the other hand, are "service resources" provided in a multi-tenant PaaS that lives outside of your VNet, and do not support deploying into a VNet.
Azure provides two options for pods running on an AKS worker nodes to access a MySQL or PostgreSQL DB instance:
- Create a firewall rule on the Azure DB Server with a range of IP addresses that encompasses all IPs of the AKS Cluster nodes (this can be a very large range if using node auto-scaling).
- Create a VNet Rule on the Azure DB Server that allows access from the subnet the AKS nodes are in. This is used in conjunction with the
Microsoft.SqlVNet Service Endpoint enabled on the cluster subnet.
VNet Rules (option #2) are recommended and preferable in this situation for several reasons. Nodes are often configured with dynamic IP addresses that can change when a node is restarted resulting in broken firewall rules that reference specific IPs. Nodes can be added to a cluster which would require updating the firewall rule to add additional IPs. VNet Rules avoid these issues by granting access to an entire subnet of AKS nodes.
Configuring a secure networking environment for AKS and Azure DB requires the following:
1) AKS cluster setup
ResourceGroup: a logical grouping a resources required for all resources.
VNet: creates a virtual network for the AKS cluster nodes.
Subnethas a range of private IPs for AKS cluster nodes
- Create an AKS cluster using the above resources.
2) Configure managed service access
VNet Service Endpoint: update the cluster subnet above with a service endpoint for
Microsoft.Sqlto enable connectivity for new Azure DB service resource.
3) Provision managed services with private IPs on the cluster’s network
- Provision managed Azure DB service instances: PostgreSQL, MySQL.
VNet Rulefor each managed service instance to allow traffic from all nodes in the cluster subnet to a given Azure DB service instance (PostgreSQL, MySQL).
Streamline AKS & Azure DB provisioning with kubectl
Crossplane v0.3 supports doing all of the above using
kubectl by following the Azure Stack Setup Guide. It uses a standalone Crossplane control cluster running on your laptop or in the cloud to create a secure networking environment in Azure and dynamically provision an AKS cluster with Azure DB provisioning enabled.
Note the following:
- The AKSClusterClass has fields for the cluster’s ResourceGroup and Subnet.
- The SQLServerClass has a field for the ResourceGroup and relies on a separate VNet Rule (per MySQL or PostgreSQL instance), to grant access to all nodes in the cluster’s subnet.
Add Azure DB provisioning to existing AKS clusters
This adds managed service CRDs to the Kubernetes API including Azure DB for MySQL and PostgreSQL. You can continue deploying to Kubernetes with the core resources you use today (deployments, pods, services, ingress) and augment that with additional Kubernetes objects (mysql.yaml, postgres.yaml) to provision managed services using
Provision securely connected DB instances
Once managed service provisioning is enabled you can dynamically provision Azure DB instances in your app project namespaces using PostgreSQL and MySQL claims, but now with secure connectivity!
Crossplane v0.3 supports provisioning securely connected Azure DB instances using VNet Service Endpoints and VNet Rules. If you have other scenarios you’d like us to support please let us know!
To learn more about key design considerations, how secure connectivity fits into Crossplane's layered architecture, and a detailed example to make it real, checkout:
- How Crossplane Enables Secure Connectivity
- Azure Stack Setup Guide - provision complete GKE environments from
- Azure Services Guide - add cloud services to existing clusters
- API Reference - all supported networking and security resources
There are many different ways to get involved in the Crossplane project, both from the user side and the developer side. Please join us in helping the project continue to grow on its way beyond the v0.3 milestone as we move from alpha to beta over the coming months!
Join the open cloud movement to help level the playing field for everyone!