Azure secure connectivity for AKS & Azure DB

Phil Prasek, Dan Mangum |

Azure Database instances can now be securely consumed from app deployments in an AKS cluster using kubectl with new Crossplane resources in the Azure Stack including: Virtual Networks (VNets), Subnets, VNet Rules, and more!

Continuing our series on cloud networking & security we’ll explore new Crossplane support for provisioning Azure networking and security resources from kubectl, so pods in an AKS cluster can securely connect to Azure DB instances (MySQL, PostgreSQL) using VNet Service Endpoints and VNet Rules.

Previously in Crossplane, all networking for AKS clusters and Azure DB instances had to be configured manually or with Infrastructure as Code tools separate from Kubernetes-native tools like kubectl, and then stitched together to obtain a complete deployment.

With Crossplane v0.3, kubectl can now be used to provision networking and security resources, the AKS cluster itself, and Kubernetes apps that dynamically consume Azure DB instances using Crossplane claims and classes.

Considerations for Azure secure connectivity

In general, Azure supports multiple options for securely connecting Azure resources:

  1. Private IP within a Virtual Network (VNet)
  • Azure resources that can be deployed into a VNet are able to communicate freely using private IP addresses with other resources in the same VNet.
  1. Virtual Network Service Endpoint
  1. Virtual Network Peering
  • VNet peering enables communication between Azure resources deployed into different VNets, as if they were on the same VNet.

AKS Clusters are a managed service, but the individual nodes are deployed into a VNet. Azure DB for MySQL and PostgreSQL, on the other hand, are "service resources" provided in a multi-tenant PaaS that lives outside of your VNet, and do not support deploying into a VNet.

Azure provides two options for pods running on an AKS worker nodes to access a MySQL or PostgreSQL DB instance:

  1. Create a firewall rule on the Azure DB Server with a range of IP addresses that encompasses all IPs of the AKS Cluster nodes (this can be a very large range if using node auto-scaling).
  2. Create a VNet Rule on the Azure DB Server that allows access from the subnet the AKS nodes are in. This is used in conjunction with the Microsoft.Sql VNet Service Endpoint enabled on the cluster subnet.

VNet Rules (option #2) are recommended and preferable in this situation for several reasons. Nodes are often configured with dynamic IP addresses that can change when a node is restarted resulting in broken firewall rules that reference specific IPs. Nodes can be added to a cluster which would require updating the firewall rule to add additional IPs. VNet Rules avoid these issues by granting access to an entire subnet of AKS nodes.

Manual steps

Configuring a secure networking environment for AKS and Azure DB requires the following:

1) AKS cluster setup

  • ResourceGroup: a logical grouping a resources required for all resources.
  • VNet: creates a virtual network for the AKS cluster nodes.
  • Subnet has a range of private IPs for AKS cluster nodes
  • Create an AKS cluster using the above resources.

2) Configure managed service access

  • VNet Service Endpoint: update the cluster subnet above with a service endpoint for Microsoft.Sql to enable connectivity for new Azure DB service resource.

3) Provision managed services with private IPs on the cluster’s network

  • Provision managed Azure DB service instances: PostgreSQL, MySQL.
  • VNet Rule for each managed service instance to allow traffic from all nodes in the cluster subnet to a given Azure DB service instance (PostgreSQL, MySQL).

Streamline AKS & Azure DB provisioning with kubectl

Crossplane v0.3 supports doing all of the above using kubectl by following the Azure Stack Setup Guide. It uses a standalone Crossplane control cluster running on your laptop or in the cloud to create a secure networking environment in Azure and dynamically provision an AKS cluster with Azure DB provisioning enabled.

Note the following:

  • The AKSClusterClass has fields for the cluster’s ResourceGroup and Subnet.
  • The SQLServerClass has a field for the ResourceGroup and relies on a separate VNet Rule (per MySQL or PostgreSQL instance), to grant access to all nodes in the cluster’s subnet.

Add Azure DB provisioning to existing AKS clusters

If you already have an existing AKS cluster then you can follow the Azure Services Guide to install an embedded Crossplane instancedirectly onto the target AKS cluster along with the Azure Stack.

This adds managed service CRDs to the Kubernetes API including Azure DB for MySQL and PostgreSQL. You can continue deploying to Kubernetes with the core resources you use today (deployments, pods, services, ingress) and augment that with additional Kubernetes objects (mysql.yaml, postgres.yaml) to provision managed services using kubectl.

Provision securely connected DB instances

Once managed service provisioning is enabled you can dynamically provision Azure DB instances in your app project namespaces using PostgreSQL and MySQL claims, but now with secure connectivity!

Beyond v0.3

Crossplane v0.3 supports provisioning securely connected Azure DB instances using VNet Service Endpoints and VNet Rules. If you have other scenarios you’d like us to support please let us know!

Learn more

To learn more about key design considerations, how secure connectivity fits into Crossplane's layered architecture, and a detailed example to make it real, checkout:

Get involved!

There are many different ways to get involved in the Crossplane project, both from the user side and the developer side. Please join us in helping the project continue to grow on its way beyond the v0.3 milestone as we move from alpha to beta over the coming months!

Join the open cloud movement to help level the playing field for everyone!

Keep up with Upbound

* indicates required