Building a Platform Abstraction for AWS Networks Using Crossplane

Crossplane helps platform engineers develop abstractions for developers. It is an open-source, multicloud control plane that handles interactions with cloud providers’ APIs for you.

In this post, I’ll show how developers can create an AWS network (VPC, Subnet, etc.) with just a single YAML request to the Kubernetes API.

Crossplane leverages Kubernetes CRD (Custom Resource Definition), which extends Kubernetes APIs with new custom objects. Along with that, Crossplane creates controllers for these objects, implementing the reconciliation loop to ensure the system’s actual state always matches the declared state.

For example, if you define a Table custom resource with spec.legs = 3, Kubernetes continuously checks:

“Does this table have three legs?”

• If it finds only two, it builds another.
• If someone manually adds a 4th, it removes the extra one.

This ensures the resource has exactly three legs—no more, no less.

Prerequisites 

• Docker and Kubernetes setup
• AWS account with user or IAM role credentials (Below, I provided steps to set up a user in AWS.)

Docker Kubernetes Setup 

I am assuming you already have a local Kubernetes setup. If not, you can install Docker Desktop from https://docs.docker.com/desktop/setup/install/mac-install/ and enable Kubernetes. 

Confirm the setup by checking the Kubernetes version. 

kubectl version

Crossplane Installation 

Now, we will install Crossplane using Helm. Helm is a packaging tool for Kubernetes resources. 

Install Helm using: 

brew install helm

Confirm, using version check.

helm version

Let's add Helm repo for Crossplane using the commands:

helm repo add crossplane-stable https://charts.crossplane.io/stable

helm repo update

And then, the command below will install Crossplane.

helm install crossplane --namespace crossplane-system --create-namespace crossplane-stable/crossplane

You can change the namespace to one of your choice, and then check the installation status using:

helm list -n crossplane-system

Provider 

In Crossplane, a provider acts like a software package or driver that enables functionality. It bundles the necessary custom resource definitions (CRDs) and controllers to interact with external systems, most often, cloud provider APIs. In our case, we’ll use provider-aws-ec2, which allows us to manage AWS EC2 instances and networking resources. 

To install Provider, save the lines below as .yaml file:

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
 name: provider-aws-ec2
spec:
 package: xpkg.upbound.io/upbound/provider-aws-ec2:v1

And apply using:

kubectl apply -f <your-file-name>.yaml

AWS Authentication 

Now that the provider is installed, we need to authenticate it with our AWS account. This requires AWS credentials, which can come from either an IAM user or an IAM role, depending on how your AWS account is configured. In this guide, I’ll use an IAM user, and the steps are as follows: 

1. Go to AWS IAM Console. 
2. Create an IAM user, use a name of your choice, e.g., crossplane-user.
3. Attach policies (Administrator Access).
4. Save aws_access_key_id and aws_secret_access_key in a file called "aws-creds" like:

[default]
aws_access_key_id = <Your value>
aws_secret_access_key = <your value>

Now, using this file, we will create a Kubernetes secret using the command below. This secret is going to be used by Crossplane controllers to interact with AWS APIs. 

kubectl create secret generic aws-creds \
  --from-file=creds=./aws-creds

To link this secret with Crossplane, we need something called a ProviderConfig. Create the provider config by saving the YAML content below into a .yaml file:

apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-creds
      key: creds

Using kubectl apply -f <your-file-name>.yaml. Note that we used the name of the secret aws-creds as a reference to this provider. 

Crossplane Composite Resource Definition (XRD)

This is where Crossplane leverages Kubernetes custom resource definitions (CRDs) to define custom resources that represent infrastructure abstractions, essentially the interface. A composite resource (XR) bundles and abstracts the underlying cloud resources, so instead of worrying about configuring VPCs, subnets, and routing, developers can simply declare that they need an AWS network. 

Think of it like ordering a dish: You care about the outcome, not the individual ingredients. To create an XRD, save the following content into a .yaml file: 

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xnetworks.aws.platformref.crossplane.io
spec:
  group: aws.platformref.crossplane.io
  names:
    kind: XNetwork
    plural: xnetworks
  claimNames:
    kind: Network
    plural: networks
  connectionSecretKeys:
    - vpcId
    - subnetIds
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                parameters:
                  type: object
                  properties:
                    vpcCidr:
                      type: string
                  required: [vpcCidr]
              required: [parameters]
            status:
              type: object
              properties:
                vpcId:
                  type: string
                subnetIds:
                  type: array
                  items:
                    type: string

And apply:

kubectl apply -f <your-file-name>.yaml

Crossplane Composition

If the XRD is the dish you want to cook, the Composition is the recipe (the implementation). It specifies all the raw materials needed. For an XNetwork resource, the Composition ensures that the subnet and VPC are created. To create a composition, save the below content into a YAML file and apply: 

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: network.aws.platformref.crossplane.io
  labels:
    crossplane.io/xrd: xnetworks.aws.platformref.crossplane.io
spec:
  compositeTypeRef:
    apiVersion: aws.platformref.crossplane.io/v1alpha1
    kind: XNetwork
  resources:
    - name: vpc
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: VPC
        spec:
          forProvider:
            cidrBlock: 10.0.0.0/16
            enableDnsSupport: true
            enableDnsHostNames: true
            region: us-east-1
          providerConfigRef:
            name: default
      patches:
        - fromFieldPath: spec.parameters.vpcCidr
          toFieldPath: spec.forProvider.cidrBlock
          type: FromCompositeFieldPath
        - fromFieldPath: spec.claimRef.name
          toFieldPath: spec.forProvider.tags.Name
          type: FromCompositeFieldPath
          transforms:
            - type: string
              string:
                fmt: "%s-vpc"
        - fromFieldPath: status.atProvider.id
          toFieldPath: status.vpcId
          type: ToCompositeFieldPath
    - name: subnet
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: Subnet
        spec:
          forProvider:
            cidrBlock: 10.0.1.0/24
            availabilityZone: us-east-1a
            mapPublicIpOnLaunch: true
            region: us-east-1
          providerConfigRef:
            name: default
      patches:
        - fromFieldPath: status.vpcId
          toFieldPath: spec.forProvider.vpcId
          type: FromCompositeFieldPath
        - fromFieldPath: spec.claimRef.name
          toFieldPath: spec.forProvider.tags.Name
          type: FromCompositeFieldPath
          transforms:
            - type: string
              string:
                fmt: "%s-subnet"
        - fromFieldPath: status.atProvider.id
          toFieldPath: status.subnetIds[0]
          type: ToCompositeFieldPath

Kubectl apply -f <your-composition-filename>.yaml

The API Call For Developers

Now we are ready to make the API call that a developer will do to create an AWS network. All a developer needs to do is make the following claim to Kubernetes, and the network will magically appear. To do so, save the below content into a YAML file.

apiVersion: aws.platformref.crossplane.io/v1alpha1
kind: Network
metadata:
  name: crossplane-demo-network
  namespace: default
spec:
  parameters:
    vpcCidr: "10.0.0.0/16"
  compositionRef:
    name: network.aws.platformref.crossplane.io

And apply:

kubectl apply -f <your-file-name>.yaml

By using XRD and Composition, you can give developers the flexibility to choose which features to enable and which parameters to provide. 

Conclusion 

Crossplane is a powerful tool for platform engineers. Without it, organizations would need to rely on tools like Terraform or CloudFormation to manually create resources and manage complexities such as state files and cloud services. With Crossplane, much of that burden is eliminated. There’s much more to explore, which I’ll cover in upcoming articles.