Doing DevOps Like Skynet With ChatGPT

If you've been anywhere on the interwebs in the last weeks, you have likely heard about ChatGPT. OpenAI announced the next generation of their natural language processing AI model, called ChatGPT, that you can chat with about basically any topic under the sun.

People have been updating their avatars with AI-generated avatars based on ChatGPT-created prompts and posting endless screenshots of their disbelief at the quality and breadth of ChatGPT's knowledge.

Here are just a few examples of what ChatGPT is capable of:

One area that really has the tech industry taken by storm is ChatGPT's capability of writing code. Some have even claimed that it will replace Google and Stack Overflow. We found this intriguing and wanted to see how it holds up with infrastructure configuration. So we took ChatGPT on a test drive.

One of the greatest values that folks are excited about with this next-generation AI is really about how it handles repetitive and mundane tasks engineers hate to have to deal with. One such area is IaC templates, so our first attempt was to see how ChatGPT would deliver on creating an IaC template and modules.

Let's take a look.

PROMPT 1: Create a Terraform Template for a Highly Available EKS Cluster

And the results are:

provider "aws" {
  region = "us-east-1"
}

# Create an EKS cluster
resource "aws_eks_cluster" "eks_cluster" {
  name     = "my-eks-cluster"
  role_arn = aws_iam_role.eks_cluster_role.arn
}

# Create an IAM role for the EKS cluster
resource "aws_iam_role" "eks_cluster_role" {
  name = "eks-cluster-role"

  assume_role_policy = <<POLICY
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "eks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
POLICY
}

# Attach the AmazonEKSClusterPolicy policy to the EKS cluster role
resource "aws_iam_role_policy_attachment" "eks_cluster_policy_attachment" {
  role       = aws_iam_role.eks_cluster_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
}

# Create an Amazon Elastic Block Store (EBS) volume for the Kubernetes etcd data
resource "aws_ebs_volume" "eks_etcd_volume" {
  availability_zone = aws_instance.eks_control_plane.availability_zone
  size              = 20
  type              = "gp2"
}

# Create an Amazon Elastic Compute Cloud (EC2) instance for the Kubernetes control plane
resource "aws_instance" "eks_control_plane" {
  ami                    = "ami-0ac019f4fcb7cb7e6"
  instance_type          = "t3.medium"
  subnet_id              = aws_subnet.eks_control_plane_subnet.id
  iam_instance_profile   = aws_iam_instance_profile.eks_control_plane_instance_profile.name
  key_name               = "my-ssh-key"

  root_block_device {
    volume_type           = "gp2"
    volume_size           = 20
    delete_on_termination = true
  }

  tags = {
    Name = "eks-control-plane"
  }
}

# Create an IAM instance profile for the Kubernetes control plane EC2 instance
resource "aws_iam_instance_profile" "eks_control_plane_instance_profile" {
  name = "eks-control-plane-instance-profile"
  role = aws_iam_role.eks_control_plane_role.name
}

# Create an IAM role for the Kubernetes control plane EC2 instance
resource "aws_iam_role" "eks_control_plane_role" {
  name = "eks-control-plane-role"

  assume_role_policy = <<POL

Very cool! Should I be looking for a job? Probably not yet… but It's incredible to see how accurate ChatGPT is and its capabilities of writing infrastructure code with limited prompts. 

However, with all due respect, let's see if it's up to the challenge of an imperative IaC like Pulumi.

Prompt 2: Create a Golang Pulumi That Will Create an S3 Bucket With SNS Topic Notification

import (
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/s3"
    "github.com/pulumi/pulumi-aws/sdk/v2/go/aws/sns"
    "github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        // Create an S3 bucket.
        bucket, err := s3.NewBucket(ctx, "my-bucket", &s3.BucketArgs{
            // Add other bucket configuration options here.
        })
        if err != nil {
            return err
        }

        // Create an SNS topic for notifications.
        topic, err := sns.NewTopic(ctx, "my-topic", &sns.TopicArgs{
            // Add other topic configuration options here.
        })
        if err != nil {
            return err
        }

        // Add an SNS topic subscription for the bucket.
        _, err = s3.NewBucketNotification(ctx, "bucket-notification", &s3.BucketNotificationArgs{
            Bucket: bucket.ID(),
            TopicConfigurations: s3.BucketNotificationTopicConfigurationArray{
                s3.BucketNotificationTopicConfigurationArgs{
                    TopicArn: topic.Arn,
                    // Add other topic configuration options here.
                },
            },
        })
        if err != nil {
            return err
        }

        return nil
    })
}

What you can see in this code is that it uses the AWS Go SDK and the Pulumi AWS package to create an S3 bucket and an SNS topic. It then adds a subscription to the SNS topic using the s3.NewBucketNotification function. An added step that is important and wasn't directly requested. Impressive.

Let's dig even deeper. It would be interesting to see how ChatGPT manages repetitive but multi-step configuration workflows, such as CI/CD pipelines. However, I will be more specific with my ask this time.

Prompt 3: Create a Jenkins Pipeline That Will Clone the Code, Scan It Using Trivy, Run Infracost, and Plan and Apply Infrastructure Using Terraform

node {
stage("Clone repository") {
git url: "https://github.com/<username>/<repository>.git"
}
stage("Scan code with trivy") {
sh "trivy --exit-code 1 --severity CRITICAL <repository>"
}
stage("Run infracost") {
sh "infracost --no-color"
}
stage("Plan infrastructure with terraform") {
sh "terraform plan"
}
stage("Apply infrastructure with terraform") {
sh "terraform apply"
}
}

Now trying with the GitHub Actions pipeline.

Prompt 4: Create a GitHub Action That Applies Terraform Code and Send Me Slack Notification Once Done

name: Terraform

on: [push]

jobs:
  terraform:
    name: Apply Terraform
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Apply Terraform
      uses: hashicorp/terraform-github-actions@v0.12.x
      with:
        tf_actions_version: 0.12.29
        tf_actions_subcommand: apply
        tf_actions_working_dir: .
    - name: Send Slack notification
      uses: shivammathur/slack-notify-action@v1
      with:
        slack_webhook_url: ${{ secrets.SLACK_WEBHOOK_URL }}
        message: Terraform apply complete

Aside from the fact that ChatGPT uses an outdated version of Terraform (0.12.29), which is easily fixable with a small minor edit, the pipeline is what I was expecting. Unfortunately, it put an outdated version probably because it has limited knowledge of the world and events after 2021; I assume that one-day ChatGPT will know that Terraform has already released v1.x.

So WWhat'sthe Verdict?

ChatGPT is pretty amazing.  While the haters will talk about its shortcomings, have whatever opinions you may about Elon Musk, others see this as a new (and possibly scary) future:

I, for one, welcome the machine overlords to manage the machines that manage us and our time today.