DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Building Scalable Data Lake Using AWS
  • Building a Scalable ML Pipeline and API in AWS
  • Breaking AWS Lambda: Chaos Engineering for Serverless Devs
  • AWS Step Functions Local: Mocking Services, HTTP Endpoints Limitations

Trending

  • Top Book Picks for Site Reliability Engineers
  • Event-Driven Architectures: Designing Scalable and Resilient Cloud Solutions
  • Docker Base Images Demystified: A Practical Guide
  • A Developer's Guide to Mastering Agentic AI: From Theory to Practice
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. AMI and Snapshot Management Using AWS Lambda

AMI and Snapshot Management Using AWS Lambda

This tutorial takes you through the process of creating an AWS Lambda function that goes through EC2 cycles and creates AMIs for instances with tags.

By 
Akhil Teej user avatar
Akhil Teej
·
Updated May. 19, 22 · Tutorial
Likes (11)
Comment
Save
Tweet
Share
23.7K Views

Join the DZone community and get the full member experience.

Join For Free

Create AMIs automatically with AWS Lambda


Automating tasks and continuously improving workflows are key parts of DevOps. In a CI/CD pipeline, being able to simplify processes is a huge plus. Not only will simplifying systemic processes make life easier for you as a developer, but there are other added benefits to gain as well. For example, you can significantly reduce the cost of running services in the cloud with refined and automated scaling of infrastructure.

AWS Lambda is the latest in cloud computing refinement which offers the ability to execute “serverless” code. Meaning, that AWS will provide the run-time platform for us. We" re now used to using Amazon Machine Images (AMIs) to store software configuration and other variables. AMIs certainly minimize human errors in a CI/CD pipeline. Problems such as misconfiguration can be avoided entirely. AWS Lambda simply supercharges the use of AMIs by making them completely serverless.

You may also enjoy: AWS Lambda Best Practices

Why Go Serverless?

Let’s not forget that AMIs need to run off of an instance, and most of the time that instance needs to be kept alive for the scripts to run properly. Automation becomes an added cost, despite bringing a lot of benefits. Switching to a serverless design makes AMIs that much more powerful.

At the same time, scripts designed to automate cloud instance creation, update, maintenance, and deletion can be triggered by events. This means you can configure the entire cloud environment to be more agile as well.

This leads to another major benefit: more automation opportunities. With AMIs, you can automate the deletion of old AMI backups without having to keep an EC2 instance running all the time. You can also trigger actions based on storage usage or other metrics captured by CloudWatch.

In this post, we will cover how to automate AMIs for your instances using AWS Lambda and CloudWatch. We have built two functions to achieve this, one is to create AMIs and the other is to delete older AMIs with retention.d

AWS Lambda is perfect for these tasks. The service is designed for small and on-demand apps or services that are meant to be triggered by specific events. AWS handles the constant monitoring for you, giving you more resources to focus on the rest of the process. Lambda currently supports the following languages: Node.js, Java, C#, and Python. We’ll be using Python3.7 to write our functions in this post. We will use CloudWatch Events to trigger the Lambda function, and we can create CloudWatch rules using Cron expressions.

Configuring AWS Lambda

Similar to other Amazon services, you need to start the configuration process by giving AWS Lambda sufficient service roles to access the rest of the environment. Lambda needs access to describe instances, create/deregister images, and delete snapshots. You can do this from the IAM menu from the AWS Management Console or by creating a policy directly. Log on to AWS Management Console and click on the IAM Service which will redirect you to the IAM console.

  • Click on "Policies" on the left pane of the console.

Policies

Policies



  1. Click on "Create policy," then choose the "JSON" tab and paste the below JSON data into the field.

JSON
 




x
37


 
1
{
2
    "Version": "2012-10-17",
3
    "Statement": [
4
        {
5
            "Sid": "VisualEditor0",
6
            "Effect": "Allow",
7
            "Action": [
8
                "ec2:DeleteSnapshot",
9
                "ec2:CreateSnapshots",
10
                "ec2:CreateSnapshot"
11
            ],
12
            "Resource": [
13
                "arn:aws:ec2:*:*:instance/*",
14
                "arn:aws:ec2:*::snapshot/*",
15
                "arn:aws:ec2:*:*:volume/*"
16
            ]
17
        },
18
        {
19
            "Sid": "VisualEditor1",
20
            "Effect": "Allow",
21
            "Action": [
22
                "ec2:DescribeImages",
23
                "ec2:DeregisterImage",
24
                "ec2:DescribeInstances",
25
                "ec2:DescribeSnapshotAttribute",
26
                "ec2:DescribeInstanceAttribute",
27
                "ec2:DescribeImageAttribute",
28
                "logs:*",
29
                "ec2:CreateImage",
30
                "ec2:DescribeSnapshots",
31
                "ec2:DescribeInstanceCreditSpecifications",
32
                "ec2:DescribeInstanceStatus"
33
            ],
34
            "Resource": "*"
35
        }
36
    ]
37
}



Create policy

Create policy


  • Click on "Visual editor"  to double check the given access, then select "Review policy."
  • Give your new policy a unique name and check the constraints for the policy name. Then click on "Create policy."
  • Now, back in the IAM Console, go to "Roles" to create an IAM Role attach the policy we have created. Start by clicking on "Create role."
  • Click on "Lambda" as the service that will use this role.

Choosing service

Choosing service


  •  Search for your created IAM Policy and click on the checkbox.
  •  Review the IAM Role and Click on Create Role.

Create Lambda Functions

The next step is adding functions to the Lambda instance. Depending on what you are trying to achieve, you can add functions that perform specific tasks under certain conditions. For example, you can create a function for making and managing AMI backups based on tags. You can use tags like “Backup” and “Delete” to organize AMI backups.

  •  Go to the "Lambda" service in your AWS Management Console. 

Lambda service

Lambda service


  •  Go to "Functions" and click on "Create function."

Create function

Create function


  •  Fill the function details in as per the below screenshot and attach the role that you have just created.

Function information

Function information


  •  Scroll down to "Function code" and copy this code into the field.

Copied code

Copied code


  • Next, scroll down to "Environment variables" and create the environment variables as below or mention the "instance id" directly as in below screenshot.
Environment variable creation
Environment variable creation


  •  Set the timeout value to 15 minutes; this is the threshold value of execution time for our Lambda function.
Setting timeout value
Setting timeout value


  •  Save the changes by clicking on "Save" in the top-right corner, and then click on "Test."
Saving
Saving


  •  Now to "Configure a test event."
  •  Create a tag “backup” for instances that you wanted to schedule for image creation.
Tag
Tag


  • Now go to the Lambda function and run it by clicking "Test." We can check the output in log output and we can also check in our CloudWatch logs. Track the created AMIs in the AMI console and you'll be able to see your respective snapshots in the Snapshot console too. 
Execution results
Execution results


  • Next, go to the CloudWatch console, click on logs and search for your Lambda functions. We can explore all the logs and also set "Expire Events After" to 30 days.
Expiration
Expiration


  •  Go back to the Lambda function. And click on the "Add trigger" in the designer console and select the CloudWatch Events.
Add trigger
Add trigger


  • We can add a Cron expression to the Lambda function, this will automatically trigger the function as per the Cron instructions. The Cron expression mentioned in the screenshot is for a daily launch at 5 pm. For more details on Cron expressions, please refer to AWS Schedule Expressions.
Add Trigger
Add Trigger


  • Similarly, repeat the above steps and create a "Delete function" and copy this code to the "Function" code and add "Environment variables" again as per the below screenshot. We can add a retention period as 30. It means the function scans the list for AMIs which are older than 30 days and deletes them accordingly. We can change the retention to 15 or 30 based upon our requirement. Also, add timeout values and triggers as a create AMI function.
Environment variable
Environment variable


  • Congratulations, you have successfully created Lambda functions and configured them to automatically manage AMIs and EBS Snapshots.

As well as CloudWatch, Lambda can capture triggers from various sources, including SNS. It also supports API calls and AWS IoT for more complex operations.

As per the above steps, the Lambda functions you create will cycle through your EC2 cycles, create a list of all instances with a suitable tag, and begin the process of creating AMIs for those instances. The created AMI backups can have metadata and tags added to them, including information about how long they need to be retained.

As shown, the same function can also automate the deletion of AMI it creates after a certain period of time has passed. Older AMIs are automatically deleted and new AMI backups are created as part of the cycle. It is a simple routine, but one that saves you a lot of time and money on maintaining AMI backups.

Further Reading

How to Backup Linux with Snapshots

AWS Amazon Machine Image AWS Lambda Snapshot (computer storage)

Published at DZone with permission of Akhil Teej. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Building Scalable Data Lake Using AWS
  • Building a Scalable ML Pipeline and API in AWS
  • Breaking AWS Lambda: Chaos Engineering for Serverless Devs
  • AWS Step Functions Local: Mocking Services, HTTP Endpoints Limitations

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!