How Do Serverless Functions Actually Work? Examining AWS Lambda to Understand Serverless

Serverless computing enables developers to assemble programmatic functions into services efficiently, flexibility, cost-effectively, and with easy scalability. But to really get the most benefit from a shift to serverless, it really helps to understand what’s going on under the hood. In this article, I’ll use the AWS Lambda service as a basis for explaining how serverless functions actually work and then how best to use them. I’ll also cover a number of available resources you can use to build out powerful services with serverless. Just don’t forget security.

First, let’s demystify the name: serverless functions do, in fact, run on servers. However, serverless costs are based purely on consumption, and users are spared from considering server-based resource terms such as server size or bandwidth. The machine that runs serverless functions is a highly customized micro VM, which builds a minimal runtime that’s designed exclusively for the function’s specified programming language.

AWS Lambda uses the Firecracker micro VM.

This diagram shows that Firecracker is a KVM-based VM, like other hypervisor systems. Firecracker stores functions in the /var/runtime/bin folder and waits until the function is called.

To understand how the function runs, let’s look at the process for running an example Python function in the microVM:

The process is a bootstrap, which collects the event containing user input, and passes it to the request handler:

True to its name (which is actually the name of the function), therequest_handlerhandles the event request and returns the result.

Taking a look at the content in the runtime folder shows the bootstrap, the library required to run the function, and the bin folder where the function resides:

As this example demonstrates, nothing about invoking a function goes beyond the rudimentary. What does make Lambda functions special, though, is the fact that AWS supplies the infrastructure and supporting components to ensure your function is fully resourced and ready to feature powerful capabilities.

Lambda Function Triggers

AWS makes it possible to invoke Lambda functions by setting up triggers within other services, enabling functions to usefully collect and process data from those services.

Services that invoke Lambda functions synchronously, making other services wait for the function to respond and automatically retrying on errors, include:

• Elastic Load Balancing (Application Load Balancer)
• Amazon Cognito
• Amazon Lex
• Amazon Alexa
• Amazon API Gateway
• Amazon CloudFront (Lambda@Edge)
• Amazon Kinesis Data Firehose
• AWS Step Functions

For services that invoke Lambda functions asynchronously, Lambda queues the event and then passes it to the function. Queuing the event delivers a success response to the other service, and there’s no follow up from that service’s perspective. In case of errors, Lambda performs retries and has a configurable dead-letter queue for failed events. Services that invoke Lambda functions asynchronously include:

• Amazon Simple Storage Service
• Amazon Simple Notification Service
• Amazon Simple Email Service
• AWS CloudFormation
• Amazon CloudWatch Logs
• Amazon CloudWatch Events
• AWS CodeCommit
• AWS Config
• AWS IoT Events

When a trigger within a service invokes a serverless function, the Firecracker micro VM launches – called the bootstrap – and sends input to the event_handler (the function). With that input, a Lambda function can operate on many services to perform useful operations, such as creating, modifying, or deleting an EC2 instance, performing a DynamoDB operation, triggering API calls, and more.

Services that Lambda functions can operate with include:

• AWS CloudFormation
• AWS IoT
• AWS Key Management Service
• AWS Lambda
• AWS Secrets Manager
• AWS X-Ray
• Alexa for Business
• Amazon API Gateway
• Amazon CloudWatch
• Amazon CloudWatch Logs
• Amazon Cognito Identity
• Amazon Cognito Sync
• Amazon DynamoDB
• Amazon EC2
• Amazon EventBridge
• Amazon Kinesis
• Amazon Resource Group Tagging API
• Amazon S3
• Amazon SNS
• Amazon SQS
• Identity and Access Management
• Manage Amazon API Gateway

It’s really important to remember that AWS charges based on the number of serverless function invocations per month (the free tier allows 1 million calls per month). That’s in addition to any separate service charges, such as for S3 storage or databases.

Best Practices for Your Serverless Functions

After a function completes its execution, the next step could be to simply return a response to the user, or you can conduct further processing in more complex scenarios. AWS offers a destination for invoking a follow-up Lambda function when necessary or for posting an event to SQS, SNS, or EventBridge for asynchronous processes. Alternatively, function results can be streamed to DynamoDB, Kinesis, or another data service.

A best practice is using a function to accomplish one single task. A function should never be interrupted in the middle of processing and called off to perform an asynchronous task, such as sending a request to an external resource or calling another Lambda function. Additional tasks such as these should be performed as the next steps and only after the function is completed. In this way, another best practice is using the function’s output to trigger another function or other actions as needed.

AWS Step Functions

Whenever a service is too complex to deliver as a single function, using AWS Step Functions is a valuable technique for placing multiple functions into a workflow. In the example below (from AWS), a call center application uses Step Functions to handle each aspect of a call in a logical process:

In this process, shifting from one Step Function call to another is known as a state transition. AWS charges for Step Functions on the basis of these state transitions, offering a free tier for up to 4000 transitions and $0.025 per 1000 after.

Advanced Topics and Resources

Beyond the basics of creating and using functions, as covered above, AWS provides documentation for numerous topics that detail how to enable functions with more advanced capabilities. Some of these useful topics include:

Using an API Gateway to support canary release deployments:

https://docs.aws.amazon.com/en_pv/apigateway/latest/developerguide/canary-release.html

Using CloudWatch or X-Ray to monitor functions and events:

https://aws.amazon.com/cloudwatch/

https://aws.amazon.com/xray/

Adding user roles to manage and secure access to functions and resources (in-line with the least privilege principle):

https://aws.amazon.com/iam/

Using Cognito to provide user sign-in, sign up, and access controls:

https://aws.amazon.com/cognito/

Building a shared library/code layer and adding it to a function:

https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

Accessing VPC through a function or using SNS/SQS to process results-=:

https://docs.aws.amazon.com/lambda/latest/dg/resource-model.html

Don’t Forget or Neglect Security

You need to utilize security scans and control function permissions to protect your serverless environment. An IDE plug-in for the 'Serverless' IDE framework (a handy open-source tool) can be used to scan serverless functions on AWS Lambda for vulnerabilities. Leverage other security tools and services that can achieve an integrated view of vulnerabilities by correlating pipeline scans with functions stored on AWS Lambda. Also, evaluate the AWS permissions granted to each and every function, and perform risk assessments of functions based on permission analysis. You will need to control any functions with permissions that are too broad to reduce those risks.

Wrapping Up

Serverless is relatively straightforward to get started with but becomes more complicated as functions leverage more advanced resources. Given the cost structure, it’s important for users to pay attention to charges in order to avoid locking in with a vendor before the true costs are understood, which can be a danger of relying on a serverless architecture. There’s a lot to learn when it comes to the many resources AWS has available, and it remains essential to properly configure those resources to minimize costs. But the benefits are ripe for the taking: serverless provides tremendous value in allowing developers to concentrate their focus on the business logic of their functions and developing, iterating, and shipping their applications with more ease.

By Xiao Sun, Principal Engineer at NeuVector.