Inside a Lambda Runtime: A Peek into the Serverless Lair

Ever wondered what it is like inside a lambda? Stop wondering. Let's find out.

Ever since they surfaced in 2014, AWS's lambda functions have made themselves a steaming hot topic, opening up whole new annals in serverless computing. The stateless, zero-maintenance, pay-per-execution goodies are literally changing—if not uprooting—the very roots of the cloud computing paradigm. While other players like Google and Microsoft Azure are entering the game, AWS is the clear winner so far.

Okay, preaching aside, what does it really look like inside a lambda function?

As per AWS folks, lambdas are driven by container technology; to be precise, AWS EC2 Container Service (ECS). Hence, at this point, a lambda is merely a Docker container with limited access from outside. However, the function code that we run inside the container has almost unlimited access to it—except root privileges— including the filesystem, built-in and installed commands and CLI tools, system metadata and stats, logs, and more. Not very useful for a regular lambda author, but could be so if you intend to go knee-deep in OS-level stuff.

Obviously, the easiest way to explore all these OS-level offerings is to have CLI (shell) access to the lambda environment. Unfortunately this is not possible at the moment; nevertheless, combining the insanely simple syntax provided by the NodeJS runtime and the fact that lambdas have a few minutes' keep-alive time, we can easily write a ten-liner lambda that can emulate a shell. Although a real "session" cannot be established in this manner (for example, you cannot run top for a real-time updating view), you can repeatedly run a series of commands as if you are interacting with a user console.

let {exec} = require('child_process');

exports.handle = (event, context, callback) => {
  console.log(event);
  exec(event.cmd, (err, stdout, stderr) => {
    console.log(stdout);
    if (err) console.log(stderr);
    callback(undefined, {statusCode: 200});
  });
}

Lucky for us, since the code is a mere ten-liner with zero external dependencies, we can deploy the whole lambda—including code, configurations and execution role—via a single CloudFormation template:

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  shell:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: shell
      Handler: index.handle
      Runtime: nodejs6.10
      Code:
        ZipFile: >
          let {exec} = require('child_process');

          exports.handle = (event, context, callback) => {
            console.log(event);
            exec(event.cmd, (err, stdout, stderr) => {
              console.log(stdout);
              if (err) console.log(stderr);
              callback(undefined, {statusCode: 200});
            });
          }
      Timeout: 60
      Role:
        Fn::GetAtt:
        - role
        - Arn
  role:
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
      - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
        - Action: sts:AssumeRole
          Effect: Allow
          Principal:
            Service: lambda.amazonaws.com

Deploying the whole thing is as easy as:

aws cloudformation deploy --stack-name shell --template-file /path/to/template.yaml \
  --capabilities CAPABILITY_IAM

or selecting and uploading the template to the CloudFormation dashboard, in case you don't have the AWS CLI to do it the (above) nerdy way.

Once deployed, it's simply a matter of invoking the lambda with a payload containing the desired shell command:

{"cmd":"the command to be executed"}

If you have the AWS CLI, the whole thing becomes way more sexy, when invoked via the following shell snippet:

echo -n "> "
read cmd
while [ "$cmd" != "exit" ]; do
  echo
  aws lambda invoke --function-name shell --payload "{\"cmd\":\"$cmd\"}" \
    --log-type Tail /tmp/shell.log --query LogResult --output text | base64 -d
  echo
  echo -n "> "
  read cmd
done

With this script in place, all you have is to invoke the script; you will be given a fake "shell" where you can execute your long-awaited command, and the lambda will execute it and return the output back to your console right away, dropping you back into the "shell" prompt:

> free

START RequestId: c143847d-12b8-11e8-bae7-1d25ba5302bd Version: $LATEST
2018-02-16T01:28:56.051Zc143847d-12b8-11e8-bae7-1d25ba5302bd{ cmd: 'free' }
2018-02-16T01:28:56.057Zc143847d-12b8-11e8-bae7-1d25ba5302bd             total       used       free     shared    buffers     cached
Mem:       3855608     554604    3301004        200      44864     263008
-/+ buffers/cache:     246732    3608876
Swap:            0          0          0

END RequestId: c143847d-12b8-11e8-bae7-1d25ba5302bd
REPORT RequestId: c143847d-12b8-11e8-bae7-1d25ba5302bdDuration: 6.91 msBilled Duration: 100 ms Memory Size: 128 MBMax Memory Used: 82 MB

>

With this contraption you could learn quite a bit about the habitat and lifestyle of your lambda function. I, for starters, came to know that the container runtime environment comprises Amazon Linux instances, with around 4GB of (possibly shared) memory and several (unusable) disk mounts of considerable size (in addition to the "recommended-for-use" 500MB mount on /tmp):

> df

START RequestId: bb0034fa-12ba-11e8-8390-cb81e1cfae92 Version: $LATEST
2018-02-16T01:43:04.559Zbb0034fa-12ba-11e8-8390-cb81e1cfae92{ cmd: 'df' }
2018-02-16T01:43:04.778Zbb0034fa-12ba-11e8-8390-cb81e1cfae92Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/xvda1      30830568 3228824  27501496  11% /
/dev/loop8        538424     440    526148   1% /tmp
/dev/loop9           128     128         0 100% /var/task

END RequestId: bb0034fa-12ba-11e8-8390-cb81e1cfae92
REPORT RequestId: bb0034fa-12ba-11e8-8390-cb81e1cfae92Duration: 235.44 msBilled Duration: 300 ms Memory Size: 128 MBMax Memory Used: 22 MB

> cat /etc/*-release

START RequestId: 6112efb9-12bd-11e8-9d14-d5c0177bc74f Version: $LATEST
2018-02-16T02:02:02.190Z6112efb9-12bd-11e8-9d14-d5c0177bc74f{ cmd: 'cat /etc/*-release' }
2018-02-16T02:02:02.400Z6112efb9-12bd-11e8-9d14-d5c0177bc74fNAME="Amazon Linux AMI"
VERSION="2017.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2017.03"
PRETTY_NAME="Amazon Linux AMI 2017.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2017.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
Amazon Linux AMI release 2017.03

END RequestId: 6112efb9-12bd-11e8-9d14-d5c0177bc74f
REPORT RequestId: 6112efb9-12bd-11e8-9d14-d5c0177bc74fDuration: 209.82 msBilled Duration: 300 ms Memory Size: 128 MBMax Memory Used: 22 MB

>

True, the output format (which is mostly raw from CloudWatch Logs) could be significantly improved, in addition to dozens of other possible enhancemenrs. So let's discuss, under comments!