Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Writing a Self-Sufficient AWS Lambda Function

DZone's Guide to

Writing a Self-Sufficient AWS Lambda Function

Take a look at how you can write a Lambda function with two different approaches, and how engineers hope Sigma IDE will make a difference.

· Cloud Zone ·
Free Resource

Discover a centralized approach to monitor your virtual infrastructure, on-premise IT environment, and cloud infrastructure – all on a single platform.

If you have already gone through the SLAppForge’s news articles and blogs, you might already know that a bunch of engineers from Sri Lanka are working hard to make the lives of serverless developers a lot easier.

While we are experimenting with various AWS services, we try a lot of crazy things to make Sigma IDE the best friend of fellow serverless developers.

If you have used AWS console to write a lambda function, you might have noticed that they have a built in IDE in the AWS console itself, where you can code interpreted languages in browser and deploy instantly. In the case of NodeJs, this functionality becomes useless when you want to use third party libraries (we have already solved this problem in Sigma though). So the only option left before we had Sigma was writing the code in your local machine and uploading it as a ZIP file with required dependencies (node_modules).

However, following a piece of code can solve this problem to a certain extent.

const { execSync } = require('child_process');
const fs = require("fs");

let read = fs.readFileSync('index.js'); //reading myself
let code = read.toString();

let requireRegex = /=\s*require\s*\(['\"](.+)['\"]\)/g;
let match = requireRegex.exec(code);

while (match != null) {
    let lib = match[1];
    match = requireRegex.exec(code);
    if (lib === 'child_process' || lib === 'fs') {
        continue;
    }

    execSync(`npm install --prefix /tmp --save ${lib}`);
}

let oldRequire = require;
require = function(module) {
    return oldRequire(`/tmp/node_modules/${module}`);
}

/* JUST FORGET EVERYTHING AND START YOUR CODE FROM THIS POINT ONWARDS*/

let reverseString = require('reverse-string');
let mathjs = require('mathjs');

exports.handler = (event, context, callback) => {
    callback(null, reverseString(' = evitavired')+mathjs.derivative('x^2 + x', 'x'));
};


In this code, the first step is reading the code itself as a String and looking for the required dependencies!


Snake eating itself
If you are not aware already, with each lambda function, you get a non persistent space of 512MB at /tmp location of the lambda container.

Next step is, executing npm install on all identified dependencies to generate the required node_modules folder.


Since, /tmp is the only writable folder in Lambda container, we have to use prefix flag as follows.

npm install --prefix /tmp --save


At the same time, you have to set HOME environment variable of your Lambda container to /tmp.

Setting environment variables in Lambda

At the end of downloading all dependencies, we change the reference of require function to one of our own, which appends /tmp/node_modules/ to all dependencies.

Now when you spawn a container for this particular lambda for the first time, it is going to download required node modules to the /tmp/node_modules directory. This is going to take a noticeable amount of time in the first run (cold start time is a problem in lambda anyway, but this approach takes even more time), and from the second time onward (just for this particular container), the execution time will be normal.

First Run (Cold Start) — Billed Duration: 1100 ms



A warm run — Billed Duration: 300 ms
Obviously this is not a production ready solution, but this might pave the path to solve a burning problem in Sigma (hopefully). We will let you know about what we solved, once we add that feature to Sigma. You will definitely love that feature. Stay tuned & hope you enjoyed!

Update (2018/02/24)

Thank you very much Eric for catching this!

Based on Eric's suggestion, we can get rid of the initial code parsing cycle as follows. This code loads dependencies dynamically at require time, still keeping our lambda function self sufficient!!


const { execSync } = require('child_process');

let oldRequire = require;
require = function(module) {
    try{
        oldRequire.resolve(`/tmp/node_modules/${module}`);
    }catch(e){
        execSync(`npm install --prefix /tmp --save ${module}`);
    }
    return oldRequire(`/tmp/node_modules/${module}`);
}

/* JUST FORGET EVERYTHING AND START YOUR CODE FROM THIS POINT ONWARDS*/

let reverseString = require('reverse-string');
let mathjs = require('mathjs');

exports.handler = (event, context, callback) => {
    callback(null, reverseString(' = evitavired')+mathjs.derivative('x^2 + x', 'x'));
};


However, for this particular example, you will hardly notice a difference in cold start time & run times. But that will be noticeable in the following example.


const { execSync } = require('child_process');

let oldRequire = require;
require = function(module) {
    try{
        oldRequire.resolve(`/tmp/node_modules/${module}`);
    }catch(e){
        execSync(`npm install --prefix /tmp --save ${module}`);
    }
    return oldRequire(`/tmp/node_modules/${module}`);
}

/* JUST FORGET EVERYTHING AND START YOUR CODE FROM THIS POINT ONWARDS*/

exports.handler = (event, context, callback) => {
    if(event.task==="reverse"){
        let reverseString = require('reverse-string');
        callback(null, reverseString(' = evitavired'));
    }else{
        let mathjs = require('mathjs');
        callback(null, mathjs.derivative('x^2 + x', 'x').toString());
    }
};


Difference Between Two Approaches

These two approaches behave differently at both startup time & run time.

In the first approach, since function preloads (downloads) all required dependencies, run time experiences a similar environment as if you’ve upload your lambda function as a ZIP file (with node_modules directory).

In the second approach, startup will be faster and run time will take some time (only at the first time, it encounters an unique dependency), since it has to download the dependencies.

Learn how to auto-discover your containers and monitor their performance, capture Docker host and container metrics to allocate host resources, and provision containers.

Topics:
aws ,serverless ,aws lambda ,sigma ,slappforge ,nodejs

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}