{{announcement.body}}
{{announcement.title}}

Alexa Skill With Local DynamoDB

DZone 's Guide to

Alexa Skill With Local DynamoDB

In this article, take a look at how to do database mocking.

· Database Zone ·
Free Resource

Mocking away dependencies — especially external dependencies — is common practice when writing tests or when you are developing locally. Dependency injection typically makes it easy to provide a mock implementation for your dependencies, e.g. a database.

In this article, we will discuss how to mock a DynamoDB.

Mocking a database is a technique that allows you to set the desired database state in your tests to let specific data-sets ready for future test execution. Using this technique, you can focus on getting the test data-sets ready once, and then use it on different test phases regarding the environments by using mocking. This technique is also useful when you are writing your code on your local laptop. In other words, Database Mocking is a simulation of a database either with few records or with an empty database.

Alexa Skills can use DynamoDB to persist data between sessions. DynamoDB is a fully managed NoSQL database offered by Amazon Web Services.

Prerequisites

Here you have the technologies used in this project

  1. Amazon Developer Account — How to get it
  2. AWS Account — Sign up here for free
  3. ASK CLI — Install and configure ASK CLI
  4. AWS CLI — Install and configure AWS CLI
  5. Node.js v10.x
  6. Java Runtime Environment (JRE) version 6.x or newer
  7. Visual Studio Code
  8. npm Package Manager

The Alexa Skills Kit Command Line Interface (ASK CLI) is a tool for you to manage your Alexa skills and related resources, such as AWS Lambda functions. With ASK CLI, you have access to the Skill Management API, which allows you to manage Alexa skills programmatically from the command line. If you want how to create your Skill with the ASK CLI, please follow the first step explained in my Node.js Skill sample. Let's start!

Creating Local DynamoDB

In this project, we are going to use the npm package dynamodb-localhost. This library works as a wrapper for AWS DynamoDB Local, intended for use in DevOps. This library is capable of downloading and installing the DynamoDB Local with a simple set of commands and passing optional attributes defined in DynamoDB Local Documentation.

If you are using Docker, this npm library executes exactly the same commands but in a different way. The Docker image and this library will run the same executable jar file.

We will use this library in order to make it easier to mock the DynamoDB. 

We need to execute these steps in order to run our local DynamoDB.

  • The first thing we need is to install this local DynamoDB using the library. This step will be achieved by running the method dynamodbLocal.install(). This method will download the latest version of the official jar file that you can find above.
  • Once the DynamoDB is installed locally, now we can start it running the method dynamodbLocal.start(options). To run these two steps synchronously we will use the npm package synchronized-promise. The optionsobject has these properties:
    • port: Port to listen on. Default: 8000
    • cors: Enable CORS support (cross-origin resource sharing) for JavaScript. You must provide a comma-separated "allow" list of specific domains. The default setting for cors is an asterisk (*), which allows public access.
    • inMemory: DynamoDB; will run in memory, instead of using a database file. When you stop DynamoDB;, none of the data will be saved. Note that you cannot specify both dbPath and inMemory at once.
    • dbPath: The directory where DynamoDB will write its database file. If you do not specify this option, the file will be written to the current directory. Note that you cannot specify both dbPath and inMemory at once. For the path, current working directory is /node_modules/dynamodb-localhost/dynamob. For example to create /node_modules/dynamodb-localhost/dynamob/ you should specify '/' with a forwardslash at the end.
    • sharedDb: DynamoDB will use a single database file, instead of using separate files for each credential and region. If you specify sharedDb, all DynamoDB clients will interact with the same set of tables regardless of their region and credential configuration.
    • delayTransientStatuses: Causes DynamoDB to introduce delays for certain operations. DynamoDB can perform some tasks almost instantaneously, such as create/update/delete operations on tables and indexes; however, the actual DynamoDB service requires more time for these tasks. Setting this parameter helps DynamoDB simulate the behavior of the Amazon DynamoDB web service more closely. (Currently, this parameter introduces delays only for global secondary indexes that are in either CREATING or DELETING status.)
    • optimizeDbBeforeStartup: Optimizes the underlying database tables before starting up DynamoDB on your computer. You must also specify -dbPath when you use this parameter.
    • heapInitial: A string that sets the initial heap size e.g., heapInitial: '2048m'. This is input to the java -Xms argument
    • heapMax: A string that sets the maximum heap size e.g., heapMax: '1g'. This is input to the java -Xmx argument
  • Once we have the DynamoDB running on http:localhost:8000 we have to create a new DynamoDB client which will connect to this local DynamoDB

So if you want to keep the information between your executions you can set inMemory to false and additionally, you can specify the dbPath where the data will be stored.

This code is located in utilities/utils.js file:

JavaScript
 




x
27


1
  function getLocalDynamoDBClient(options) {
2
 
          
3
        //Javascript Promise used for installing and starting local DynamoDB
4
        const initializeClient = () => {
5
            return new Promise((resolve, reject) => {
6
                dynamodbLocal.install(() => {
7
                    if (!options) reject(new Error('no options passed in!'))
8
                    dynamodbLocal.start(options);
9
                    resolve();
10
                });    
11
            })
12
        };
13
 
          
14
        //install & start synchronously
15
        let syncInitialization = sp(initializeClient)
16
        syncInitialization();
17
 
          
18
        //configuration for creating a DynamoDB client that will connect to the local instance
19
        AWS.config.update({
20
          region: 'local',
21
          endpoint: 'http://localhost:' + options.port,
22
          accessKeyId: 'fake',
23
          secretAccessKey: 'fake',
24
        });
25
    
26
        return new AWS.DynamoDB();
27
  }


Using Local DynamoDB

Now we have our DynamoDB running on our laptop and a client configured ready to connect to it. It is time to set up the Alexa Skill to use this client. Before this, it is important to notice that a very powerful feature of the new Alexa SDK, is the ability to save session data to DynamoDB with one line of code. But in order to activate this feature, you have to tell to the ASK persistence adapter you are going to use and which client will use this adapter. We need to add the npm package ask-sdk-dynamodb-persistence-adapter to create our persistence adapter.

This code is located in utilities/utils.js file:

JavaScript
 




x
17


1
  function getPersistenceAdapter(tableName, createTable, dynamoDBClient) {
2
 
          
3
    let options = {
4
        tableName: tableName,
5
        createTable: createTable,
6
        partitionKeyGenerator: (requestEnvelope) => {
7
          const userId = Alexa.getUserId(requestEnvelope);
8
          return userId.substr(userId.lastIndexOf(".") + 1);
9
        }
10
    }
11
    //if a DynamoDB client is specified, this adapter will use it. e.g. the one that will connect to our local instance
12
    if(dynamoDBClient){
13
        options.dynamoDBClient = dynamoDBClient
14
    }
15
 
          
16
   return new DynamoDbPersistenceAdapter(options);
17
  }



Once we have the local DynamoDB running, the client created, the persistence adapter created and using this client, it is time to set the adapter to our Skill.

This is how our index.js looks like:

JavaScript
 




x
33


1
  var local = process.env.DYNAMODB_LOCAL
2
  let persistenceAdapter;
3
  //depending if we have enabled the local DynamoDB, we create de persistence adapter with or without local client
4
  if(local === 'true'){
5
    let options = { port: 8000 }
6
    let dynamoDBClient = getLocalDynamoDBClient(options); 
7
    persistenceAdapter = getPersistenceAdapter("exampleTable", true, dynamoDBClient);
8
  }else{
9
    persistenceAdapter = getPersistenceAdapter("exampleTable", true);
10
  }
11
 
          
12
  /**
13
   * This handler acts as the entry point for your skill, routing all request and response
14
   * payloads to the handlers above. Make sure any new handlers or interceptors you've
15
   * defined are included below. The order matters - they're processed top to bottom 
16
   * */
17
  exports.handler = Alexa.SkillBuilders.custom()
18
      .addRequestHandlers(
19
          LaunchRequestHandler,
20
          HelloWorldIntentHandler,
21
          HelpIntentHandler,
22
          CancelAndStopIntentHandler,
23
          FallbackIntentHandler,
24
          SessionEndedRequestHandler,
25
          IntentReflectorHandler)
26
      .addErrorHandlers(
27
          ErrorHandler)
28
      .withPersistenceAdapter(persistenceAdapter)
29
      .addRequestInterceptors(
30
          LocalisationRequestInterceptor)
31
      .addResponseInterceptors(
32
          SaveAttributesResponseInterceptor)
33
      .lambda();



Finally, we have an example of persisting the data in our SaveAttributesResponseInterceptor interceptor located in interceptors folder:

JavaScript
 




x
17


1
  const Alexa = require("ask-sdk-core");
2
 
          
3
  module.exports = {
4
    SaveAttributesResponseInterceptor: {
5
      async process(handlerInput, response) {
6
        if (!response) return; 
7
 
          
8
        const { attributesManager, requestEnvelope } = handlerInput;
9
 
          
10
        console.log(
11
          "Saving to persistent storage:" + JSON.stringify(requestEnvelope)
12
        );
13
        attributesManager.setPersistentAttributes(requestEnvelope);
14
        await attributesManager.savePersistentAttributes();
15
      },
16
    },
17
  };



As you can see, the interceptor above is storing in the DynamoDB the incoming request. This is just a silly example used to show you how it works.

Running the DynamoDB Locally With Visual Studio Code

The launch.json file in .vscode folder has the configuration for Visual Studio Code which allow us to run our lambda locally:

JavaScript
 




x
23


 
1
  {
2
    "version": "0.2.0",
3
    "configurations": [
4
          {
5
              "type": "node",
6
              "request": "launch",
7
              "name": "Launch Skill",
8
              "env": {
9
                  "DYNAMODB_LOCAL": "true"
10
              },
11
              // Specify path to the downloaded local adapter(for nodejs) file
12
              "program": "${workspaceRoot}/lambda/custom/local-debugger.js",
13
              "args": [
14
                  // port number on your local host where the alexa requests will be routed to
15
                  "--portNumber", "3001",
16
                  // name of your nodejs main skill file
17
                  "--skillEntryFile", "${workspaceRoot}/lambda/custom/index.js",
18
                  // name of your lambda handler
19
                  "--lambdaHandler", "handler"
20
              ]
21
          }
22
      ]
23
  }



This configuration file will execute the following command:

  node --inspect-brk=28448 lambda\custom\local-debugger.js --portNumber 3001 --skillEntryFile lambda/custom/index.js --lambdaHandler handler


This configuration uses the local-debugger.js file which runs a TCP server listening on http://localhost:3001

For a new incoming skill request a new socket connection is established. From the data received on the socket the request body is extracted, parsed into JSON and passed to the skill invoker's lambda handler. The response from the lambda handler is parsed as a HTTP 200 message format as specified here The response is written onto the socket connection and returned.

After configuring our launch.json file and understanding how the local debugger works, it is time to click on the play button:

After executing it, you can send Alexa POST requests to http://localhost:3001.

NOTE: If you want to start the local DynamoDB you have to set to true the environment variable DYNAMODB_LOCAL in this file.

Debugging and Testing the Skill With Visual Studio Code

Following the steps before, now you can set up breakpoints wherever you want inside all JS files in order to debug your skill.

In my post talking about Node.js Skill you can see how to test your Skill either directly with Alexa Developer Console or locally with Postman.

Checking the Local DynamoDB

When we are running the DynamoDB locally, this local instance we will set up a shell in http://localhosta:8000/shell

In that shell we can execute queries in order to check the content of our local database. These are some example of queries you can do:

  1. Get all the content of our table:
JavaScript
 




x
26


1
  //GET ALL VALUES FROM TABLE
2
 
          
3
  var params = {
4
      TableName: 'exampleTable',
5
 
          
6
      Select: 'ALL_ATTRIBUTES', // optional (ALL_ATTRIBUTES | ALL_PROJECTED_ATTRIBUTES |
7
                                //           SPECIFIC_ATTRIBUTES | COUNT)
8
      ConsistentRead: false, // optional (true | false)
9
      ReturnConsumedCapacity: 'NONE', // optional (NONE | TOTAL | INDEXES)
10
  };
11
 
          
12
 
          
13
  AWS.config.update({
14
    region: "local",
15
    endpoint: "http://localhost:8000",
16
    accessKeyId: "fake",
17
    secretAccessKey: "fake"
18
  });
19
 
          
20
  var dynamodb = new AWS.DynamoDB();
21
 
          
22
 
          
23
  dynamodb.scan(params, function(err, data) {
24
      if (err) ppJson(err); // an error occurred
25
      else ppJson(data); // successful response
26
  });



Then we can show the data of the table:

  1. Get the information of our table:
JavaScript
 




x
18


 
1
  //GET TABLE INFORMATION
2
  var params = {
3
      TableName: 'exampleTable',
4
  };
5
 
          
6
  AWS.config.update({
7
    region: "local",
8
    endpoint: "http://localhost:8000",
9
    accessKeyId: "fake",
10
    secretAccessKey: "fake"
11
  });
12
 
          
13
  var dynamodb = new AWS.DynamoDB();
14
 
          
15
  dynamodb.describeTable(params, function(err, data) {
16
      if (err) ppJson(err); // an error occurred
17
      else ppJson(data); // successful response
18
  });



Now we can show the information of our table:

These queries are using the AWS SDK for JavaScript.

This local DynamoDB is accessible by the AWS CLI as well. Before using the CLI, we need to create a fake profile that will use the region, accessKeyId and secretAccessKey used by our local database and client. So in our ~/.aws/credentials we have to create the fake profile:

Shell
 




x


1
  [fake]
2
  aws_access_key_id=fake
3
  aws_secret_access_key=fake



And in our ~/.aws/config we set the local region for our fake profile:

Shell
 




xxxxxxxxxx
1


1
  [fake]
2
  region=local



After creating it, now we can execute queries using the AWS CLI using our fake profile:

  aws dynamodb list-tables --endpoint-url http://localhost:8000 --region local --profile fake


This command will return a list of tables in our local database:

JSON
 




x


 
1
  {
2
      "TableNames": [
3
          "exampleTable"
4
      ]
5
  }



You can find more information about how to make queries with the AWS CLI here

Extra

Of course, if you do not want to use the npm package dynamodb-localhost, AWS offers to us other ways to run a local instance. These ways are:

  1. Docker image. All info here.
  2. Maven dependency if you have your skill in java using Maven or Gradle. All info here

At the end, those solutions will run exactly the same as npm package dynamodb-localhost but in some different ways. Choose the one that fits you better!

Conclusion

This was a basic tutorial to mock a DynamoDB with our Alexa Skills using Node.js. With this technique, you can easily make changes to your unit test data and run experiments. This is will make your tests more believable with "real" data and a "real" environment.

How many of you have had an issue with production where it works on staging, but doesn't work on production and the source code is the same in both environments?

This is one example of being able to take/save data from/to a local DynamoDB (with the query above) run it to find out issues, how it works, etc.

In the long run, this makes your unit tests even more valuable to you.

You can find the code in my GitHub.

I hope it will be useful! If you have any doubts or questions, do not hesitate to contact me or put a comment below.

Happy coding!

Topics:
alexa, alexa skill development, alexa skills, alexa skills development, aws, database, dynamodb, node, npm, tutorial

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}