{{announcement.body}}
{{announcement.title}}

How to Monitor Your Alexa Skill

DZone 's Guide to

How to Monitor Your Alexa Skill

In this post, I present a solution for monitoring your Alexa Skills through software debugging and use of AWS Lambda.

· IoT Zone ·
Free Resource

Software debugging is an essential capability for developers, but debugging serverless architectures requires a dedicated approach. The backend of an Alexa Skill is in most cases serverless functions, using, for example, AWS Lambda. Debugging, monitoring and testing are the top challenges when it comes to serverless, with debugging being the most common challenge.

Serverless functions are, by design, very hard to attach anything to. As a result, we’re witnessing new solutions for debugging serverless, both locally and remotely. In this post, I present a solution for monitoring your Alexa Skills.

Monitoring the Insides of Our Skill

Our Skill Backend by nature is event-driven. This makes debugging harder as minor differences in the event content, format or order can make big differences. Simulating those events is potentially a big difference that can make it harder to reproduce and test.

When facing a bug in a Skill, reproducing it reliably can be half the battle. In many data-driven environments, the main factor in actually reproducing the bug is getting the right input data. Migrating data from production to development can easily take hours (or even days) and can be hampered by security, compliance and other considerations. Production debugging tools allow developers to study the bug ‘in the wild’ in production without wasting time and resources on data migration, and without exposing sensitive data.

There are a lot of tools that help us to debug and monitor serverless applications. Making a previous study looking for which one will be better to monitor an Alexa Skill, I've chosen Sentry

Let's go deeper!

Sentry

Sentry is an open-source company, providing an application monitoring platform that helps you identify issues in real-time. Sentry fundamentally is a service that helps you monitor and fix crashes in real-time. The server is in Python, but it contains a full API for sending events from any language, in any application. We are going to focus on Sentry concepts of Context, breadcrumbs, and environments to make proper monitoring of our skills.

Contexts

Sentry supports additional context with events. Often this context is shared among any event captured in its lifecycle, and includes the following components:

Which contexts are we going to use? these:

  • User: the user of the Alexa requests
  • Tags: it allows us to classify and contextualize every request to monitor and track all requests well. We will set up these tags:
    • request_id: the id of the request received.
    • application_id: the id of our skill. It allows us to make a quick search to check all the requests of one skill.
    • session_id: the id of the session of one user using one skill. It will help us to track a single session of one user.
    • person_id: with personalization, your skill can distinguish between individual speakers in the account. So if Alexa recognize the current voice we will receive the User and Person objects which include the person_id property.

Environments

With environments, we can filter events depending on the environment of execution i.e dev/prod and depending as well of the release of our skill backend. You can filter releases by which environment they’ve been deployed to. For example, a release linked to a QA deploy and a Prod deploy will appear in your view when filtering by QA as well as Prod.

Breadcrumbs

Sentry supports a concept called Breadcrumbs, which is a trail of events that happened before an issue. Often these events are very similar to traditional logs, but also have the ability to record more rich structured data. In these breadcrumbs, we will put relevant data of our Alexa Skill such as requests (contexts, session), responses, the overall time of execution, etc.

Sentry in Our Skill

I am not going to start from scratch in this post. I will reuse the project I have used in my post-Alexa Skill, AWS CloudFormation y Serverless Application Model (SAM).

Setting up Sentry in Our Skill

First of all, what we have to add is the Sentry dependency in the pom.xml file:

XML
 




x


1
<dependency>
2
  <groupId>io.sentry</groupId>
3
  <artifactId>sentry</artifactId>
4
  <version>1.7.30</version>
5
</dependency>



After that, we need to initialize the Sentry client. I initialize it at the same time the lambda is called with the method Sentry.init();:

Java
 




x
27


 
1
 public class App extends SkillStreamHandler {
2
    
3
        private static Skill getSkill() {
4
            //SENTRY INITIALIZATION
5
            Sentry.init();
6
            return Skills.standard()
7
                    .addRequestHandlers(
8
                            new CancelandStopIntentHandler(),
9
                            new HelloWorldIntentHandler(),
10
                            new HelpIntentHandler(),
11
                            new LaunchRequestHandler(),
12
                            new SessionEndedRequestHandler(),
13
                            new FallbackIntentHandler(),
14
                            new ErrorHandler())
15
                    .addExceptionHandler(new MyExceptionHandler())
16
                    .addRequestInterceptors(new LogRequestInterceptor())
17
                    .addResponseInterceptors(new LogResponseInterceptor())
18
                    // Add your skill id below
19
                    //.withSkillId("[unique-value-here]")
20
                    .build();
21
        }
22
    
23
        public App() {
24
            super(getSkill());
25
        }
26
    
27
    }



When Sentry.inti() is called it will read the sentry.properties resource file which has the following properties:

  • dsn: the unique URL of your Sentry project. You can get it online in your Sentry project. For example ttp://fasdfasd@sentry.io/asdfas
  • release: to make better monitoring of our skill, it is good to have this property set to filter when something happens.
  • environment: it is also important to set because of knowing the current environment of execution. E.g.: dev, prod.
  • stacktrace.app.packages: the java package you want to monitor. In our case com.xavidop,alexa.helloworld

Sentry in Our Skill Interceptors

After adding the dependency, now we are going to focus on our two interceptors. LogRequestInterceptor and LogResponseInterceptor. Why those two?

  • In LogRequestInterceptor we have the request received which have all the information we need to create our Sentry User and Sentry tags to monitor the current request:
Java
 




xxxxxxxxxx
1
41


 
1
 public class LogRequestInterceptor implements RequestInterceptor {
2
     
3
         static final Logger logger = LogManager.getLogger(LogRequestInterceptor.class);
4
         @Override
5
         public void process(HandlerInput input) {
6
             TimeUtilities.start = new Date().getTime();
7
             HashMap<String, String> data = new HashMap<String,String>();
8
 
          
9
             //SETTING RELEVANT TAGS TO MAKE SEARCHES IN SENTRY
10
             Sentry.getContext().addTag("request_id", input.getRequestEnvelope().getRequest().getRequestId());
11
             Sentry.getContext().addTag("application_id", input.getRequestEnvelope().getSession().getApplication().getApplicationId());
12
             Sentry.getContext().addTag("session_id", input.getRequestEnvelope().getSession().getSessionId());
13
           if(input.getRequestEnvelope().getContext().getSystem().getPerson() != null){
14
                 Sentry.getContext().addTag("person_id", input.getRequestEnvelope().getContext().getSystem().getPerson().getPersonId());
15
             }
16
     
17
            
18
            //SET EXTRA USEFUL DATA FOR THE BREADCRUMB
19
             data.put("Request", input.getRequestEnvelope().getRequest().toString());
20
             data.put("context", input.getRequestEnvelope().getContext().toString());
21
             data.put("Session", input.getRequestEnvelope().getSession().toString());
22
             data.put("Version", input.getRequestEnvelope().getVersion());
23
     
24
 
          
25
             //CREATING THE USER OF THIS REQUEST FOR A FUTURE SEARCHES
26
             HashMap<String, Object> userData = new HashMap<String,Object>();
27
             userData.put("device_id",input.getRequestEnvelope().getContext().getSystem().getDevice().getDeviceId());
28
             Sentry.getContext().setUser(
29
                     new UserBuilder()
30
                             .setData(userData)
31
                             .setUsername(input.getRequestEnvelope().getSession().getUser().getUserId()).build()
32
             );
33
     
34
             Sentry.getContext().recordBreadcrumb(
35
                     new BreadcrumbBuilder()
36
                             .setLevel(Breadcrumb.Level.DEBUG)
37
                             .setTimestamp(new Date())
38
                             .setData(data)
39
                             .setMessage("New request recieved").build()
40
             );
41
     
42
             logger.info(input.getRequest().toString());
43
         }
44
     }



In LogResponseInterceptor we have here the response we are going to send and we can calculate the time in milliseconds od the current execution. As these are the last lines of our skill during the execution, one of the main tasks of this interceptor is to send all the Sentry event to the cloud with the method Sentry.capture(). Finally, we clean everything with Sentry.clearContext() for future Alexa requests:

Java
 




x
31


 
1
public class LogResponseInterceptor implements ResponseInterceptor {
2
 
          
3
    static final Logger logger = LogManager.getLogger(LogRequestInterceptor.class);
4
    @Override
5
    public void process(HandlerInput input, Optional<Response> output) {
6
        TimeUtilities.end = new Date().getTime();
7
        HashMap<String, String> data = new HashMap<String,String>();
8
 
          
9
        //GET THE RESPONSE AND PUT IT AS DATA OF THE BREADCRUMB
10
        data.put("Response", output.get().toString());
11
 
          
12
        //GET THE TIME OF THE EXECUTION
13
        long time = TimeUtilities.end - TimeUtilities.start;
14
        data.put("Overall Time", String.valueOf(time) + "ms.");
15
 
          
16
        //CREATE A BREADCRUMB WITH THE INFO OF THE RESPONSE
17
        Sentry.getContext().recordBreadcrumb(
18
                new BreadcrumbBuilder()
19
                        .setLevel(Breadcrumb.Level.DEBUG)
20
                        .setTimestamp(new Date())
21
                        .setData(data)
22
                        .setMessage("New response").build()
23
        );
24
 
          
25
        Sentry.capture("request - " + Sentry.getContext().getTags().get("request_id"));
26
        //CLEAN CONETXT FOR NEW REQUESTS
27
        Sentry.clearContext();;
28
 
          
29
        logger.info(output.toString());
30
    }
31
}



Logging in Our Skill With Sentry

One of the most important tasks in a serverless architecture is the log. It is essential to know what had happened during an execution. In this example, we will use an auxiliary Class called LogUtilities which have only one method log(String toLog) this method has two tasks:

  1. Log using log$j2 or lambda4j libraries to have structured log files.
  2. Create a Sentry breadcrumb that will be added to the rest of the breadcrumbs and will be available in our Sentry console as part of the event generated by the request. It will help us to track what has happened in a request.

So with this class, you can call it wherever you want to lag and add your breadcrumbs. It will help you to monitor and have good tracking of your skills.

Catching Exceptions in Our Skill With Sentry

The last but not the least, when we are talking about monitoring, are exceptions.

In our Alexa Skill, we have one place to catch all the Exceptions. This MyExceptionHandler is our exception handler. When we have an exception, the LogResponseInterceptor is not going to be executed. This is why here we capture the exception with Sentry and we clean its context as well:

Java
 




x
18


 
1
public class MyExceptionHandler implements ExceptionHandler {
2
    @Override
3
    public boolean canHandle(HandlerInput input, Throwable throwable) {
4
        return throwable instanceof Exception;
5
    }
6
 
          
7
    @Override
8
    public Optional<Response> handle(HandlerInput input, Throwable throwable) {
9
        //CAPTURING THE EXCEPTION
10
        Sentry.capture(throwable);
11
        //CLEANING CONTEXT
12
        Sentry.clearContext();
13
 
          
14
        return input.getResponseBuilder()
15
                .withSpeech("An error was encountered while handling your request. Try again later.")
16
                .build();
17
    }
18
}



The Power of Sentry

Now we have set our Skill with Sentry. Let's send an Alexa request to its backend to see what happened and then we will check the Sentry console:

WOW! We have a new entry!

alexa skill monitoring

Now click on the item and let's see what is inside:

amazon requests

Here you have the information about the event: time of the Alexa request was sent and all the tags. Every Tag is clickable to filter depending on your needs.

If we scroll down, we will see the breadcrumbs. It means a quick view of what happened during that Alexa request:

requests and breadcrumbs

NOTE: The data of the request and response have been removed for this example.

And then if we continue scrolling down we will see the User (Alexa username and Alexa device) information and the information about the Sentry SDK used.

usernames and device names

If we have an exception, we can see in Sentry Dashboard as well:

AskSdkException

We can also see all the execution information clicking on the event:

Event analysis

You can see the full stacktrace clicking on the Raw button:

Raw data

Before concluding this topic, I would like to add that with Sentry you can make searches of any tag, user, environment, release you have.

For example:

  • Give me all the Alexa requests that came from one user and one skill:
    • request_id:amzn1.echo-API.request.[unique-value-here] application_id:amzn1.ask.skill.[unique-value-here]
  • Give me all the Alexa requests that came from one session:
    • session_id:amzn1.echo-API.session.[unique-value-here]
  • Give me all the Alexa requests that came from the current release:
    • release: XXX

You can save these searches as quick ones and it will be available in one click.

Conclusion

With the help of Sentry, we quickly went from having zero knowledge to understanding the error. Using the right tools at the right time can help tremendously with these kinds of issues.

I've made this example in Java but you can use it in other languages that Alexa supports because Sentry is available in a lot of programming languages such as NodeJS, Python, Java, Kotlin, C#, PHP, Ruby, Go, iOS Android, etc.

You can take a look at all the Sentry documentation here.

Regarding the pricing, Sentry has a development plan that comes with 5.000 events per month. In our case, 5.000 Alexa requests from our Skill. You can see plans here.

That’s all, folks! You can find all the code in my GitHub.

I hope it will be useful! If you have any doubts or questions do not hesitate to contact me or put a comment below!

Topics:
alexa, alexa skill, alexa skill development, alexa skills, alexa skills developer, alexa skills development, amazon alexa, amazon web services, aws, iot

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}