Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Implement a Custom Alexa Skill Using Spring Cloud Microservices (Part 2)

DZone's Guide to

Implement a Custom Alexa Skill Using Spring Cloud Microservices (Part 2)

Continue learning how to create and implement your own custom skills for Alex with the help of Spring Cloud and microservices.

· IoT Zone
Free Resource

Cisco IoT makes digital transformation a reality in factories, transportation, and utilities. Learn how to start integrating with Cisco DevNet.

Picking up where we left off, in this section, we will provide more technical details for the Patient Monitor Lambda function and the medical record management service. In particular, we will review important parts of the code. The complete code base can be downloaded from here.

The Patient Monitor Lambda function is based on Alexa Skills Kit version 1.1.2. The medical record management service is based on Spring Cloud Brixton release. For building and running both applications, we used Java version 1.8. For the build and test instructions see the README file in download location.

Review of the Patient Monitor Lambda Function

Stream Handler

The MonitorSpeechletRequestStreamHandler class belongs to the com.demo.alexa.serviceclient.monitor package. This is a simple class that works as an entry point to receive the requests from a user via the Alexa service. It extends com.amazon.speech.speechlet.lambda.SpeechletRequestStreamHandler.

It stores unique identifiers of the custom skills supported by this lambda function in a Set<String> object. In the code below, amzn1.ask.skill.953c1281-e884-4041-b705-7b4445659694 is the unique identifier of Patient Monitor skill automatically created when the skill was configured in the configuration console. The constructor invokes super() by passing the name of the class implementing the com.amazon.speech.speechlet.Speechlet interface (com.demo.alexa.serviceclient.monitor.MonitorSpeechlet, in our case) and the supported application IDs.

public class MonitorSpeechletRequestStreamHandler extends SpeechletRequestStreamHandler {

    private static final Set<String> supportedApplicationIds;

    static {
        supportedApplicationIds = new HashSet<String>();
        supportedApplicationIds.add("amzn1.ask.skill.953c1281-e884-4041-b705-7b4445659694");
    }

    public MonitorSpeechletRequestStreamHandler() {
        super(new MonitorSpeechlet(), supportedApplicationIds);
    }
}


Speechlet

The MonitorSpeechlet class belongs to the com.demo.alexa.serviceclient.monitor package. This class implements com.amazon.speech.speechlet.Speechlet and provides all the logic for the custom skill. It maintains conversations with the user and interfaces with the medical record management service via REST calls.

Due to space limitations, we will only list the Speechlet methods onSessionStartedonLaunchonIntent, and onSessionEnded in this article. Other methods will be discussed without code listing.

We start with various variable declarations and initializations.

public class MonitorSpeechlet implements Speechlet {

    // Logger
    static final Logger log = Logger.getLogger(MonitorSpeechlet.class);

    // Configuration properties - to be initialized in a static block below
    private static final Properties configProperties;

    // Intents
    private static final String RECORD_INTENT = "RecordIsIntent";
    private static final String MRN_INTENT = "MRNIsIntent";
    private static final String PATIENT_CORRECT_INTENT = "PatientCorrectIsIntent";
    private static final String PATIENT_WRONG_INTENT = "PatientWrongIsIntent";
    private static final String TEMPERATURE_INTENT = "TemperatureIsIntent";
    private static final String PULSE_INTENT = "PulseIsIntent";
    private static final String DIASP_INTENT = "DiaspIsIntent";
    private static final String SYSP_INTENT = "SyspIsIntent";
    private static final String BYE_INTENT = "ByeIsIntent";
    private static final String ABNORMAL_VITALS_INTENT = "AbnormalVitalsIsIntent";
    private static final String READ_VITALS_INTENT = "ReadVitalIsIntent";

    // Slots
    private static final String MRN_SLOT = "mrn";
    private static final String TEMPERATURE_SLOT = "temperature";
    private static final String PULSE_SLOT = "pulse";
    private static final String SYSP_SLOT = "sysp";
    private static final String DIASP_SLOT = "diasp";

    // Vital names
    private static final String TEMPERATURE = "temperature";
    private static final String PULSE = "pulse";
    private static final String SYSP = "systolic pressure";
    private static final String DIASP = "diastolic pressure";

    // Keys - those are the attribute names to hold variables in session
    private static final String PATIENT_KEY = "Patient_Key";
    private static final String TEMPERATURE_KEY = "Temperature_Key";
    private static final String PULSE_KEY = "Pulse_Key";
    private static final String SYSP_KEY = "Sysp_Key";
    private static final String DIASP_KEY = "Diasp_Key";
    private static final String EXISTING_TEMPERATURE_KEY = "Existing_Temperature_Key";
    private static final String EXISTING_PULSE_KEY = "Existing_Pulse_Key";
    private static final String EXISTING_SYSP_KEY = "Existing_Sysp_Key";
    private static final String EXISTING_DIASP_KEY = "Existing_Diasp_Key";

    // Configuration file
    private static final String CONFIG_FILE_NAME = "configuration.properties";

    // Configuration file entries below help construct the REST endpoint
    // for the medical record management service
    private static final String SERVICE_ENDPOINT = "serviceEndpoint";
    private static final String CONTEXT_PATH = "contextPath";
    private static final String ID_QRY_PATH = "idQryPath";
    private static final String ABNORMAL_QRY_PATH = "abnormalQryPath";
    private static final String SAVE_VITALS_PATH = "saveVitalPath";

    // Other constants
    private static final String REQ_METHOD = "GET";


In a static initializer block, we load the configuration file.

    static {
        InputStream is = null;
        configProperties = new Properties();
        try{
          is = MonitorSpeechlet.class.getClassLoader().getResourceAsStream(CONFIG_FILE_NAME);
          configProperties.load(is);
        }catch (IOException e) {
        log.error("Cannot load properties: " + e.getLocalizedMessage());
        }finally{
          try {
            is.close();
          }catch (IOException e) {
            log.error("Cannot close stream: " + e.getLocalizedMessage());
          }
        }
    }


Speechlet onSessionStarted Method

com.amazon.speech.speechlet.Session represents a single execution of a Speechlet. It will consist of one or more intents expressed by the user during a conversation. When a session starts, onSessionStarted method will be called on MonitorSpeechlet. A Session object, similar to javax.servlet.http.HttpSession, can be used to store lightweight attributes. The onSessionStarted method should be used for initialization logic.

    public void onSessionStarted(final SessionStartedRequest request, 
                                 final Session session)
            throws SpeechletException {
        log.info("onLaunch requestId: " + request.getRequestId() + 
                 ", sessionId: " +
        session.getSessionId());
        log.info("User: " + session.getUser().getUserId() +
                ", Access Token: " + session.getUser().getAccessToken());
    }


For simplicity, our sample application does not have authentication/authorization features. See this Amazon reference regarding how to add authentication/authorization to a custom skill. The method call Session.getUser().getAccessToken() above illustrates how to obtain the access token associated with the user accessing the custom skill. 

Speechlet onLaunch Method

The onLaunch method will be called if user starts our custom skill by sending a request without a specific intent. (Another way for user to initiate a skill is by directly invoking an intent. See this Amazon reference.) In that method, because user has not yet expressed an intent to be concerned with, we return a generic welcome response via getWelcomeResponse method.

    public SpeechletResponse onLaunch(final LaunchRequest request, 
                                      final Session session)
            throws SpeechletException {
        log.info("onLaunch requestId: " + request.getRequestId() + ", sessionId: " +
                session.getSessionId());
        return getWelcomeResponse();
    }


The getWelcomeResponse, in turn, invokes getSpeechletResponse method with the greeting that will be read back to the user.

 private SpeechletResponse getWelcomeResponse() {
        String speechText =
                "Welcome to the patient monitor. You can record patient vitals, ask about current vitals or " +
                        " query patients with abnormal vitals";
        String repromptText = speechText;
        return getSpeechletResponse(speechText, repromptText, true);
    }


We borrowed the getSpeechletResponse method from the Alexa Skills Kit Samples. There are three parameters to the method:

  • String speechText: Information to be read back to the user.

  • String repromptText: If a skill is expecting a response from the user after the information is read back but no response is received that maps to an inten,t then this text is read back to the user.

  • Boolean isAskResponse: True if and only if the skill is expecting a response from the user after the information is read back. The repromptText is not sent if this variable is false.

home card is a companion application available for Fire OS, Android, iOS, and desktop web browsers, used to describe or enhance the voice interaction. The getSpeechletResponse method starts with generating a SimpleCard, i.e. one that consists only of text, with text version of the message. Then, if isAskResponse is true, a response is returned that expects user to answer with an intent (SpeechletResponse.newAskResponse). If isAskResponse is false, a response is returned that does not expect an answer (SpeechletResponse.newTellResponse).

    private SpeechletResponse getSpeechletResponse(String speechText, String repromptText,
            boolean isAskResponse) {
        SimpleCard card = new SimpleCard();
        card.setTitle("Patient Monitor");
        card.setContent(speechText);

        // Create the plain text output.
        PlainTextOutputSpeech speech = new PlainTextOutputSpeech();
        speech.setText(speechText);

        if (isAskResponse) {
            PlainTextOutputSpeech repromptSpeech = new PlainTextOutputSpeech();
            repromptSpeech.setText(repromptText);
            Reprompt reprompt = new Reprompt();
            reprompt.setOutputSpeech(repromptSpeech);
            return SpeechletResponse.newAskResponse(speech, reprompt, card);
        } else {
            return SpeechletResponse.newTellResponse(speech, card);
        }
    }


Speechlet onSessionEnded Method

When a session ends, the onSessionEnded method can be used to perform any cleanup logic. Below, we just create a log entry as our application does not need any cleanup upon calling of this method.

    public void onSessionEnded(final SessionEndedRequest request, 
                               final Session session)
            throws SpeechletException {
        log.info("onSessionEnded requestId: " + request.getRequestId() 
                 + ", sessionId: " +
                session.getSessionId());
    }


Speechlet onIntent Method

The onIntent method is called upon receiving an intent from user to generate the corresponding response. We have the following intents and corresponding utterances in the sample application. 

Intent

Description

Sample Utterances

RecordIsIntent

User wants to record vitals for a patient.

  • Record patient vitals.

MRNIsIntent

User is supplying the medical record number of a patient.

  • Medical record number is {mrn}.
  • MRN is {mrn}.

PatientCorrectIsIntent

User is confirming that patient read back by skill in response to a medical record number is the correct patient.

  • Patient name is correct.
  • Confirmed.
  • Yes.

PatientWrongIsIntent

User is stating that patient read back by skill in response to a medical record number is not the correct patient.

  • Patient name is wrong.
  • No.
  • Not confirmed.

TemperatureIsIntent

User is entering temperature measurement for the patient.

  • Temperature is {temperature}.

PulseIsIntent

User is entering pulse measurement for the patient.

  • Pulse is {pulse}.

DiaspIsIntent

User is entering diastolic pressure measurement for the patient.

  • Diastolic pressure is {diasp}.

SyspIsIntent

User is entering systolic pressure measurement for the patient.

  • Systolic pressure is {sysp}.

ByeIsIntent

User wants to disconnect.

  • That's all.  
  • Thanks bye.

AbnormalVitalsIsIntent

User wants to learn names of the patients who have vitals in abnormal range.

  • Read me the patients with abnormal vitals.
  • Get me the patients with abnormal vitals.
  • Who are the patients with abnormal vitals?
  • Are there patients with abnormal vitals?

ReadVitalIsIntent

User wants to learn the vitals of a patient.

  • Read me the vitals of patient {mrn}.
  • Read me the vitals of {mrn}.
  • What are the vitals of patient {mrn}?
  • What are the vitals of {mrn}?


Those intents constitute three main conversations between user and the skill. For each conversation an example is given below. (An arrow from user to Alexa indicates an utterance from the user. An arrow from Alexa to user indicates the response.) 

1. Obtaining Patients Who Have Abnormal Vitals

Abnormal vitals.

2. Obtaining Vitals of a Patient

Read vitals.


3. Entering Vitals of a Patient

Enter vitals.


Those conversations can be merged. For example, user can first obtain patients who have abnormal vitals and then enter vitals for a specific patient.

Below is a listing of the onIntent method. IntentRequest.getIntent is going to return a String object exactly matching one of the intents we had configured the skill with. For each of those intents we developed an "intent handler", a method that processes the intent and returns the corresponding response. The onIntent method calls the appropriate handler for the particular intent sent by the user.

    public SpeechletResponse onIntent(final IntentRequest request, 
                                      final Session session)
            throws SpeechletException {
        Intent intent = request.getIntent();
        String intentName = (intent != null) ? intent.getName() : null;

        switch (intentName) {
            case RECORD_INTENT:
                return recordIntentHandler(session);

            case MRN_INTENT:
                return mrnIntentHandler(session, intent);

            case PATIENT_WRONG_INTENT:
                return wrongPatientIntentHandler(session);

            case PATIENT_CORRECT_INTENT:
                return correctPatientIntentHandler();

            case TEMPERATURE_INTENT:
                return vitalIntentHandler(session, intent, 
                            TEMPERATURE_SLOT, TEMPERATURE, TEMPERATURE_KEY);   

            case PULSE_INTENT:
                return vitalIntentHandler(session, intent, 
                               PULSE_SLOT, PULSE, PULSE_KEY);   

            case SYSP_INTENT:
                return vitalIntentHandler(session, intent, 
                               SYSP_SLOT, SYSP, SYSP_KEY);   

            case DIASP_INTENT:
                return vitalIntentHandler(session, intent, 
                               DIASP_SLOT, DIASP, DIASP_KEY);   

            case ABNORMAL_VITALS_INTENT:
                return abnormalVitalsIntentHandler(session);

            case READ_VITALS_INTENT:
                return readVitalIntentHandler(session, intent);

            case BYE_INTENT:
                return byeIntentHandler(session);          

            default:
                throw new SpeechletException("Invalid Intent");
        }
    }


The intent handler methods in our speechlet implementation are discussed in the below table.

Method Description

recordIntentHandler

Called when user wants to record vitals for a patient. It cleans up session from any previous patient information and initiates conversation with the user by asking medical record number of the patient.

mrnIntentHandler


As part of the conversation with user who wants to record a patient vital, this handler is called when user provides the medical record number of the patient. This handler obtains the patient information from medical record management service, and stores the information in session. Then, reads patient name back to user asking confirmation.

wrongPatientIntentHandler


Called if user does not confirm the patient name returned by mrnIntentHandler. In response, it removes the patient information from session and initiates conversation again by asking medical record number of the patient.

correctPatientIntentHandler

Called if user confirms the patient name returned by mrnIntentHandler. In response, it asks patient to enter a vital to be recorded.

vitalIntentHandler

For the particular patient in session, it stores a vital temporarily in session to be saved later on. Because user enters one vital at a time, this handler is called once for each vital. Later on, when user disconnects or indicates another intent, the corresponding intent handler will gather all the vitals from session and save them in medical record management service at once. 

abnormalVitalsIntentHandler


Called when user wants to get a list of patients who have any vitals in abnormal range. Firstly, if there are any vitals to be recorded, it saves them in the medical record management service. Then, it cleans up the session and queries the medical record management service for a list of patients who have any vitals in abnormal range. If list is not empty, reads the patients to user.

readVitalIntentHandler


Called when user wants to learn about the vitals of a specific patient by supplying its medical record number. Firstly, if there are any vitals to be recorded, it saves them in the medical record management service. Then, it cleans up the session and queries the medical record management service for the particular patient. If patient is found, reads back vitals of the patient to user. If patient does not exist, it lets user know that patient cannot be found and requests the medical record number of the patient again.

byeIntentHandler


Called when patient terminates connection with the skill, i.e. disconnects. Firstly, if there are any vitals to be recorded, it saves them in the medical record management service. Then, it cleans up the session. Finally, it returns a bye message to user.


As an example, we review mrnIntentHandler below. The method first obtains the medical record number and invokes getPatientByNumber to get the patient information from medical record management service. If patient cannot be found then it returns a response requesting user to repeat the medical record number. Otherwise, it continues with storing patient's existing vitals in the session. Finally, it returns patient's name to user asking for confirmation.

The getPatientByNumber() method will make a REST call to the medical record management service to obtain patient information. That REST call will be handled by the WebPatientController.byPNumber method, to be discussed soon.

    protected SpeechletResponse mrnIntentHandler(Session session, 
                                                 Intent intent){
        String mrn = intent.getSlot(MRN_SLOT).getValue();
        if(mrn != null){
            Patient patient = getPatientByNumber(mrn);
            log.info("mrn: " + mrn);
            if(patient != null){
                log.info("patient mrn: " + patient.getNumber());
            }

            if(patient == null){
                return repeatRequestForMRN();
            }else{
                session.setAttribute(PATIENT_KEY, 
                                     patient.getNumber().toString());

                Integer existingTemperatureI = patient.getTemperature();
                String existingTemperature = 
                  existingTemperatureI==null?"":existingTemperatureI.toString();
                session.setAttribute(EXISTING_TEMPERATURE_KEY,
                                     existingTemperature);

                Integer existingPulseI = patient.getPulse();
                String existingPulse = 
                  existingPulseI==null?"":existingPulseI.toString();
                session.setAttribute(EXISTING_PULSE_KEY,existingPulse);

                Integer existingSyspI = patient.getSysp();
                String existingSysp = 
                  existingSyspI==null?"":existingSyspI.toString();
                session.setAttribute(EXISTING_SYSP_KEY,existingSysp);

                Integer existingDiaspI = patient.getDiasp();
                String existingDiasp = 
                  existingDiaspI==null?"":existingDiaspI.toString();
                session.setAttribute(EXISTING_DIASP_KEY,existingDiasp);

                String speechText = "Please confirm patient's name is " + 
                  patient.getName();
                String repromptText = speechText;
                return getSpeechletResponse(speechText, repromptText, true);
            }
        }else{
            return repeatRequestForMRN();
        }
    }



Due to space limitations, we do not discuss remaining intent handlers and the helper methods used by intent handlers in this article. Please see the source code and the comments therein. 

That's enough for now. Stay tuned for our next post, which will delve into the medical record management service we need to help bring this to life.

Cisco is a software company. Surprised? Don’t be. Join DevNet to explore APIs, tools, and techniques that developers are using to add collaboration, IoT, security, network priority, and more!

Topics:
iot ,amazon echo ,alexa ,lambda function

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}