Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Using Your Voice and Android to Control IoT Devices

DZone's Guide to

Using Your Voice and Android to Control IoT Devices

Learn how to build an Android app that lets you communicate and issue commands to control IoT devices — in this case, an ESP8266 WeMos and a Neopixel ring.

· IoT Zone
Free Resource

Build an open IoT platform with Red Hat—keep it flexible with open source software.

In this post, we will describe how to use your voice to control IoT devices. In other words, in this article, we will use our voice to send a set of commands to an IoT device. During this post, we will build a voice-activated IoT project. This is an interesting topic because this project uses a different way to interact with IoT devices. Usually, we are used to interacting with a device with a simple user interface exposed by the device or via smartphone app that sends commands to the device.

Project Overview

The idea that stands behind this project is exploring how to use voice commands to control a device like an Arduino or ESP8266. To build this voice-activated project, we will develop an Android app to capture the user's voice and transform it into a set of commands that are sent to the device. The picture below describes the project overview:

Image title

In more detail, this project is made up by two different sub-systems:

  • An Android app

  • An IoT app

The Android app interacts with the user and listens to the voice commands. Next, the app translates the voice commands into commands that the IoT device can understand. In this article, for the IoT device, we will use an ESP8266 WeMos that controls a Neopixel Ring. You can use an Arduino Uno instead of the ESP or an MKR1000.

Developing a Speech Recognition Android App

The first step in this project is developing an Android app that recognizes user speech. Fortunately, Android provides a built-in system that is able to recognize the user's words. The app user interface is very simple. There is only one button that we use to start sending commands:

Image title

The layout is very simple, as shown below:

<android.support.constraint.ConstraintLayout 
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto" 
    xmlns:tools="http://schemas.android.com/tools" 
    android:layout_width="match_parent" 
    android:layout_height="match_parent" 
    tools:context="com.survivingwithandroid.voice.MainActivity" 
    android:id="@+id/mainView" 
    android:background="@drawable/bg_gradient">

   <Button 
        android:layout_width="wrap_content" 
        android:layout_height="wrap_content" 
        app:layout_constraintLeft_toRightOf="parent" 
        app:layout_constraintRight_toLeftOf="parent" 
        android:text="Send command" 
        android:id="@+id/btnCommand" 
        app:layout_constraintBottom_toBottomOf="parent" 
        android:layout_marginBottom="15dp"/>

</android.support.constraint.ConstraintLayout>


The next step is overriding the onCreate method in MainActivity.java:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    btn = (Button) findViewById(R.id.btnCommand);
    btn.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            startVoiceCommand();
        }
    });
}


startVoiceCommand() is the method that handles voice interaction with the user.

In this context, to capture the user's voice, the app uses an intent, delegating all the hard work to the Android OS:

private void startVoiceCommand() {
    Log.d(TAG, "Starting Voice intent...");
    Intent i = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
    i.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
    i.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());
    i.putExtra(RecognizerIntent.EXTRA_PROMPT, "Tell me, I'm ready!");
    try {
       startActivityForResult(i, REQ_SPEECH_RESULT);
    }
    catch (Exception e) {
        Snackbar.make(v, "Speech to text not supported", Snackbar.LENGTH_LONG).show();
    }
}


This method is very simple. It invokes the intent RecognizerIntent.ACTION_RECOGNIZE_SPEECH , providing some configuration parameter, such as the current locale and the message we want to show to the user. When the user clicks on the button, the app shows the dialog waiting for the voice input. Finally, the app starts the intent, waiting for the result:

Image title

The app overrides the method onActivityResult:

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    super.onActivityResult(requestCode, resultCode, data);

    // Check the Request code
    if (requestCode ==  REQ_SPEECH_RESULT) {
        Log.d(TAG, "Request speech result..");
        ArrayList&lt;String&gt; results = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
        String command = results.get(0);
        Log.d(TAG, "Current command ["+command+"]");
        // Now we send commands to the IoT device
    }
}


In the method above, the app extracts the command and invokes the ESP8266 to set the ring LED color. In this example, the Android IoT voice app handles simple commands like Red, Green, Blue, and so on.

Exchanging Data With the ESP8266

Now it's time to implement network communication. By now, we can suppose that the ESP8266 exposes a method that the Android app invokes to set the ring colors. In this context, we can suppose that the ESP8266 exposes a RESTful API. In order to invoke this API, the app uses an HTTP connection. To this purpose, it is necessary to create a new class called IoTConnectionHandler that handles all the network details. The class is shown below:

public class IoTConnectionHandler {
    private static IoTConnectionHandler me;
    private OkHttpClient client;
    private static final String TAG = IoTConnectionHandler.class.getName();

    private static final String IOT_URL = 
          "http://192.168.1.9:8080/ring?param=0";

    private IoTConnectionHandler() {
        client = new OkHttpClient();
    }

    public static IoTConnectionHandler getInstance() {
        if (me == null)
            me = new IoTConnectionHandler();

        return me;
    }

    public void sendData(String data) {
        Request req = new Request.Builder()
                      .url(IOT_URL + data)
                      .build();

        client.newCall(req).enqueue(new Callback() {
            @Override
            public void onFailure(Call call, IOException e) {
                Log.e(TAG, "Connection error", e);
            }

            @Override
            public void onResponse(Call call, Response response) throws IOException {
                Log.i(TAG, "Command sent");
            }
        });
    }
}


It is very simple and uses the OkHTTP library. Notice that the data parameter holds the color hex code retrieved from the voice command.

The next part is implementing the IoT side of the project that receives the color hex code using the API exposed by the device to set the LED color.

Exposing the API, Controlling the Device

In this step of this voice-controlled IoT device, we will develop the code necessary to:

  • Expose an API invoked by the Android app
  • Control a Neopixel RGB ring

Before diving into the IoT project details, it is useful to know how to connect the ESP8266 to the Neopixel RGB ring:

Image title

To simplify the code development, the IoT device uses these two libraries:

The first one is used to control the LED ring while the second library is necessary to expose some functions in the sketch as an API. If you are new to this library, you can read my previous post describing how to expose Arduino functions as an API.

The code below is the sketch:

#include <Adafruit_NeoPixel.h>
#include <ESP8266WiFi.h>
#include <aREST.h>

#define PIN D2
#define NUMS 12
#define SERVER_PORT 8080

// Neopixel rings
Adafruit_NeoPixel pixels =
    Adafruit_NeoPixel(12, PIN, NEO_GRB + NEO_KHZ800);

aREST rest = aREST();

char * ssid = "xxxxx";
char * pwd = "xxx";

// Let us create the server
WiFiServer server(SERVER_PORT);

void setup() {
    Serial.begin(9600);
    pixels.begin();
    pixels.setBrightness(85);

    // Register the function
    rest.function("ring", setColor);
    WiFi.begin(ssid, pwd);
    Serial.println("Connecting to WiFi...");
    while (WiFi.status() != WL_CONNECTED) {
        delay(1000);
        Serial.println("Try again....");
    }

    Serial.println("WiFi connected...");

    // let us start the server
    server.begin();
}

void loop() {

    WiFiClient client = server.available();
    if (!client) {
        return;
    }

    while (!client.available()) {
        delay(1);
    }

    rest.handle(client);
}

int setColor(String color) {
    Serial.println("Hex color [" + color + "]");
    long tmpColor = strtol( & ("#" + color)[1], NULL, 16);

    Serial.println("Int [" + String(tmpColor) + "]");

    int r = tmpColor << 16;
    int g = tmpColor << 8 & 0xFF;
    int b = tmpColor & 0xFF;

    Serial.print("Red [" + String(r) + "]");
    Serial.print("Green [" + String(g) + "]");
    Serial.println("Blue [" + String(b) + "]");

    for (int i = 0; i & lt; 12; i++)
        pixels.setPixelColor(i, pixels.Color(r, g, b));

    pixels.show();

    return 1;
}


In detail, at the beginning, the sketch tries to connect to the Wi-Fi network. For that, you have to provide the Wi-Fi SSID and the password. Once the connection is established, the sketch configures the server and its port. Moreover, it declares the function we want to export as our API.

The last of part is the method used to control the Neopixel ring.

As soon as we send a voice command to the IoT device, the app shows these logs:

Image title

Summary

At the end of this post, you know how to use voice activation to control an IoT device. In more detail, you have explored how to connect and Android app with an ESP8266 and how to use your voice to interact with it.

Download Red Hat’s blueprint for building an open IoT platform—open source from cloud to gateways to devices.

Topics:
iot ,android ,voice command ,iot devices ,tutorial

Published at DZone with permission of Francesco Azzola, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}