Sensor Management in Android, Mr Brown Can Moo

The Android development kit provides great emulators to allow you to test your applications as it would be emulated on a device. There's a catch, not all features are available in the emulator as they are in the actual device. This becomes clearly apparent when you start to develop applications for possible the camera, or even accelerator, compass, or orientation sensors. You can't just pick up your computer and tilt it and move it around, we know that doesn't work. This article will explain how to overcome these shortcomings of the emulated environment.

Some may have seen the iPhone application called “Can Moo”. This is the veritable “Cow in a can” toy, that us kids grew up with, which if you up-end the can; a cow's moo can be heard. This application that will demonstrate how to re-create this toy on the Android, is a spin off with a sprinkling of Dr. Seusse, called “Mr. Brown in a Can”. “Mr. Brown Can Moo, Can You” is the whimsical play that teaches kids various sounds animals make.

The requirements of this application are sound to play the cow moo. The ability to sense when the phone's sensor has been tipped beyond a certain point to trigger that sound. The last requirement for the emulated android environment, which will leverage OpenIntents. This is a googlecode project that allows managing sensor data via http. It can be found at http://code.google.com/p/openintents/ Download the sensor simulator 1.0 beta http://code.google.com/p/openintents/downloads/detail?name=sensorsimulator-1.0.0-beta1.zip&can=2&q= , and unzip this into a folder.

It will also be assumed that the development environment is Eclipse with the Android SDK and plugins already installed and configured.

Start this project by selecting New → Android Project. The project name “MrBrown”. Select Android 1.6, you can select others, but make sure you also set up the appropriate emulator for your project version. The application name chosen was “MrBrownCanMoo”. Then select a package name of your choosing. Finally for the Activity type in “MooCan” and click Finish. This will create a simple “HelloWorld” starter application.

If you haven't created an emulator for your project version, create one using the “Android SDK and AVD Manager”. Using the Eclipse plug-in this is the smart phone icon in the tool bar. Once it is created launch the emulator by selecting the virtual device and clicking start.

Once, the emulator is started, go to the location where the sensor simulator 1.0 beta file was extracted, and in the bin/ folder is a file called “SensorSimulatorSettings.apk”. This needs to be installed into the virtual device.

To do this, simply use the command: %ANDROID_SDK_HOME%/tools/adb install SensorSimulatorSettings.apk

The result should be a success message in the command prompt. If not, it may be because you unlocked the screen on the emulator.

Go to the application screen of the emulator and you should see a “Sensor Simulator” application, and select it. The next step involves configuring the sensor simulator that looks like a Star Trek phaser.

From the same director that you typed in the AVD command to install the SensorSimulatorSettings.apk file is a sensorsimulator.jar. Type the following command: java -jar sensorsimulator.jar

A swing application will launch, and start listening to a port socket. This can be changed as needed for your environment.

Go back to the android emulator and click the Sensor Simulator application if you haven't done so already. This opens up a Settings screen. Enter the IP address that shows in the Swing application, and the port number, the default being port 8010. You can't use localhost, or 127.0.0.1, it should be the machine's ip address. This is because the emulator has it's own internal loopback.

We need to test that the simulator swing application and the Sensor Simulator in the Android emulator are working. Click the testing tab, and click on the “Connect” button. If everything checks out you should have orientation, accelerometer, and magnetic field check boxes. Click on Orientation. We can ignore the update rate for now. This is used more for game development.

Go back to the Sensor Simulator swing application and you can either click and drag the 3d image of the android device or use the yaw, pitch, roll sliders next to it. Either case as you move the settings in the Sensor Simulator swing application you should see the numbers changing in the android sensor simulator application, as well.

You can now shut down the emulator. This will be how the application we will develop will be able to “sense” movement in an emulated environment.

The application is going to be pretty straight forward. The only difference is when it is ready to deploy in an actual device a couple of lines of code will have to be switched out to make sure we are using the actual device sensor and not and emulated one.

The first step is to allow the application we are developing use permissions to the Internet. This is again because we are using a port to talk to the Sensor Simulator swing application. Therefore, add this line to the AndroidManifest.xml inside the <manifest> tags.

<uses-permission android:name="android.permission.INTERNET"></uses-permission>

The following image resources are added to the res/drawable/ folder.

Let's change the default main layout that has just text, in res/layout/. We will remove the TextView object and add an ImageView object referencing the “can”.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    >
<ImageView android:id="@+id/ImageView01" android:layout_width="wrap_content" android:layout_height="wrap_content" android:src="@drawable/mrbrowncan"></ImageView>
</LinearLayout>

   This application will use the android api SensorManager ultimately, but for testing it we will have to use the SensorSimulator. We will also have to import reference to the sensorsimulator-lib.jar in our Eclipse project, which is located in the lib/ directory of the sensor simulator 1.0 beta folder. Declare both a SensorManager and SensorManagerSimulator with the same variable name and comment the SensorManager out, as such:

//SensorManager sm = null;
SensorManagerSimulator sm = null;

  Notice, that the SensorManagerSimulator also needs the sm.connectSimulator(). The actual SensorManager does not need this call. When it's time to release to an actual device you will uncomment the SensorManager line, and comment out the two lines in SensorManagerSimulator.

     // get reference to SensorManager
     //sm = (SensorManager) getSystemService(SENSOR_SERVICE);		 
     sm = SensorManagerSimulator.getSystemService(this, SENSOR_SERVICE);
     sm.connectSimulator();

To listen for changes in the SensorManger we have to implement SensorListener, and then register a listener with the manager. There are two methods that can be overriden in the android Activity object. The onResume(), and onStop() method. These are both part of the Activity lifecycle of an android application. In onResume(), we will register the sensor listener, and subsequently onStop() unregister the listener. This looks like this:

        @Override
	protected void onResume() {
		super.onResume();
		// register this class as a listener for the orientation and
		// accelerometer sensors
		sm.registerListener(this, SensorManager.SENSOR_ORIENTATION
				| SensorManager.SENSOR_ACCELEROMETER,
				SensorManager.SENSOR_DELAY_NORMAL);
	}

	@Override
	protected void onStop() {
		// unregister listener
		sm.unregisterListener(this);
		super.onStop();
	}

The real workings of the application will happen during the SensorListening events in onSensorChange call. We will work on the pitch axis since it makes more sense that you tip the top of the phone down and back up, as it were a “cow in the can” toy. Roll and yaw for such a toy does not trigger the cow sound.

In addition, we need to load a media player so that when the sensor reaches beyond a certain pitch the player plays the sound of the cow. A wav file supplies us the resource for the sound. This will be put in the res/raw/ folder. Creating a media player is as simple as follows:

mp = MediaPlayer.create(this, R.raw.cow_toy);

  The values are an array of floats. The element [1] in the array represents our pitch. Element [0] is yaw, and element [2] is roll, for the orientation sensor.

Once the value of the pitch is beyond 25 degrees or less than 91 degree, that indicates beyond the level plane of 0 then the media player starts playing. This would indicate a transition to a vertical position. The same holds true for -20 degrees as a downward position of pitch. Again the media player is created and plays the cow sound.

The full code should look as follows:

import org.openintents.sensorsimulator.hardware.SensorManagerSimulator;

import android.app.Activity;
import android.hardware.SensorListener;
import android.hardware.SensorManager;

import android.media.MediaPlayer;
import android.os.Bundle;
import android.util.Log;

public class MooCan extends Activity implements SensorListener {
	//SensorManager sm = null;
	SensorManagerSimulator sm = null;
	MediaPlayer mp;

	boolean flip = false;

	/** Called when the activity is first created. */
	@Override
	public void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);
		setContentView(R.layout.main);
		// get reference to SensorManager
		 //sm = (SensorManager) getSystemService(SENSOR_SERVICE);		 
		sm = SensorManagerSimulator.getSystemService(this, SENSOR_SERVICE);
		sm.connectSimulator();		
		mp = MediaPlayer.create(this, R.raw.cow_toy);
	}

	public void onSensorChanged(int sensor, float[] values) {
		synchronized (this) {
			if (sensor == SensorManager.SENSOR_ORIENTATION) {
				if (values[1] > 25 && values[1] < 91) {
					if (!mp.isPlaying() && flip == false) {
						try {
							mp = MediaPlayer.create(this, R.raw.cow_toy);
							mp.start();	
							flip = true;
						} catch (IllegalStateException e) {
							// TODO Auto-generated catch block
							e.printStackTrace();
						} 
					} 
				} else if (values[1] < -20) {
					if (!mp.isPlaying() && flip == true) {
						mp = MediaPlayer.create(this, R.raw.cow_toy);
						mp.start();	
						flip = false;			
					}
				}
			}
		}
	}

	public void onAccuracyChanged(int sensor, int accuracy) {
	}

	@Override
	protected void onResume() {
		super.onResume();
		// register this class as a listener for the orientation and
		// accelerometer sensors
		sm.registerListener(this, SensorManager.SENSOR_ORIENTATION
				| SensorManager.SENSOR_ACCELEROMETER,
				SensorManager.SENSOR_DELAY_NORMAL);
	}

	@Override
	protected void onStop() {
		// unregister listener
		sm.unregisterListener(this);
		super.onStop();
	}
}

At this point, you should have the Sensor Simulator swing application started: java -jar sensorsimulator.jar . Then run the application from your Eclipse IDE. Either slide the pitch slider on the swing application, or rotate the wireframe image of the device within the Sensor Simulator swing application. At a certain point the sound of a cow should trigger.

To deploy to an actual android device, you simply have to uncomment the line for the instantiation of the SensorManager and comment out the lines to the SensorManagerSimulator, and it's connectSimulator line. The rest of the code base remains the same.

This is a very basic application just to demonstrate how one can test and build an android application that makes use of the sensor orientation, and accelerometer through the emulator. Just as other parts of physical devices in the android emulator are not readily available this is one solution to allow you to test and develop towards an end product.