Neuroph: Smart Java Apps with Neural Networks (Part 3)

Neural networks can learn to map input onto output data, and are used for tasks like image recognition, automated classification, prediction, and artificially intelligent game characters. In part 3 of this series, the NetBeans team interviewed Zoran Sevarac from Neuroph about adding an image recognition task to a Java application. Part one of the interview discusses what neural networks are and why the NetBeans IDE was the natural choice for the Neuroph development team. In part two of the interview we had a closer look at use cases of neural networks—and what they can do for you!

Neural Networks - the Why and the How

 Why use neural networks for image recognition?

Although there are other techniques for image recognition, neural networks are an interesting solution due to the following properties:

• They can learn to recognizeor classify images.
• They are able to generalize, which means that they can recognize images they've never 'seen' before, and which are similar to the images they already know.
• They are noise resistant, which means that they can recognize damaged or noisy images.

I don't know anything about neural networks, can I use image recognition?

Yes, you can use image recognition, even if you don't know anything about neural networks. We've tried to keep all of the neural network-related stuff under the hood, and just provide the image recognition interface. Basic knowledge about neural networks is helpful for training neural networks, but you'll get the idea how this works while experimenting.

How do I use neural networks for image recognition?

First you transform the training images into a form which can input into neural networks: The Neuroph library provides the FractionRgbData class which transforms a BufferedImage into normalized RGB vectors. Next you train the neural network to learn these image vectors. The type of neural network we use is the Multi-Layer Perceptron with a Back Propagation learning rule.

Picture 1. Feeding image vectors into a neural network

Neural Networks - Use Cases

Can I use Neuroph for face recognition?

Our image recognition approach can learn to distinguish between a fixed set of faces, but it is not an appropriate solution for more advanced face recognition. It won't recognize any variations like different angles, shades, different hairstyle, etc.

Can I use Neuroph to recognize online images or captchas?

Yes, online images can be recognized using a method which accepts image URLs as input arguments. Captchas are not as easy: The main issues are to identify letters and digits, and to remove background noise and distortion. This means before feeding the data into the neural network, some image preprocessing is necessary. This could be an interesting experiment to try.

Can Neuroph be used to search images or recognize parts of an image?

Neuroph's image recognition approach is not intended for this task, probably there are other, more efficient techniques. Brute-force scanning whole image for specific image parts of a given size can be done, if you know exactly what are you searching for and you know where to look.

Neuroph - Features and Requirements

What image formats are supported?

Supported image formats are BMP, JPG, PNG and GIF.

What is the maximum image size?

Theoretically, the maximum is not constrained, but in practice it depends on available memory and time. Training on large images requires more complex neural networks with more neurons and layers; the training requires more time, and it can be tricky to tweak the network and training parameters. If you have issues with a creating network of a certain size, try to increase the maximum heap size.

How fast is Neuroph?

For smaller images the training process only takes a few minutes. For large images and large numbers of images, training can take a few hours. But once we have trained the network, the recognition itself is pretty fast.

Is there a difference between recognizing black/white and color images?

It's the same principle, the only difference is, for black/white images you need smaller networks with less neurons, and the training is faster. Recognizing images in full RGB color requires three times more input neurons than recognizing black/white images.

Neuroph - Usage

How do I use the Neuroph library for image recognition?

There are three steps:

1. Prepare the training images that you want to recognize.
2. Create, train, and save the image recognition network using the GUI tool from the easyNeurons application.
3. Deploy the saved neural network to your application by using our Neuroph Neural Network Java library, neuroph.jar.

What software do I need to use Neuroph Image Recognition?

To use Neuroph Image Recognition you need:

• Java JDK 1.6
• The Neuroph Framework 2.3.1 (download)
  • Including the easyNeurons GUI application for training image recognition networks, and
  • the Java Neural Network library for deploying trained neural networks to your application.

How do I add an image recognition task to my Java application?

This sample code shows you how to make use of a trained image recognition neural network in your application. A step-by-step tutorial how to train image recognition neural network is available here.

import org.neuroph.core.NeuralNetwork;import org.neuroph.contrib.imgrec.ImageRecognitionPlugin;import java.util.HashMap;import java.io.File;import java.io.IOException;public class ImageRecognitionSample {  public static void main(String[] args) {    // load trained neural network saved with easyNeurons    // (specify existing neural network file here)    NeuralNetwork nnet = NeuralNetwork.load("MyImageRecognition.nnet");    // get the image recognition plugin from neural network    // (image recognition interface for neural network)    ImageRecognitionPlugin imageRecognition = (ImageRecognitionPlugin)nnet.getPlugin        (ImageRecognitionPlugin.IMG_REC_PLUGIN_NAME);    try {        // image recognition is done here (specify some existing image file)        HashMap output = imageRecognition.recognizeImage(new File("someImage.jpg"));        System.out.println(output.toString());    } catch(IOException ioe) {        ioe.printStackTrace();    }  }}

As you can see, the following steps are needed to use a neural network in your application:

1. Load a trained neural network.
2. Get the image recognition plugin from the neural network.
3. Call the recognizeImage() method on an image.
4. Evaluate the resulting HashMap output.

The actual recognition step is just one method call. It returns a HashMap with image labels as keys, and their degree of recognition as values. The higher the degree of recognition, the more likely it is the image that you trained it to recognize. The method recognizeImage() can take a File, URL, or BufferedImage as input.

Other important classes from the neuroph.jar library are:

FractionRgbData – takes a BufferedImage as input and creates RGB data used by neural networks.
ImageRecognitionPlugin – provides the image recognition interface for the neural network, so the user doesn't even have to know that there is a neural network behind the scenes.
ImageSampler – provides methods to dump image samples from the screen (very exciting!) and scale images.

Can you share any recommended best practices?

• Scale all images used for training to the same dimensions.
• If color is not important for your task, use black/white, it will train faster.
• Use the same image dimensions for training and recognition.
• If you get out-of-memory exceptions for bigger images, increase the JVM heap size using -Xms and -Xmx options.
• Use the fastest CPU you can get, since training neural network can take a while.

How can I best use Neuroph with an IDE?

The members of our NetBeans User Group are working on a Neuroph plugin for the NetBeans IDE. We plan to make specialized components for image recognition, OCR, classification, prediction, and approximation, available from the NetBeans Palette.

To make this possible, we need to provide a way for users to customize the underlying neural network, to train the neural network with their data. Presently this is done with the easyNeurons GUI, but we plan to provide the easyNeurons training UI directly inside the NetBeans IDE. We will need a new project type for neural networks, training sets, and test sets. It would also be nice to develop a wizard which will guide inexperienced users.

When we put all this together, we see that we're talking about porting the existing easyNeurons application to the NetBeans platform. We are aware that this will require a lot of work! Presently we don't have enough experience with the NetBeans Platform, and there's only a small number of active developers in our team. We have only just started our local NetBeans User Group (six developers with Neuroph experience), but our members will attend the NetBeans Platform Training, so we believe this can be accomplished!

Neuroph – Next Steps

We see that Neuroph is in active development. What are your future plans?

We plan to release the NEAT algorithm support developed by Aidan Morgan as part of the Neuroph framework. NEAT (NeuroEvolution of Augmenting Topologies) uses a genetic algorithm to evolve the neural network topology (the number of layers and neurons) along with weights. This approach can save time that is usually spent deciding on a network topology for a particular problem.

We also plan to provide a specialized time series prediction API.

Additionally we are working on an OCR (Optical Character Recognition) library based on the existing image recognition. We hope to provide an out-of-the-box OCR component and tools. We have just released our first demo applications for OCR and NEAT support, one for hand-written letter recognition, and one for text recognition from an image. Try them now!

Thank you for the interview, Zoran! Learn more at the Neuroph project home.