DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Building Scalable Real-Time Apps with AstraDB and Vaadin
Register Now

Trending

  • How to LINQ Between Java and SQL With JPAStreamer
  • Extending Java APIs: Add Missing Features Without the Hassle
  • Auditing Tools for Kubernetes
  • The SPACE Framework for Developer Productivity

Trending

  • How to LINQ Between Java and SQL With JPAStreamer
  • Extending Java APIs: Add Missing Features Without the Hassle
  • Auditing Tools for Kubernetes
  • The SPACE Framework for Developer Productivity
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Deployment
  4. Demystifying AI/ML Microservice with TensorFlow

Demystifying AI/ML Microservice with TensorFlow

Muhammad Afzal user avatar by
Muhammad Afzal
·
Sep. 30, 20 · Tutorial
Like (8)
Save
Tweet
Share
2.03K Views

Join the DZone community and get the full member experience.

Join For Free

Overview

This tutorial will walk you through how to build and deploy a sample microservice application using Red Hat OpenShift Container Platform. The scope of this service is to perform prediction of handwritten digit from 0 to 9 by accepting image pixel array as a parameter. This sample application also provides a HTML file that offer you a canvas to draw your number and convert it into an image pixel array. The intention of this article is to give you a high level idea, the same concept can be taken to the next level for addressing many other complex use cases.

Image recognition is one of the major capabilities of deep learning. In this article, we will be identifying our own hand written digit. However, in order to accurately predict what digit is it, learning has to be performed, so that it understands the different characteristics of each digit as well as the subtle variations in writing the same digit. Thus, we need to train the model with a dataset of labelled handwritten digit. This is where MNIST dataset comes handy.

MNIST dataset


MNIST dataset is comprised of 60,000 small 28x28 square pixel gray scale images and 10,000 test images. These are handwritten single digit images from 0 to 9. Instead of downloading it manually, we can download it using Tensorflow Keras API

We will be performing following steps in this exercise

  1. Build and train the model using MNIST dataset
  2. Saving the learned model in persistent storage
  3. Build and launch microservice for prediction using the provided dockerfile
  4. Generate image pixel array of your handwritten digit using the provided HTML file
  5. Perform prediction using microservice by providing image pixel array as a parameter to this service.

You can find the dockerfile and python code by clicking the following git repository.

https://github.com/mafzal786/tensorflow-microservice.git

Model Training

Below is the python code that performs building, compiling, training, and saving the model based on MNIST dataset. This code can also be executed in Jupyter Notebook. It is important to save this model in a persistent storage once the model training is successfully completed so that it can be used by the microservice application for prediction. Also make sure that saved model file should be accessible by the container running the microservice.

Python
x
64
 
1
import tensorflow as tf
2
print('Tensorflow version:' + tf.__version__)
3
4
import numpy as np
5
from tensorflow.keras.datasets import mnist
6
7
(X_train, y_train), (X_test, y_test) = mnist.load_data()
8
input_shape = [28, 28, 1]
9
10
X_train = tf.reshape(X_train, [X_train.shape[0]] + input_shape)
11
X_test = tf.reshape(X_test, [X_test.shape[0]] + input_shape)
12
13
X_train = tf.cast(X_train, dtype=tf.float32)
14
X_test = tf.cast(X_test, dtype=tf.float32)
15
16
X_train /= 255
17
X_test /= 255
18
19
print(X_train.shape)
20
print(X_test.shape)
21
22
from sklearn.preprocessing import OneHotEncoder
23
24
# Reshape so labels can be onehot encoded.
25
y_train = tf.reshape(y_train, [-1, 1])
26
y_test = tf.reshape(y_test, [-1, 1])
27
28
# Set sparse to False in order to return an array.
29
encoder = OneHotEncoder(sparse=False)
30
31
y_train = tf.convert_to_tensor(encoder.fit_transform(y_train))
32
y_test = tf.convert_to_tensor(encoder.fit_transform(y_test))
33
34
model = tf.keras.Sequential([
35
  tf.keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=input_shape),
36
  tf.keras.layers.Conv2D(64, (3, 3), activation='relu'),
37
  
38
  tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
39
  tf.keras.layers.Dropout(0.25) ,
40
    
41
  tf.keras.layers.Flatten(),
42
  tf.keras.layers.Dense(128, activation='relu'), 
43
  tf.keras.layers.Dropout(0.5) ,
44
  tf.keras.layers.Dense(10, activation='softmax')
45
])
46
47
48
epoch=10
49
batchsize=32
50
51
loss_fn = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
52
optimizer = tf.keras.optimizers.Adam()
53
54
model.compile(optimizer=optimizer, loss=loss_fn, metrics=['accuracy'])
55
56
# Train our model!
57
h = model.fit(x=X_train, y=y_train, epochs=epoch, validation_data=(X_test, y_test), batch_size=batchsize)
58
59
score = model.evaluate(X_test, y_test, verbose=0)
60
print('Test loss:', score[0])
61
print('Test accuracy:', score[1])
62
63
# save your model
64
model.save('model.h5')


Prediction Microservice With Flask

Following instructions will help you with building the microservice using Red Hat OpenShift Container Platform. For the sake of simplicity, it’s a single source file and written in python. This source file is copied inside the container image via Dockerfile during the container image build process, which will be discussed later in this article. The sole purposes of this microservice to is to predict the handwritten digit using the already learned model. It simply takes the image pixel array as a parameter and predict it.

Python
xxxxxxxxxx
1
30
 
1
import os
2
import numpy as np
3
from flask import Flask
4
from flask import request
5
from tensorflow.keras.models import load_model
6
7
app = Flask(__name__)
8
9
@app.route("/")
10
def index():
11
    var_html = "<h1>Tensorflow handwritten digit prediction microservice. </h1><br> Usage: <url>/predict?pixelarray=0,0,0,0,0,0,0.03137254901960784,0.5764705882352941,1,1,1,1,1,1,0.17254901960784313,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.011764705882352941,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.984313725490196,0.07450980392156863,0,0,0,0,0,0,0,0,0,0,1,1,1,0.6196078431372549,0,0,0,0,0,0,0,0,0,0,0,0.8980392156862745,1,1,1,1,0,0,0,0,0,0,0,0,0.48627450980392156,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0.8470588235294118,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0784313725490196,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.07058823529411765,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.7098039215686275,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.7568627450980392,1,0.7843137254901961,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0.3176470588235294,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.4470588235294118,1,1,1,0,0,0,0,0,0,0,0,0,0.8980392156862745,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.03137254901960784,0,0,0,0,0,0,0,0,0,0,0,0,0,0.06274509803921569,0.07058823529411765,0.07058823529411765,0.803921568627451,1,1,0.8941176470588236,0.07058823529411765,0.06666666666666667,0,0,0,0.0392156862745098,1,1,1,0.5098039215686274,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.9254901960784314,1,0.6901960784313725,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5254901960784314,1,0.07450980392156863,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0.9254901960784314,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.3803921568627451,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0.24705882352941178,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.34509803921568627,1,0,0,0,0,0,0.9803921568627451,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0.3686274509803922,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0.9882352941176471,1,1,1,1,0.6784313725490196,0.08235294117647059,0,0,0,0,0,0.4235294117647059,1,1,1,1,1,1,1,0.3333333333333333,0,0,0,0,0,0,0,0,0,0.10588235294117647,0.8117647058823529,1,1,1,1,1,1,1,1,1,1,1,1,0.21568627450980393,0.12941176470588237,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
12
    return var_html
13
14
@app.route("/predict")
15
def predict():
16
    pixelarray = request.args.get('pixelarray')
17
    
18
    arr_digit=pixelarray.split(',')
19
    arr32_digit = np.array(arr_digit,dtype='float32')
20
21
    model = load_model("model.h5")
22
    predict_img = arr32_digit.reshape(1,28,28,1)
23
    prediction = model.predict(predict_img)
24
    response = "<h1>Your handwritten digit is: {}</h1>".format(prediction.argmax())
25
    return response
26
27
28
if __name__ == "__main__":
29
    port = int(os.environ.get("PORT", 8080))
30
    app.run(host='0.0.0.0', port=port)


Dockerfile

Below is the dockerfile that will be used to initiate the image build and launch the container in OpenShift platform. requirements.txt file contains the pre-requisite packages for this microservice to work. requirements.txt file contains the following.

Plain Text
 




xxxxxxxxxx
1


 
1
Flask==0.12.1
2
tensorflow==2.3.0
3
scikit-learn==0.22.1



Dockerfile
xxxxxxxxxx
1
27
 
1
# ARG ubuntu_version=16.04
2
#FROM ubuntu:${ubuntu_version}
3
#Use ubuntu 16:04 as your base image
4
FROM ubuntu:16.04
5
#Any label to recognise this image.
6
7
#install the below packages on the ubuntu image
8
RUN apt-get update -y && apt-get install -y gnupg2 wget openjdk-8-jre python3-pip python3-dev \
9
&& cd /usr/local/bin \
10
&& ln -s /usr/bin/python3 python \
11
&& pip3 install --upgrade pip setuptools
12
13
COPY ./requirements.txt app/requirements.txt
14
15
WORKDIR /app
16
# install flask, tensorflow, scikit-learn, pyspark
17
RUN pip3 install -r requirements.txt
18
19
EXPOSE 8080
20
21
COPY . /app
22
23
# if execute the trianing inside this same container 
24
# RUN python training.py
25
26
ENTRYPOINT [ "python3" ]
27
CMD [ "app.py" ]

Launch the Microservice

Now login to your Red Hat OpenShift cluster. Click Developer and then click Add to create an application. Click “From Dockerfile”. This will import your dockerfile from your git repo and initiate the image build and deploy the container.

Red Hat OpenShift


Supply the Git Repo URL. The dockerfile for this project is located at https://github.com/mafzal786/tensorflow-microservice.git. Also give name to your application. Click create.

Supplying Git repo URL

This will create the build config and build will start. To view the logs for your build, click “Builds” under Administrator tab and click the build as shown below. Once the build is completed, container image is pushed to your configured image registry.

Checking logs


Log output

After the build is completed and image is pushed to the registry, OpenShift will launch the container.

OpenShift launching container

OpenShift will create a route for this service to be exposed by giving an externally reachable hostname. Service end point will be available in Networking →Routes tab. Clicking on the Location as shown below will launch the microservice in the browser.

Routes in OpenShift

The below figure shows when the microservice is launched in the browser.

Microservice launched in browser

Or run the following CLI as below

Shell
xxxxxxxxxx
1
 
1
oc get routes/ms-handwritten-digit-prediction-git


Output from CLI

Shell
xxxxxxxxxx
1
 
1
curl http://`oc get routes/ms-handwritten-digit-prediction-git — template=’{{.spec.host}}’`


Canvas for Writing Digits

Below html file offers you a canvas to draw digit. Copy below code and save it as html on your computer and then launch it in the browser. Click “Convert” button to convert the image into image pixel array. Then click “Copy Image Pixel Array in Clipboard” button to copy the pixel array in your clipboard for providing it to the microservice as a parameter for prediction.

HTML
xxxxxxxxxx
1
33
 
1
<!DOCTYPE html>
2
<html>
3
  <head>
4
   
5
6
</head>
7
<body onload="prepareSimpleCanvas();">
8
<div class="row" style="padding-bottom: 5px">
9
        <div class="col-md-1"></div>
10
        <div  class="col-md-2">
11
        <label for="comment">Draw you Number:</label><br>
12
            <div class="form-group" id="canvasSimpleDiv"></div>
13
14
15
            <div>
16
                <button class="btn btn-default btn-xs" id="clearCanvas">Clear</button>
17
                <button class="btn btn-default btn-xs btn-primary" id="convertToArray">Convert</button>
18
            </div>
19
        </div>
20
        <div class="col-md-8">
21
            <div class="form-group"><br>
22
                <label for="comment">Digit Image Pixel Array:</label><br>
23
                <textarea readonly id="array" class="form-control" rows="5" style="width:500px;"></textarea>
24
                <br>
25
            <button class="btn btn-default btn-xs" onclick="copyToClipboard('array')">Copy Image Pixel Array in Clipboard</button>
26
            </div>
27
        </div>
28
        <div class="col-md-1"></div>
29
    </div>
30
    <code>Note: If you cannot draw anything on canvas, make change in the code (for example, add space) and rerun it!</code>
31
32
</body>
33
</html>


Draw Your Number

Drawing number on canvas

Perform Prediction Using the Microservice

Now pass this image pixel array to the microservice as a parameter as show below.

Plain Text
 




xxxxxxxxxx
1


 
1
# curl http://`oc get routes/ms-handwritten-digit-prediction-git — template=’{{.spec.host}}’`/predict?pixelarray=0,0,0,0,0,0,0.03137254901960784,0.5764705882352941,1,1,1,1,1,1,0.17254901960784313,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.011764705882352941,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.984313725490196,0.07450980392156863,0,0,0,0,0,0,0,0,0,0,1,1,1,0.6196078431372549,0,0,0,0,0,0,0,0,0,0,0,0.8980392156862745,1,1,1,1,0,0,0,0,0,0,0,0,0.48627450980392156,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0.8470588235294118,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.0784313725490196,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.07058823529411765,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.7098039215686275,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.7568627450980392,1,0.7843137254901961,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0.3176470588235294,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.4470588235294118,1,1,1,0,0,0,0,0,0,0,0,0,0.8980392156862745,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.03137254901960784,0,0,0,0,0,0,0,0,0,0,0,0,0,0.06274509803921569,0.07058823529411765,0.07058823529411765,0.803921568627451,1,1,0.8941176470588236,0.07058823529411765,0.06666666666666667,0,0,0,0.0392156862745098,1,1,1,0.5098039215686274,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.9254901960784314,1,0.6901960784313725,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.5254901960784314,1,0.07450980392156863,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0.9254901960784314,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.3803921568627451,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0.24705882352941178,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.34509803921568627,1,0,0,0,0,0,0.9803921568627451,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0.3686274509803922,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0.9882352941176471,1,1,1,1,0.6784313725490196,0.08235294117647059,0,0,0,0,0,0.4235294117647059,1,1,1,1,1,1,1,0.3333333333333333,0,0,0,0,0,0,0,0,0,0.10588235294117647,0.8117647058823529,1,1,1,1,1,1,1,1,1,1,1,1,0.21568627450980393,0.12941176470588237,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
2
<h1>Your handwritten digit is: 3</h1>



You can run the same in the browser as follows.

Prediction from microservice


microservice Build (game engine) TensorFlow Container Data structure application Plain text OpenShift

Published at DZone with permission of Muhammad Afzal. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • How to LINQ Between Java and SQL With JPAStreamer
  • Extending Java APIs: Add Missing Features Without the Hassle
  • Auditing Tools for Kubernetes
  • The SPACE Framework for Developer Productivity

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: