Deep Learning with Spring Boot and DJL

In this tutorial we review how to create a sample Deep Learning Java app using Spring Boot, DJL and Tensorflow.

David Kiss

Jun. 03, 20 · Tutorial

Like (1)

Save

12.32K Views

Overview

This is another post on Spring Boot that will show how to build a sample web application using Deep Java Library (DJL), an open-source Deep Learning library for Java to diagnose COVID-19 on X-ray images.

The sample app is a Spring Boot based version of DJL's similar COVID-19 example and it has a simple static HTML page built using Twitter Bootstrap and JQuery where users can submit an image URL to a REST api where the DJL library will download the image and predict if it's an X-ray image of lungs infected with COVID-19 or not.

The link to the source code is included at the end of this post.

Disclaimer: This is only a demo application based on the dataset at https://github.com/ieee8023/covid-chestxray-dataset and it SHOULD NOT be used for actual medical diagnosis.

Deep Java Library

As mentioned earlier, DJL is a Java-based library that supports multiple
Deep Learning frameworks like Apache MxNet, PyTorch and Tensorflow. Since most Deep Learning engines are built using Python and not in Java, DJL built engine adapters to access each of these engines’ native shared library.

DJL does it in an elegant way making it dead simple to switch from one framework to the other depending on the use case.

Dependencies

The app needs the Spring Boot web starter:

    Java
   
          x
         
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

And the commons-io library for some basic I/O operations:

    Java
   
xxxxxxxxxx

       <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.6</version>
       </dependency>

The Lombok library, too, as I'm too lazy to write the getters and setters:

    Java
   
xxxxxxxxxx

       <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
       </dependency>

And finally the DJL dependencies for this sample app:

    Java
   
xxxxxxxxxx

       <dependency>
            <groupId>ai.djl</groupId>
            <artifactId>api</artifactId>
            <version>${ai.djl.version}</version>
       </dependency>
       <dependency>
            <groupId>ai.djl.tensorflow</groupId>
            <artifactId>tensorflow-api</artifactId>
            <version>${ai.djl.version}</version>
       </dependency>
       <dependency>
            <groupId>ai.djl.tensorflow</groupId>
            <artifactId>tensorflow-engine</artifactId>
            <version>${ai.djl.version}</version>
       </dependency>
       <dependency>
            <groupId>ai.djl.tensorflow</groupId>
            <artifactId>tensorflow-native-auto</artifactId>
            <version>${tensorflow-native-auto.version}</version>
            <scope>runtime</scope>
       </dependency>
       <dependency>
            <groupId>net.java.dev.jna</groupId>
            <artifactId>jna</artifactId>
            <version>${jna.version}</version>      <!-- overrides default spring boot version to comply with DJL -->
       </dependency>

Here's the list of Maven properties needed for the DJL dependency versions:

    Java
   
xxxxxxxxxx

   <properties>
        <java.version>1.8</java.version>
        <ai.djl.version>0.5.0</ai.djl.version>
        <jna.version>5.3.0</jna.version>
        <tensorflow-native-auto.version>2.1.0</tensorflow-native-auto.version>
   </properties>

XRayApplication class

This class's main() method will fire up the Spring Boot application and it looks like most other Application class files:

    Java
   
xxxxxxxxxx

@SpringBootApplication public class XRayApplication { 
    public static void main(String[] args) {
         SpringApplication.run(XRayApplication.class, args);
    } 
}

Configuration

In order to configure the DJL library, let's create a DjlConfig class with the @Configuration annotation.

This class will define a ZooModel Spring Bean that will help predicting if the submitted image URL belongs to a COVID-19 infected lung:

    Java
   
xxxxxxxxxx

    @Bean
    public ZooModel xrayModel() throws Exception {
        Criteria<BufferedImage, Classifications> criteria =
                Criteria.builder()
                        .setTypes(BufferedImage.class, Classifications.class)
                        .optTranslator(new XrayTranslator())
                        .build();
        return ModelZoo.loadModel(criteria);
    }

What this code says is that we create a ZooModel object with a BufferedImage input and Classifications (more on that later) output type and it uses an XrayTranslator object to transform the input images to a format needed by the Deep Learning model to function properly.

Here's the code for the XrayTranslator which is an inner class within DjlConfig:

    Java
   
xxxxxxxxxx

   public static final class XrayTranslator implements Translator<BufferedImage, Classifications> {
         private static final List<String> CLASSES = Arrays.asList("covid-19", "normal");
         @Override
         public NDList processInput(TranslatorContext ctx, BufferedImage input) {
            NDArray array =
                    BufferedImageUtils.toNDArray(
                            ctx.getNDManager(), input, NDImageUtils.Flag.COLOR);
            array = NDImageUtils.resize(array, 224).div(255.0f);
            return new NDList(array);
        }
        @Override
        public Classifications processOutput(TranslatorContext ctx, NDList list) {
            NDArray probabilities = list.singletonOrThrow();
            return new Classifications(CLASSES, probabilities);
        }
    }

Covid19Service

The Covid19Service class will handle the business logic to diagnose the X-ray images and as you'll see, surprisingly, it's really just few lines of code:

    Java
   
xxxxxxxxxx

@Service
public class Covid19Service {
    @Autowired
    private ZooModel xrayModel;
    public String diagnose(String imageUrl) {
        try (Predictor<BufferedImage, Classifications> predictor = xrayModel.newPredictor()) {
            Classifications result = predictor.predict(BufferedImageUtils.fromUrl(imageUrl));
            return "Diagnose: "
                    + result.best().getClassName()
                    + " , probability: "
                    + result.best().getProbability();
        } catch (Exception e) {
            throw new RuntimeException("Failed to diagnose", e);
        }
    }
}

The ZooModel bean created in the DjlConfig class is autowired and used in the diagnose()method that has an imageUrl parameter.

Within the method we create a Predictorobject using the try-resource block (as the predictor needs to be closed after execution) and use it to run the BufferedImage(created using the imageUrl parameter) through a pre-trained Tensorflow model.

For more details on the model visit: https://www.pyimagesearch.com/2020/03/16/detecting-covid-19-in-x-ray-images-with-keras-tensorflow-and-deep-learning/.

Once the diagnose()method is run, the Classificationsresult object will show if the lungs on the X-ray image were infected with COVID-19 or not and with what probability.

Covid19Controller

This controller class defines the REST API to diagnose X-ray images which will be consumed by our simple front-end app:

    Java
   
xxxxxxxxxx

@RestController
@RequestMapping(value = "/api/v1", produces = MediaType.APPLICATION_JSON_VALUE)
public class Covid19Controller {
    private final Covid19Service covid19Service;
    public Covid19Controller(Covid19Service covid19Service) {
        this.covid19Service = covid19Service;
    }
    @GetMapping("/covid19/diagnose")
    public ResponseEntity diagnose(@RequestParam String imageUrl) {
        String answer = covid19Service.diagnose(imageUrl);
        return ResponseEntity.ok(Collections.singletonMap("result", answer));
    }
}

The @RestControllerannotation tells Spring that in our MVC design this is a Controller bean that defines the REST api

The @RequestMappingannotation tells Spring that paths of all REST apis within this class should be prefixed with /api/v1and all REST apis will return application\jsonresponses.

The Covid19Servicethat we discussed earlier is autowired in the constructor and later on used by the diagnose REST api at the GET /api/v1/covid19/diagnosepath.

The diagnose api takes an imageUrlrequest param and returns a JSON document with the String representation of the results.

Front-end

The Spring Boot app has a simple static index.html file as a front-end client for the diagnose REST API and it uses Twitter Bootstrap for the responsive design and JQuery to make the REST API call:

    JavaScript
   
xxxxxxxxxx

<head>
    <link rel="stylesheet" href="/css/bootstrap.min.css"/>
    <script src="/js/jquery.min.js"></script>
</head>

The file has an HTML form that can capture an X-ray image URL from the user:

    HTML
   
xxxxxxxxxx

       <form id="diagnoseForm" class="mb-4">
            <div class="input-group">
                <input type="url" id="imageUrl" class="form-control" required
                       placeholder="Enter a image url"
                       aria-label="Image URL">
                <div class="input-group-append">
                    <button class="btn btn-outline-primary">Submit</button>
                </div>
            </div>
       </form>

Once the form is submitted, the REST API may take a while to respond. In the meantime the page will show a spinner and once the response is received, the text will be displayed within the diagnose div:

    HTML
   
xxxxxxxxxx

       <div class="row ml-1">
            <div id="spinnerDiagnose" class="text-primary" role="status">
                <span class="sr-only">Loading...</span>
            </div>
            <div id="diagnose"></div>
       </div>

See below the Javascript code:

    JavaScript
   
xxxxxxxxxx

$( "#diagnoseForm" ).submit(function( event ) {
  const imageUrl = $('#imageUrl').val();
  $('#spinnerDiagnose').addClass('spinner-border');
  $('#diagnose').html('');
  $.ajax('/api/v1/covid19/diagnose?imageUrl='+imageUrl)
  .done(data => {
    $('#spinnerDiagnose').removeClass('spinner-border');
    $('#diagnose').html(data.result);
  })
  .fail(err => {
    $('#spinnerDiagnose').removeClass('spinner-border');
    $('#diagnose').html('Failed to get answer');
  });
  event.preventDefault();
});

When the form's submit event is triggered, the code gets the imageUrl value, show's the spinner, clears the content of the diagnose div from previous runs and calls the diagnose REST api with the imageUrl.

In case of a successful response the code hides the spinner and displays the results within the diagnosediv.

In case of an error the code also hides the spinner and displays a generic error message.

Running the app

The app needs Tensorflow to be downloaded first before it can be run.

Run the following command in the projects root folder:

    Shell
   
xxxxxxxxxx

mkdir models
cd models
curl https://djl-tensorflow-javacpp.s3.amazonaws.com/tensorflow-models/covid-19/saved_model.zip | jar xv cd .. ./mvnw spring-boot:run -Dai.djl.repository.zoo.location=models/saved_model

Then visit http://localhost:8080/index.html to get diagnose on X-ray image URLs. Sample images to use:

Overview

In this tutorial, we reviewed how to create a sample Deep Learning Java app using Spring Boot, DJL, and Tensorflow.

The source code for the post is available at https://github.com/davidkiss/djl-spring-boot-xray.

Spring Framework Spring Boot Deep learning

Published at DZone with permission of David Kiss, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Comments

Trending

Trending

Deep Learning with Spring Boot and DJL

In this tutorial we review how to create a sample Deep Learning Java app using Spring Boot, DJL and Tensorflow.

Overview

Deep Java Library

Dependencies

XRayApplication class

Configuration

Covid19Service

Covid19Controller

Front-end

Running the app

Overview

Trending

Partner Resources