Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

What Most People Are Searching on Google? Let's Explore Google Hot Trends…

DZone's Guide to

What Most People Are Searching on Google? Let's Explore Google Hot Trends…

Learn how to explore Google trends with this Spring Batch app example.

· Java Zone ·
Free Resource

How do you break a Monolith into Microservices at Scale? This ebook shows strategies and techniques for building scalable and resilient microservices.

In this article, we will see a second example Spring batch application. To create sample Spring Batch application, we will make use of Google hot searches feed. Google provides hot trends app where you can choose the country to find the hot searches happening.

Google Hot Searches

               









The Google hot searches are available as an atom feed (http://www.google.com/trends/hottrends/atom/feed?pn=p3). To use Spring Batch framework, we should add the below dependencies to pom.xml.


<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
    <version>3.0.3.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-beans</artifactId>
    <version>4.1.4.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
    <version>1.2.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-context</artifactId>
    <version>4.1.4.RELEASE</version>
</dependency>

We have used feed4j  to parse the atom feed. Download the feed4j jars and keep the dependencies in pom.xml.


<dependency>
    <groupId>feed4j</groupId>
    <artifactId>feed4j</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/feed4j.jar</systemPath>
</dependency>
<dependency>
    <groupId>nekohtml</groupId>
    <artifactId>nekohtml</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/nekohtml.jar</systemPath>
</dependency>
<dependency>
    <groupId>xercesImpl</groupId>
    <artifactId>xercesImpl</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xercesImpl.jar</systemPath>
</dependency>
<dependency>
    <groupId>xml-apis</groupId>
    <artifactId>xml-apis</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xml-apis.jar</systemPath>
</dependency>

Now, we are ready to implementItem Reader to get the hot trends feed and parse it.


package org.smarttechie.reader;

import it.sauronsoftware.feed4j.FeedParser;
import it.sauronsoftware.feed4j.bean.Feed;
import it.sauronsoftware.feed4j.bean.FeedItem;

import java.net.URL;
import java.util.ArrayList;
import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;

/**
* The reader gets the atom feed and parses it.
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsReader implements ItemReader<List<GoogleTrendItem>>{

private Log log = LogFactory.getLog(GoogleTrendsReader.class);

/**
* The reader will make call to google hot trends API
*/
@Override
public List<GoogleTrendItem> read() throws Exception, UnexpectedInputException,
ParseException, NonTransientResourceException {
          URL url = new URL("http://www.google.com/trends/hottrends/atom/feed?pn=p3");
          Feed feed = FeedParser.parse(url);
          int items = feed.getItemCount();
          log.debug("The number of hot trends " + items);
          List<GoogleTrendItem> trends = null;
          if (items > 0) {
            trends = new ArrayList<GoogleTrendItem>();
              for (int i = 0; i < items; i++) {
                   FeedItem item = feed.getItem(i);
                   GoogleTrendItem trend = new GoogleTrendItem();
                   trend.setTitle(item.getTitle());
                   trend.setLink(item.getLink().toString());
                   trend.setDescription(item.getDescriptionAsText());
                   trends.add(trend);
        }
    }
    return trends;
  }
 }

The Item Writer implementation is given below.



package org.smarttechie.writer;

import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemWriter;

/**
* The writer to write the list of google trends
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsWriter implements ItemWriter<List<GoogleTrendItem>>{

private Log log = LogFactory.getLog(GoogleTrendsWriter.class);

@Override
public void write(List<? extends List<GoogleTrendItem>> trends)
throws Exception {
        if (trends != null && !trends.isEmpty()) {
           List<GoogleTrendItem> items = trends.get(0);
              if (items != null && !items.isEmpty()) {
                 for (GoogleTrendItem trend : items) {
                   log.debug("Title: " + trend.getTitle());
                   log.debug("Link: " + trend.getLink());
                   log.debug("Plain text description: " + trend.getDescription());
       }
     }
   }
  }
 }

The sequence diagram of the above components interaction is given below.

Google Trends Spring Batch













The batch configuration is given below.


package org.smarttechie;

import java.util.List;

import org.smarttechie.model.GoogleTrendItem;
import org.smarttechie.reader.GoogleTrendsReader;
import org.smarttechie.writer.GoogleTrendsWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class GoogleTrendsBatchConfiguration {

@Bean
public ItemReader<List<GoogleTrendItem>> reader() {
   GoogleTrendsReader reader = new GoogleTrendsReader();
   return reader;
}

@Bean
public ItemWriter<List<GoogleTrendItem>> writer() {
   GoogleTrendsWriter writer = new GoogleTrendsWriter();
   return writer;
}

@Bean
public Step step1(StepBuilderFactory stepBuilderFactory, ItemReader<List<GoogleTrendItem>> reader,
ItemWriter<List<GoogleTrendItem>> writer) {
    return stepBuilderFactory.get("step1")
    .<List<GoogleTrendItem>, List<GoogleTrendItem>> chunk(1)
    .reader(reader)
    .writer(writer)
    .build();
}

@Bean
public Job importUserJob(JobBuilderFactory jobs, Step s1) {
  return jobs.get("importHotSeachesJob").preventRestart()
  .start(s1)
  .build();
 }
}

The above implementation doesn’t have ItemProcessor which is optional. The source code created as part of this article is available here.

How do you break a Monolith into Microservices at Scale? This ebook shows strategies and techniques for building scalable and resilient microservices.

Topics:
google search ,java ,spring

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}