Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

What Most People Are Searching on Google? Let's Explore Google Hot Trends…

DZone's Guide to

What Most People Are Searching on Google? Let's Explore Google Hot Trends…

Learn how to explore Google trends with this Spring Batch app example.

Free Resource

Learn how to troubleshoot and diagnose some of the most common performance issues in Java today. Brought to you in partnership with AppDynamics.

In this article, we will see a second example Spring batch application. To create sample Spring Batch application, we will make use of Google hot searches feed. Google provides hot trends app where you can choose the country to find the hot searches happening.

Google Hot Searches

               









The Google hot searches are available as an atom feed (http://www.google.com/trends/hottrends/atom/feed?pn=p3). To use Spring Batch framework, we should add the below dependencies to pom.xml.


<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
    <version>3.0.3.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-beans</artifactId>
    <version>4.1.4.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
    <version>1.2.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-context</artifactId>
    <version>4.1.4.RELEASE</version>
</dependency>

We have used feed4j  to parse the atom feed. Download the feed4j jars and keep the dependencies in pom.xml.


<dependency>
    <groupId>feed4j</groupId>
    <artifactId>feed4j</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/feed4j.jar</systemPath>
</dependency>
<dependency>
    <groupId>nekohtml</groupId>
    <artifactId>nekohtml</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/nekohtml.jar</systemPath>
</dependency>
<dependency>
    <groupId>xercesImpl</groupId>
    <artifactId>xercesImpl</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xercesImpl.jar</systemPath>
</dependency>
<dependency>
    <groupId>xml-apis</groupId>
    <artifactId>xml-apis</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>C:/Users/test/GT_WS/Google_Trends/lib/xml-apis.jar</systemPath>
</dependency>

Now, we are ready to implementItem Reader to get the hot trends feed and parse it.


package org.smarttechie.reader;

import it.sauronsoftware.feed4j.FeedParser;
import it.sauronsoftware.feed4j.bean.Feed;
import it.sauronsoftware.feed4j.bean.FeedItem;

import java.net.URL;
import java.util.ArrayList;
import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;

/**
* The reader gets the atom feed and parses it.
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsReader implements ItemReader<List<GoogleTrendItem>>{

private Log log = LogFactory.getLog(GoogleTrendsReader.class);

/**
* The reader will make call to google hot trends API
*/
@Override
public List<GoogleTrendItem> read() throws Exception, UnexpectedInputException,
ParseException, NonTransientResourceException {
          URL url = new URL("http://www.google.com/trends/hottrends/atom/feed?pn=p3");
          Feed feed = FeedParser.parse(url);
          int items = feed.getItemCount();
          log.debug("The number of hot trends " + items);
          List<GoogleTrendItem> trends = null;
          if (items > 0) {
            trends = new ArrayList<GoogleTrendItem>();
              for (int i = 0; i < items; i++) {
                   FeedItem item = feed.getItem(i);
                   GoogleTrendItem trend = new GoogleTrendItem();
                   trend.setTitle(item.getTitle());
                   trend.setLink(item.getLink().toString());
                   trend.setDescription(item.getDescriptionAsText());
                   trends.add(trend);
        }
    }
    return trends;
  }
 }

The Item Writer implementation is given below.



package org.smarttechie.writer;

import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.smarttechie.model.GoogleTrendItem;
import org.springframework.batch.item.ItemWriter;

/**
* The writer to write the list of google trends
* @author Siva Janapati
* @version 1.0
*
*/
public class GoogleTrendsWriter implements ItemWriter<List<GoogleTrendItem>>{

private Log log = LogFactory.getLog(GoogleTrendsWriter.class);

@Override
public void write(List<? extends List<GoogleTrendItem>> trends)
throws Exception {
        if (trends != null && !trends.isEmpty()) {
           List<GoogleTrendItem> items = trends.get(0);
              if (items != null && !items.isEmpty()) {
                 for (GoogleTrendItem trend : items) {
                   log.debug("Title: " + trend.getTitle());
                   log.debug("Link: " + trend.getLink());
                   log.debug("Plain text description: " + trend.getDescription());
       }
     }
   }
  }
 }

The sequence diagram of the above components interaction is given below.

Google Trends Spring Batch













The batch configuration is given below.


package org.smarttechie;

import java.util.List;

import org.smarttechie.model.GoogleTrendItem;
import org.smarttechie.reader.GoogleTrendsReader;
import org.smarttechie.writer.GoogleTrendsWriter;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class GoogleTrendsBatchConfiguration {

@Bean
public ItemReader<List<GoogleTrendItem>> reader() {
   GoogleTrendsReader reader = new GoogleTrendsReader();
   return reader;
}

@Bean
public ItemWriter<List<GoogleTrendItem>> writer() {
   GoogleTrendsWriter writer = new GoogleTrendsWriter();
   return writer;
}

@Bean
public Step step1(StepBuilderFactory stepBuilderFactory, ItemReader<List<GoogleTrendItem>> reader,
ItemWriter<List<GoogleTrendItem>> writer) {
    return stepBuilderFactory.get("step1")
    .<List<GoogleTrendItem>, List<GoogleTrendItem>> chunk(1)
    .reader(reader)
    .writer(writer)
    .build();
}

@Bean
public Job importUserJob(JobBuilderFactory jobs, Step s1) {
  return jobs.get("importHotSeachesJob").preventRestart()
  .start(s1)
  .build();
 }
}

The above implementation doesn’t have ItemProcessor which is optional. The source code created as part of this article is available here.

Understand the needs and benefits around implementing the right monitoring solution for a growing containerized market. Brought to you in partnership with AppDynamics.

Topics:
google search ,java ,spring

Published at DZone with permission of Siva Prasad Rao Janapati, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}