DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones AWS Cloud
by AWS Developer Relations
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Partner Zones
AWS Cloud
by AWS Developer Relations
Securing Your Software Supply Chain with JFrog and Azure
Register Today

Trending

  • Tech Hiring: Trends, Predictions, and Strategies for Success
  • Getting Started With Istio in AWS EKS for Multicluster Setup
  • Top 10 Engineering KPIs Technical Leaders Should Know
  • What Is Envoy Proxy?

Trending

  • Tech Hiring: Trends, Predictions, and Strategies for Success
  • Getting Started With Istio in AWS EKS for Multicluster Setup
  • Top 10 Engineering KPIs Technical Leaders Should Know
  • What Is Envoy Proxy?
  1. DZone
  2. Coding
  3. Frameworks
  4. Image Loader Powered by Apache Camel

Image Loader Powered by Apache Camel

Bilgin Ibryam user avatar by
Bilgin Ibryam
·
Apr. 27, 13 · Interview
Like (0)
Save
Tweet
Share
6.07K Views

Join the DZone community and get the full member experience.

Join For Free

 

Live #Olympics Image Stream

(The application has been improved and moved to www.livephotostream.com read about the move here)
This is a very short post about a very simple application way back during the London 2012 Olympics and based on Apache Camel examples, I've created an application that displays twitter images related to the Olympics in real time. It listens for tweets containing images, filters out duplicates, and sends the images to the browser using websockets every 5 seconds. See it live running on amazon micro instance or check it out from my github account.

This is an end-to-end push application: once a user pushes an image to twitter, twitter pushes the image to the camel application, and then the camel application pushes the image to all the clients. It is built using only "free stuff": Twitter's free streaming API, Apache Camel framework, Amazon's free micro instance and done during my free time. Here is the essence:
public void configure() throws Exception {

    from("twitter://streaming/filter?type=event&keywords=" + searchTerm)

            .to("log:tweetStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")

            .process(new ImageExtractor())

            .process(new Statistics())

            .filter(body().isInstanceOf(Tweet.class))

            .idempotentConsumer(header(UNIQUE_IMAGE_URL), MemoryIdempotentRepository.memoryIdempotentRepository(10000))

            .to("log:imageStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")

            .throttle(1).timePeriodMillis(5000).asyncDelayed().callerRunsWhenRejected(false)

            .marshal().json(JsonLibrary.Jackson)

            .to("websocket:camelympics?sendToAll=true");
}

Line by line explanation
Twitter pushes tweets containing the #Olympics and #London2012 tags to the app:
from("twitter://streaming/filter?type=event&keywords=" + searchTerm)
Log statistical information about the number of messages every minute. Not visible the users:
.to("log:tweetStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")
Extract images from the tweets which has media associated with:
.process(new ImageExtractor())
Put the current number of tweets and images in the message:
.process(new Statistics())
Filter out all the tweets which doesn't contain images:
.filter(body().isInstanceOf(Tweet.class))
Filter out duplicated images, identified by their url:
.idempotentConsumer(header(UNIQUE_IMAGE_URL), MemoryIdempotentRepository.memoryIdempotentRepository(10000))
Log again, the messages which reached this far in the route:
.to("log:imageStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")
Let images go with 5 seconds difference, so the user can enjoy it. Also important - don't block the twitter listener by using callerRunsWhenRejected if the image buffer fills up, Twitter will block you:
.throttle(1).timePeriodMillis(5000).asyncDelayed().callerRunsWhenRejected(false)
Serialize into json:
.marshal().json(JsonLibrary.Jackson)
Push it to the users:
.to("websocket:camelympics?sendToAll=true");
The application can be run also locally and allows filtering images not only for the Olympics but for any keywords passed as argument. Don't forget to use your own twitter oath tokens when using locally though.

Enjoy the Olympics.

Apache Camel Loader (equipment)

Published at DZone with permission of Bilgin Ibryam, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Trending

  • Tech Hiring: Trends, Predictions, and Strategies for Success
  • Getting Started With Istio in AWS EKS for Multicluster Setup
  • Top 10 Engineering KPIs Technical Leaders Should Know
  • What Is Envoy Proxy?

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com

Let's be friends: