DZone
Big Data Zone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
  • Refcardz
  • Trend Reports
  • Webinars
  • Zones
  • |
    • Agile
    • AI
    • Big Data
    • Cloud
    • Database
    • DevOps
    • Integration
    • IoT
    • Java
    • Microservices
    • Open Source
    • Performance
    • Security
    • Web Dev
DZone > Big Data Zone > Keeping the Web API Layer in Kafka With a REST Proxy

Keeping the Web API Layer in Kafka With a REST Proxy

Kafka is the quickest way I have seen to get started with real-time data streams. However, I've noticed many Apache products diverting from REST.

Kin Lane user avatar by
Kin Lane
·
Oct. 09, 17 · Big Data Zone · Opinion
Like (4)
Save
Tweet
16.71K Views

Join the DZone community and get the full member experience.

Join For Free

I’m slowly learning more about Kafka and the other messaging and data streaming solutions gaining traction in the API space.

Image title

If you aren’t on the Kafka train yet, according to their website:

“Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.” I’m still learning about how Kafka works, and with no real production experience, it is something that is taking time.

As part of my conversations on the subject, I was introduced to Confluent, a platform version of Kafka, which is the quickest way I have seen to get started with real-time data streams. As part of the Confluent offering, I noticed they have a REST proxy for which you can find the API documentation here. You can find the code for the Kafka REST proxy on GitHub. According to the GitHub repo:

“The Kafka REST Proxy provides a RESTful interface to a Kafka cluster. It makes it easy to produce and consume messages, view the state of the cluster, and perform administrative actions without using the native Kafka protocol or clients.”

I’ve noticed that many of the other messaging and data streaming solutions out of Apache these days have diverted from using REST, which makes sense for speed and scale — but when it comes to reaching a wider audience, I can still see the need to have RESTful API, delivering a kind of multi-speed solution that allows developers to pick their speed based upon their skills, awareness, and need. I’m feeling like the platform approach of Confluent, combined with a RESTful layer, will give them an advantage over other Kafka service providers or just deploying the open-source solution out of the box.

REST isn’t always the most efficient or scalable solution, but when it comes to reaching a wide audience of developers and allowing consumers to get up and running quickly, REST is still a sensible approach. Honestly, I don’t think it is just REST; it is also about leveraging the web — not that everyone understands the web, but I think it is what a large number of developers have been exposed to, and have been building on in the last decade. I can see high-volume API solutions in the future often having a native protocol and client, but also supporting REST and gRPC to make their solutions more accessible, performant, scalable, and quickly adopted and integrated alongside existing infrastructure.

REST Web Protocols API kafka Web API

Published at DZone with permission of Kin Lane, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Portfolio Architecture Examples: Retail Collection
  • Ultra-Fast Microservices: When Microstream Meets Payara
  • Refactoring Java Application: Object-Oriented And Functional Approaches
  • Deployment of Low-Latency Solutions in the Cloud

Comments

Big Data Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • MVB Program
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends:

DZone.com is powered by 

AnswerHub logo