DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. Streaming Technology as Event-Driven Infrastructure When Done Precisely

Streaming Technology as Event-Driven Infrastructure When Done Precisely

Let's take a look at how streaming APIs needs to be more event-driven as long as it's done correctly and precisely.

Kin Lane user avatar by
Kin Lane
·
Aug. 06, 18 · Opinion
Like (3)
Save
Tweet
Share
6.68K Views

Join the DZone community and get the full member experience.

Join For Free

Streaming APIs have an image problem. Because of the dominance of Websockets when it comes to delivering real-time APIs, many people have a view of streaming APIs as being all about high volume data flows and not something that is precise or event-driven. Because of the popularity of streaming APIs like the Twitter firehose, and financial or news APIs delivered via on or off Websocket streams, people associate what we do with a pretty narrow view of the data landscape. This is something we are working to correct, as we position streaming technology as more of an event-driven approach when done precisely.

First, Streamdata.io uses Server-Sent Events (SSE) to deliver our streams. It is a one-way streaming standard that uses HTTP, where Websockets is a two-way streaming standard using TCP. The technology we use is more precise and reflects the world of web APIs more than WebSockets does. Next, we use JSON PATCH to deliver incremental updates to our consumers, pushing updates whenever something has changed with a feed. By default, we aren't just about turning on and off streams, we are about establishing sustained connections with existing web APIs, and getting streams of data pushed whenever something changes. The streaming is just one variable in an event-driven equation. Nothing is streamed unless an event occurs, which depends on how broad or precise you define the query you are sending to an API being proxied by Streamdata.io.

Proxying APIs with Streamdata.io isn't just about streaming, and turning on the data fire hose, as many people assume. Streamdata.io is about turning on precise streams of data that push updates when things have changed, and some meaningful event has occurred. You can proxy the Twitter API for a specific Twitter account and establish streams of data whenever someone tweets, or maybe when they like, follow or perform some other relevant action. You can proxy a subreddit on the popular Reddit site, and establish a stream that only pushes updates when someone mentions a specific keyword or topic. Establishing streams on top of existing APIs, but making sure they are more precise, event-driven streams of data, not just fire hose streams of data that you simply turn on or off. Making streaming more about event-driven infrastructure, than it is ever about just real-time infrastructure.

We are working hard to differentiate what Streamdata.io does from other streaming providers. Demonstrating that we aren't just a real-time company and are more about delivering event-driven infrastructure on top of existing web APIs. Helping companies begin to understand how they can evolve beyond just being request and response driven, and get more efficient about how they deliver information to their consumers. Leveraging their existing infrastructure, but making sure they are moving it forward in a way that is keeping pace with where the rest of the API landscape is headed. Providing webhooks, streams, subscriptions, and other event-driven approaches to orchestrating data and content using low-cost web infrastructure.

Infrastructure IT Stream (computing) Data (computing)

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Building a Scalable Search Architecture
  • How and Why You Should Start Automating DevOps
  • The 12 Biggest Android App Development Trends in 2023
  • An Introduction to Data Mesh

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: