Over a million developers have joined DZone.

The Most Valuable Twitter API Paths to Consider for Streaming

DZone's Guide to

The Most Valuable Twitter API Paths to Consider for Streaming

Find out 15 of the most valuable Twitter API paths to consider for streaming, depending on the use case and you as a user.

· Integration Zone ·
Free Resource

How to Transform Your Business in the Digital Age: Learn how organizations are re-architecting their integration strategy with data-driven app integration for true digital transformation.

We are creating AWS Lambda functions for a variety of Twitter APIs, which allow us to stream Twitter data into Amazon S3 data lakes. We've already made one for Twitter searches, allowing anyone to stream a specific search into an AWS data lake. As part of this work, we wanted to share the entire list of Twitter API paths we are developing connectors for and will be publishing to the AWS Serverless Application Repository so that anyone can begin streaming data into their data lakes within their own AWS infrastructure.

As part of our profiling of Twitter for the Streamdata.io API Gallery, we've identified 15 separate API paths as having a potential for streaming, depending on the user as well as the use case. Providing us with the Twitter API resources we wanted to make available via serverless streams using AWS Lambda:

As we said, we've already turned the Search Tweets path into a streaming connector, but we should have the rest of these ready by the end of the week and published into the AWS Serverless Application Gallery. To operate each one, you will need a Streamdata.io account, as well as a Twitter account, however, once you have your Streamdata.io key, and Twitter OAuth token, all you have to do is deploy the function to your account, enter the values, and you can begin streaming. It is up to you how long you run the streams for and orchestrate them using AWS Cloudwatch events, where you can run indefinitely or based upon a schedule or other events.

Streamdata.io provides an alternative for the recent deprecation by Twitter of their streaming endpoints. You can proxy the Twitter REST APIs using Streamdata.io and deliver to multiple web or mobile clients, or you can connect and stream into an Amazon S3 data like, and use the data to train machine learning applications and other applications. Our goal with this work is to make all APIs as plug and play as possible, allowing anyone to quickly integrate with a variety of 3rd party APIs using their existing infrastructure. Making AWS Lambda a pretty compelling approach to providing plug and play connectors that are deployable with a single click, and begin streaming into your existing data lakes for use across a variety of existing applications.

Make your mark on the industry’s leading annual report. Fill out the State of API Integration 2019 Survey and receive $25 to the Cloud Elements store.

integration ,aws lambda ,apis ,twitter api ,data lakes ,rest

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}