Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

The Most Valuable Twitter API Paths to Consider for Streaming

DZone's Guide to

The Most Valuable Twitter API Paths to Consider for Streaming

Find out 15 of the most valuable Twitter API paths to consider for streaming, depending on the use case and you as a user.

· Integration Zone ·
Free Resource

The State of API Integration 2018: Get Cloud Elements’ report for the most comprehensive breakdown of the API integration industry’s past, present, and future.

We are creating AWS Lambda functions for a variety of Twitter APIs, which allow us to stream Twitter data into Amazon S3 data lakes. We've already made one for Twitter searches, allowing anyone to stream a specific search into an AWS data lake. As part of this work, we wanted to share the entire list of Twitter API paths we are developing connectors for and will be publishing to the AWS Serverless Application Repository so that anyone can begin streaming data into their data lakes within their own AWS infrastructure.

As part of our profiling of Twitter for the Streamdata.io API Gallery, we've identified 15 separate API paths as having a potential for streaming, depending on the user as well as the use case. Providing us with the Twitter API resources we wanted to make available via serverless streams using AWS Lambda:

As we said, we've already turned the Search Tweets path into a streaming connector, but we should have the rest of these ready by the end of the week and published into the AWS Serverless Application Gallery. To operate each one, you will need a Streamdata.io account, as well as a Twitter account, however, once you have your Streamdata.io key, and Twitter OAuth token, all you have to do is deploy the function to your account, enter the values, and you can begin streaming. It is up to you how long you run the streams for and orchestrate them using AWS Cloudwatch events, where you can run indefinitely or based upon a schedule or other events.

Streamdata.io provides an alternative for the recent deprecation by Twitter of their streaming endpoints. You can proxy the Twitter REST APIs using Streamdata.io and deliver to multiple web or mobile clients, or you can connect and stream into an Amazon S3 data like, and use the data to train machine learning applications and other applications. Our goal with this work is to make all APIs as plug and play as possible, allowing anyone to quickly integrate with a variety of 3rd party APIs using their existing infrastructure. Making AWS Lambda a pretty compelling approach to providing plug and play connectors that are deployable with a single click, and begin streaming into your existing data lakes for use across a variety of existing applications.

Your API is not enough. Learn why (and how) leading SaaS providers are turning their products into platforms with API integration in the ebook, Build Platforms, Not Products from Cloud Elements.

Topics:
integration ,aws lambda ,apis ,twitter api ,data lakes ,rest

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}