Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Eventing and Bulk With Cloud Elements

DZone's Guide to

Eventing and Bulk With Cloud Elements

Check out how you can take advantage of Cloud Elements' frameworks out of the box to get your integrations ready to go faster than ever.

Free Resource

The Integration Zone is brought to you in partnership with Cloud Elements.  What’s below the surface of an API integration? Download The Definitive Guide to API Integrations to start building an API strategy.

Integrations are all about enabling communication between your disconnected APIs. So, when building a new integration, one of your top priorities should be to create frameworks for data transfer. Specifically, methods for detecting events in your endpoints and for sending and receiving new data rapidly.

Luckily, we like to give your integrations superpowers, so every single one of our elements comes with pre-built, standardized event handling and bulk action frameworks. If you use Cloud Elements, you can take advantage of these frameworks out of the box to get your integrations ready to go faster than ever.

Eventing

Normally for detecting events happening within an endpoint, you have two options: webhooks or polling. For many reasons, which you can learn about here, webhooks tend to be the better option, and for vendors that directly support webhooks, you can set those up through Cloud Elements.

Salesforce, for example, has built-in functionality to post events to webhooks, and you can configure that functionality through the Cloud Elements UI and make sure that any data you receive is standardized. Unfortunately, there are still many API vendors that do not offer webhook functionality out of the box. For these APIs, you'll need to use a polling framework.

Polling works by querying the endpoint for any objects that have changed within a set polling interval. If the query finds anything, it will return the new or changed data. While this sounds like a simple process, successfully implementing polling can be very complicated and varies from vendor to vendor.

One of the great features of Cloud Elements is that all of our elements have a pre-built polling framework, so we handle the messy backend and automatically update any new data, just like using a webhook.

To set up a simple polling framework in Cloud Elements, all you have to do is set up the polling event type and provide some objects to monitor for changes. For example if you set up Cloud Elements to poll contacts every 15 minutes, that will say, "every 15 minutes, go out and check to see if any contacts have changed, and if that is the case, post the changes to a webhook."

Bulk

Cloud Elements has two ways of implementing bulk actions with our elements. For the endpoints that natively support batch or bulk APIs, we can take advantage of those directly, but many do not support this functionality. For these endpoints, we've built a custom framework on top of the standard API to sort of create fake bulk actions.

For example, if you want to pull all the contacts out of an element that doesn't have a native batch or bulk API, you can implement this add-on framework which will one-by-one, pull out all those contacts and push them into a single file that you can then download.

One thing to note about our bulk framework is that it works asynchronously. What that means is that in order to create a bulk download file, you'll actually execute two API calls. The first one will be the POST /bulk/query call, which will create a bulk job and then pull, individually, all the contacts out of that endpoint. The second one you'll use is the GET /bulk/{id}/{object name} , which will actually grab the downloaded file once that first bulk job is finished. We have a similar process for uploading in bulk to an endpoint as well.

For any of the endpoints that do not natively support bulk or batch, our bulk framework is definitely something you'll want to know, since it is an enhancement on top of the base API that is only available through our platform. It gives you a big advantage over a standard DIY integration because you can use it to sync first-time data or import existing data from one CRM system to another, so it's definitely worth playing around with.

Haven't tried Cloud Elements yet? Log in for free and check out how our catalog of over 120 standardized APIs can get your integrations up and running in no time.

The State of API Integration Report provides data from the Cloud Elements platform and will help all developers navigate the recent explosion of APIs and the implications of API integrations to work more efficiently in 2017 and beyond.

Topics:
integration ,api ,eventing ,bulk ,polling

Published at DZone with permission of Vivek Ravishankar, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}