Webhooks are HTTP callbacks that are triggered by events specified in the code by the application developer; the event data is then recorded and pushed to an endpoint URL. Webhooks are a powerful way of exchanging data.

Let’s take an example.

Imagine that you go to your neighborhood grocery store to pick up mangoes, but unfortunately, today, the shop is out of the mangoes that you want. The delivery boy who is supposed to bring the mangoes to the store is stuck in traffic. So, the shop owner asks you to come back in an hour. You go back home and return to the shop after an hour, but the delivery boy has not yet arrived with the mangoes. The story repeats, and after multiple attempts, you finally get your beloved mangoes. This is exactly how APIs work. You keep making requests, you keep wasting resources, but you won’t get data until it is available.

Now imagine that, instead of you going to the shop, you leave your address with the shopkeeper and ask him to deliver the mangoes as soon as they arrive. This will help you save your time, effort and resources. This is exactly how Webhooks work.

One of the advantages of Webhooks is that, since there’s no request required for a Webhook, it sends the data to the endpoint as soon as it’s available and, hence, the data is made available in near real time.

In this blog, we explore the idea of using Webhooks to analyze web performance data from Catchpoint in your Amazon Elasticsearch service.

Step 1: Setting Up the AWS Instance

Here are the steps to set up your own AWS instance.

Log into your AWS account.

Go to Services and select Elasticsearch Service under the Analytics section.

Create a Domain and select the Elasticsearch version you want, then click next.

Configure the cluster as per your requirements and click Next.

Set up your access policy, make sure you allow the Catchpoint Webhook domain/IP so that it can post data, then click Next.

Confirm your changes and create the AWS ES instance. Wait for the Domain Status to change to Active. That’s when the service is ready to ingest data. Here, the endpoint URL and Kibana URL are important.

Create an index to push data to AWS ES.

For example: The following curl request will create an index called cpindex

curl -X PUT http://search-catchpoint-test-imwp2rphrbhue6k2gizw6c5afm.ap-south.es.amazonaws.com/cpindex  -H 'cache-control: no-cache' -H 'content-length: 225' -H 'content-type: application/json'

While sending data, don’t forget to add a type to this index.

So, the new post URL will look like, here data is the AWS ES type:

http://search-catchpoint-test-imwp2rphrbhue6k2gizw6c5afm.ap-south.es.amazonaws.com/cpindex /data

The following lines are taken from the Elasticsearch documentation:

"For example, let’s assume you run a blogging platform and store all your data in a single index. In this index, you may define a type for user data, another type for blog data, and yet another type for comments data."

Step 2:  Setting Up the Webhook to Post Data to the AWS ES Instance Endpoint

Select a test for which you want to enable Webhooks. Go to Advanced Options in the test setup page and enable the test data Webhook option.

In the Webhook Settings, copy the AWS ES endpoint URL and save.

Step 3: Verifying the Setup

Once the test settings have been saved and a test run is executed, you can view the index in the AWS ES indices tab.

You should be able to view the results in Kibana.

That’s how you can leverage the power of Webhooks for your performance analysis.