Over a million developers have joined DZone.

Building Effective Software for Load Testing of REST APIs

DZone 's Guide to

Building Effective Software for Load Testing of REST APIs

· Cloud Zone ·
Free Resource

If you are building an Internet services company these days one thing that will absolutely be expected of you is that you will provide an API or platform for your service. This API may allow users to drive, pull data from, or extend the service. Integrating tools and services together via API is becoming more and more common. The very fast adoption of OAuth is a testament to this!

Speaking of OAuth, in order to interoperate with some of the most popular platforms on the Internet today you will need to implement an API such as OAuth in your application. So no matter how you slice it you will probably need to either design and develop or implement an API for your service.

From the very start Cloud Assault has been focused on building software that does an excellent job of load testing APIs. To this point, we recently shipped a huge feature that enables this vision and so I wanted to spend some time putting down some practical advise about how to load test APIs specifically.

The new feature is Simple Payload support. Load tests can now be configured to take a payload input that will be sent to the server as a part of each request. By crafting your requests and load testing your APIs you can determine a number of things including:

  • How well does the API scale?
  • Does my rate-limiting strategy work appropriately and as expected?
  • At what point does performance degrade to such a state that I would better serve my customers by sending back a "We're overloaded" error message. 

Getting to know a Hypermedia API

I wrote up a VERY simple application called Hypercontact for use in this blog (githubAPI). It is a simple REST API for storing contacts. It was built in ruby using sinatra, roar, and mongoid. It uses MongoDB for its backend and it is hosted up on Heroku at http://hypercontact.heroku.com

The API only has a few content types worth worrying about:

/* application/vnd.hypercontact+json
 * A JSON object representing a single contact record. 
 * This contains all the data bout a particular contact
  "name":"Adam", /* contact name */
  "address":"1 Some St.", /* street address */
  "city":"Someville", /* city */
  "state":"MA", /* state or territory */
  "zip":"00000", /* postal code */
  "notes":"blah blah" /* free form text */

/* application/vnd.hypercontact-list+json
 * An array of contacts 
  { /* ... an application/vnd.hypercontact+json object */ },
  { /* ... an application/vnd.hypercontact+json object */ }

/* application/vnd.hypercontact-error+json
 * a JSON object containing error information. 
  "error":"reason" /* description of error that occurred

Hypercontact exposes a JSON API that lets you do CRUD operations on a big list of contacts. Exercising a REST/hypermedia API can be either easy or hard depending on the content-type you need to send to your API. If you are using JSON then its really straightforward. I am a huge fan of the tool RESTClient for exercising REST APIs and generating requests. curl works great as well!

To get started we should probably just see whats going on at the root of the API (/)

aschepis-air:~ aschepis$ curl -v hypercontact.heroku.com
 GET / HTTP/1.1
 User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
 Host: hypercontact.heroku.com
 Accept: application/json

 HTTP/1.1 200 OK
 Date: Tue, 20 Mar 2012 10:14:12 GMT
 Content-Type: text/html;charset=utf-8
 Connection: keep-alive
 Link: http://hypercontact.heroku.com/contacts; rel=contacts
 Content-Length: 0

Awesome. So we can see here that at the root there is no real content to see, but what we do see is a Link header that tells us the contacts are at /contacts. Let's GET that:

aschepis-air:~ aschepis$ curl -v hypercontact.heroku.com/contacts -H "Accept: application/vnd.hypercontact-list+json"
 GET /contacts HTTP/1.1
 User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
 Host: hypercontact.heroku.com
 Accept: application/json

 HTTP/1.1 200 OK
 Date: Tue, 20 Mar 2012 10:19:13 GMT
 Content-Type: application/vnd.hypercontact-list+json
 Link: http://hypercontact.heroku.com/contacts; rel=new
 Content-Length: 621

   { "name":"Adam", "address":"1 Hypermedia Lane",
      "city":"WorldWideWebville", "state":"MA",
      "notes":"a cool guy.",
   { "name":"Bill", "address":"1 Hypermedia Lane",
      "city":"WorldWideWebville", "state":"MA",
      "notes":"a cool guy.",
   { "name":"Joe", "address":"1 Hypermedia Lane",
      "city":"WorldWideWebville", "state":"MA",
      "notes":"a cool guy.",

Awesome. So when we get that, we get a list of contacts. Each contact has an embedded link array that contains an href we can use to go fetch it directly. Also notice that there is a Link header with the rel attribute set to "new." This is telling us the URL we need to POST to to create a new contact. For now, let's fetch an existing contact:

aschepis-air:~ aschepis$ curl -v http://hypercontact.heroku.com/contacts/4f6294b50d78d774ff000001 -H "Accept: application/vnd.hypercontact+json"
 GET /contacts/4f6294b50d78d774ff000001 HTTP/1.1
 User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8r zlib/1.2.5
 Host: hypercontact.heroku.com
 Accept: application/vnd.hypercontact+json

 HTTP/1.0 200 OK
 Date: Tue, 20 Mar 2012 16:30:24 GMT
 Content-Type: application/vnd.hypercontact+json
 Content-Length: 206
 Connection: close

{"name":"Adam","address":"1 Hypermedia Lane",
 "notes":"a cool guy.",

Now that we have mapped out the surface of the API we have a pretty good idea of what it can do:

  1. Show what is available via the API (GET /)
  2. List all of the existing contacts (GET /contacts)
  3. Create new contacts (POST /contacts)
  4. Retrieve an existing contact's details (GET /contacts/<id>)

Load Testing

As a developer who is responsible for this software once it is released to production I'm going to want to have some kind of idea about how well it will scale. It may not need to handle a lot of traffic but having some context around the limitations of your API and the hotspots is really important when it comes to ops, debugging live issues, and planning future development and improvements.

I'm not too worried about the GET to / yet because it is just returning static content. If i'm only going to test a few things i'm concerned with the call that gets the list of all contacts and with the call that creates a new contact, so we're going to use Cloud Assault to actually test those APIs.

First we will create a test for the GET to /contacts since it is the simplest. We want to get an idea of how the API will scale as load increases so we will want to run a test that has a gradual ramp in the number of connected users.

The API isn't exactly a speed demon, but we've got some interesting data here! Most of this is due to the very inexpensive (read: free) hosting. It is a Sinatra app running on a free Heroku Dyno connecting to a free standard MongoDB hosted at MongoLab. Given the constraints its performing pretty well.  Looking at the # of requests per second we see it is averaging almost 105 requests per second. We can see that the response latency increases linearly with the number of concurrent users and that the number of connections per second stayed roughly the same throughout the course of the test. This seems to indicate that even under heavy load the result will be timeouts as opposed to a server crash that shuts everybody out.

The next test we want to run will be to test the ability to create new contacts. We know what the JSON should look like based on the description of the application/vnd.hypercontact+json media type so we can craft a payload and put it into a file:

We can use this in our load test:

Definitely some more interesting data here. We've got a graph showing the Avg. Response Time along with the Requests served per second as the number of concurrent users increases. There are a couple intermittent spikes in the graph but for the most part it looks like the create contact API takes about a second to run and we're doing a little more than 200 per second.

What now?

Well, we've gotten a grasp of how the Hypercontact API scales. If we want to start making improvements we would want to look at the servers to see exactly where time is being spent while processing requests and fix the bottlenecks and hotspots. If you're using server metrics suites like New Relic you can dig in to see what exactly was taking the time, but that is another blog post for another time!

Do It Yourself

Whether your API is for accessing data, extending your platform, or serving the data for a mobile app scalability is critical to your success. If you're interested in trying this for yourself create an account at Cloud Assault and try it yourself! As always, if you have any issues just hit us up!


Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}