RESTful (or simply, REST) APIs and web services are continually becoming a core part of modern web applications thanks to the simplicity, scalability, and flexibility they provide. Security vulnerabilities in REST APIs expose the same risks as traditional websites and web-applications, however, some characteristics of REST APIs make it challenging for automated web security scanners to perform proper REST API Security Testing for vulnerabilities.
This post will discuss these limitations, and it will also show how security testing tools can be used to overcome the hurdles in REST API security testing.
What is a REST API?
Before we start digging into challenges in automating REST API security testing, it's important to understand what REST APIs are.
RESTful APIs and web services allow for a separation of concerns between the front-end, or presentation-layer; and the backend, or data-access layer. What makes RESTful APIs even more appealing is that the same REST API could potentially be used both by a web application, as well as other clients such as a mobile application.
Thanks to their simplicity, RESTful web services are rapidly replacing older web service technologies such as SOAP (Simple Object Access Protocol). Unlike SOAP, RESTful APIs typically expose functionality based on HTTP verbs or methods. These methods not only include the commonly known GET and POST methods but also methods such as PUT, PATCH and DELETE.
What's more, unlike in the days of SOAP APIs, REST APIs don't necessarily speak XML - you'll find that most modern-day REST APIs make use of JSON or XML (or both) as their HTTP request/response format.
Automatically Scanning REST APIs
Scanning RESTful APIs With SPA Frontends
If the RESTful web service has a Single Page Application (SPA) front-end, you can point the scanner directly to the SPA and scan it. The crawler can interact with the front-end application and issue requests like a regular user would. This means that the crawl structure identified by the crawler can be used to test the underlying web service for vulnerabilities.
Scanning RESTful APIs Without Frontends
Like SOAP's Web Service Definition Language (WSDL), RESTful web services may make use of Web Application Definition Language (WADL). While WADL definitions can be provided to the scanner directly, it is not very common for RESTful services to have an accompanying WADL definition. Since web services alone have no links to navigate through or forms to submit (as opposed to an SPA), a web crawler needs to be aware of the web service's structure before it can test it.
One of the ways to work around this is to record requests made by an API client in a format that can be consumed by automated tools. Many security testing tools support a number of formats, including HTTP Archive (HAR) files, Acunetix Proxy Log files, Telerik Fiddler SAZ files and Portswigger Burp Suite state files.
For applications with existing end-to-end tests, the most efficient way of doing this would be to run end-to-end tests by proxying traffic through an HTTP proxy that can generate one of the above machine-readable formats (e.g. HAR, Telerik Fiddler or Portswigger Burp Suite). Which can then be imported into your scanning tool.
Alternatively, requests via a proxy can be manually recorded and saved to a file (e.g. HAR, Telerik Fiddler or Portswigger Burp Suite) which may then be imported into the scanner. From here, you can make requests based on the provided file. You tool will automatically detect the input scheme, be it JSON or XML, and it will individually test a scheme's properties for a variety of vulnerabilities.
Authentication and Rate-Limiting
The large majority of REST APIs also require authentication, which in most cases is accomplished via an HTTP header passed in each HTTP request. Scanning tools allow you to define custom headers, which will be used during a crawl or a scan.
The final obstacle to REST API security testing is rate limits. Rate limits are limits to the number of requests that can be imposed by the application during a time window. While it is advised that rate-limiting is disabled during a scan, if this is not possible, tools allow throttling scans for requests to be sent at a lower frequency. The scan speed may be adjusted from the main Target screen.