Problems Missed When Only Testing APIs Through UI

UI testing is an important part of quality assurance. Specifically, UI testing refers to the practice of testing front-end components to make sure that they do what they’re supposed to. If a user clicks the Login button, the login modal appears. If they click a link, they’re brought to the appropriate part of the application. With automation platforms, these individual tests can be linked together into workflows and automated. Business-driven development style tests can be created in this fashion. The UI can be tested to see that each individual path that a user may take is functional and that the interface is responding appropriately. Other platforms exist that allow these workflows to be tested on simulated resolutions and devices, ensuring that the user experience is consistent across all possible combinations of browser and device.

API testing lives a layer below UI testing. The UI is fed by these APIs and renders the DOM based upon conditions set by both the user and the developer. These conditions determine the sort of API call that’s made to populate the viewport. When we’re UI Testing, it could be argued that we are indirectly testing the API layer. It’s actually pretty fair to say so. Many of the actions that our UI platform will take will issue API calls. If the DOM rerenders correctly, we can assume to an extent that the API call was successful. The dangerous ground here is the assumption.

Let's use a retail web application as an example. Our UI platform can log in, execute a search for a product, add the product to the cart and check out. Every step in the chain is successful, which is great news. Our site is behaving as it should and we can deploy. A few weeks after deployment, we take a look at our sales numbers and 15% of our products are almost completely unsold. It turns out that those individual product entries in the database have a null field that prevents them from ever appearing in a search. It was an unfortunate data entry error, but there was no validation on the database side that prevented the entering of this null value. The products still appear if a user views all products, which would account for the limited number of sales, but they never appear in a search.

The above problem is one that we have actually encountered at API Fortress, and the unfortunate fact is that a UI Testing platform will never even see this issue. The client was unable to figure out why sales for certain products were disproportionately low. When we tested the API layer, we found that a large number of products failed to meet schema compliance for the API. As a consequence, they could be pulled with an “all products” or “single product” call, but not with a search. As most users leverage a search function to find what they want to purchase, the company was losing sales. Once we tested the API layer, we found the problem quickly. This allowed the client company to fix the database entries and give their customer base full access to their product line.

The commonly held belief that UI testing inherently tests the API layer is built on an assumption. The assumption is that it tests the API layer thoroughly, or even thoroughly enough. This can be a very dangerous assumption for a QA team. Testing the API layer, which typically goes through changes at a slower pace than the UI layer, allows a testing team to first ensure that the data being passed to the UI is formatted perfectly and that the server is responding appropriately. Once this process is done, UI Testing can verify that the front end is working properly. Without testing the API layer, silent errors in business-critical processes can remain silent.