A/B Comparison of Microservice API Between Environments
Deployments of versions of API across environments, regions, and availability zones need to be measured for latencies, deployment issues, and availability.
Join the DZone community and get the full member experience.Join For Free
These days there are versions of Microservice APIs that are deployed across various regions, availability zones, and environments. Not all the parameters for their use, latency, and availability are known during deployments. Development teams, Integrators and deployment teams of the system need to be aware of the best case, worst case, and scalability limits with load and during off-peak hours.
Typically an API is a method call that has various version information identified in the request header and calls may be made referring to the URL across different environments and an API gateway would resolve the endpoint to the service. This process may have unintentional consequences by way of the implementation of the API across environments.
For example, there are maybe 2 versions of a GET API that may have been implemented on a <QA> and an <DEV> environment and also they may support different languages.
Comparing the response of the API when called with load may be achieved with load testing tools like SOAPUI, Jmeter, or Gatling. The user ramp time and continuous load need to be considered when looking for the scalability issues.
Typically Gatling would generate a log that lists the simulation log with details of the test run listed as a Simulation log.
Gatling typically would list profiles for the scenarios that have various loads being added on the server/microservices to simulate resilience and availability. Part of the challenge is to ensure that the Gatling reports may be compared across environments to identify where in the API 'call chain' the delays are. Typically these are the stages of an API call from the Client/Server session:
- API Gateway processing delays - depending on the PUT, POST, or GET call the server would check if the request exists on the server cache(Redis, etc.)
- Cache reload time, sometimes the server does not have the resource ready and would need to fetch the resource link/file which takes time
- Processing time for the controller, this metric has 3 sections related to the controller action based on the API being a GET, PUT, or POST method call. The best median and the worst-case delay need to be factored in to ensure that the processing is uniform or has better access for calls with the most processing delay.
Identifying the Delay chain is useful to ensure that the A/B Testing between deployed environments for the same API or for the different versions of the API on the same environment are comparable.
Gatling generates a simulation log that lists the profile of the load which may be compared and plotted between 2 different runs.
java -jar C:\ProgApps\Gatling\Report\gatling-report-5.0-capsule-fat.jar C:\ProgApps\Gatling\results\qa-20210525124615449\simulation.log C:\ProgApps\Gatling\results\vnv-20210525125152306\simulation.log -f C:\Progapps\Gatling\Report\QA-VnV\Report
Nuxeo Gatling report: https://github.com/nuxeo/gatling-report
Opinions expressed by DZone contributors are their own.