Contract Testing in HL Tech With Judge Dredd, Part 1

In microservices architecture, it is crucial to ensure that services are able to communicate with each other. What if, by accident, a developer introduces a change which will make inter-service communication impossible? An answer to this threat is contract testing. Judge Dredd, an open source project developed within HL Tech, is a solution to this problem. It aims at performing contract tests between any services deployed within any environment under Judge Dredd's jurisdiction.

Microservices Environment and the Threat of Broken Communication

The typical microservices environment consists of many services. In order to provide business value, they need to cooperate. Therefore, they need to communicate with each other. There are many technologies supporting such communication, synchronous or asynchronous, like REST and JMS. Protocols specify how services should communicate – the format of messages exchanged between both parties is based on the protocol. Specified fields must be present in the message to fulfill conditions specified by the protocol, but they have custom value to provide a context for communication.

Picture 1 - microservices environment

In a dynamic microservices environment each service is constantly subject to change. Services might be developed by different teams, which might not cooperate with each other on a daily basis. What is more, many new services might be deployed on the environment, some might be removed. Therefore, communication between services might be broken. Imagine a scenario in which service 1 needs service 2 to perform a crucial operation. The message format is established and introduced on both sides. If service 2 is changed so that the functionality used by service 1 is no longer provided, service 1 is unable to communicate with service 2 (it knows nothing about this change). Similarly, if service 1 is changed so that messages with different format are sent to service 2, service 2 is unable to process them. Communication is broken as well.

Therefore, the question I would like to answer in this article is:

How can we ensure that changes to any service do not break communication with another service?

Contract Testing as a Solution

Firstly, let me describe how I understand the meaning of a contract. A contract is a set of rules describing communication between services. There are two possibilities: a service that is willing to use functionalities provided by another service is called a consumer; a service providing functionalities to another service is called a provider. Please note that one service can be a consumer of functionalities but can provide another as well. Therefore, a contract is established between a consumer and a provider.

A consumer willing to specify its part of a contract should express its expectations from providers. Expectations describe a reason why the consumer needs to communicate with providers along with a format of requests which it sends and the responses it expects to receive. Each interaction with the same provider must be specified. Expectations must contain information about the providers.

A provider willing to specify its part of a contract should express its capabilities. Capabilities describe what the provider can offer to consumers along with the format of requests it accepts and responses it sends back (if any). Capabilities do not contain information about consumers.

Imagine two simple services: one processing customer billings and one sending e-mail notifications.

Picture 2 – Sample contract

The e-mail service has one capability – it offers any other interested service that it will send an e-mail to a specified destination with any provided data. Then it claims to send back a notification about success or failure of sending the e-mail. It specifies the message format of request and response as well.

Conversely, the billing service has one expectation from the e-mail service – it wants to inform stakeholders about billings using e-mail notifications. It expects to receive a notification of whether the e-mail was sent to all interested parties. It specifies what message format it plans to use as well.

A contract in this example would be an agreement between the billing service and the e-mail service, whereby all conditions are specified, which need to be fulfilled to enable communication between both services and thus enable the billing service to use the e-mail service's functionality.

The purpose of contract testing is to verify if any change made to a consumer or to a provider (both of which are independently developed) which breaks communication between it and any other cooperating service. When expectations of a billing service change so that they do not match e-mail service capabilities, contract tests should fail and prevent a new version of the service from being deployed. When capabilities of a billing service change so that they do not match consumer expectations, contract tests should fail as well and prevent a new version of the service from being deployed. Moreover, in case a service is planned to be deployed on multiple environments, contracts must be verified against all environments before actual deployment takes place. It is possible that there are different versions of the same service deployed in each environment.

Do Contract Tests Guarantee That a Consumer Will Get What it Wants?

Contract tests do not verify whether a provider is able to perform its task. There is a possibility that the contract is not broken, but the provider’s implementation is wrong, and it creates responses with the correct syntax, but they do not provide any actual value. A similar situation happens with the consumer. It might send the correct message, but, due to some incorrect implementation, it may not be able to correctly use the response.

Therefore, it is of importance not to treat contract tests as the only tests which are required to verify if two services are able to cooperate correctly. You need to verify if the provider indeed offers the functionality it promises and if the customers correctly use that logic.

Contract tests should be treated as a subtype of integration tests. They test if one service is able to integrate with another. To verify that functionality is correctly implemented, other types of tests from the test pyramid should be introduced.

Picture 3 – Testing pyramid

Judge Dredd as an Implementation of Concept – Main Features

Judge Dredd is an open source project realizing the concept presented above. It was implemented in HL Tech. It is currently used in the company to globally verify that any change in a service does not break a contract with any other service. Its main features are described below.

Judge Dredd is technology agnostic. This means that it was designed to be easily extendable to support any protocol with any format of contracts with minimal effort. For now, REST communication is supported. Expectations must be provided in Pact format, which is described in later sections of this article. Capabilities must be provided in Swagger format, described in later sections of this article as well. An external comparator is used to compare expectations in Pact format with capabilities in Swagger format.

Adding other communication technologies to Judge Dredd is as simple as adding a new comparator for a new protocol to it, which can compare a new format of expectations and capabilities planned to be introduced.

Judge Dredd must know the expectations and capabilities of all services in all environments. They are provided by each service willing to be under Judge Dredd's jurisdiction. They are kept in a database along with the service name, version, and protocol to which they refer. Judge Dredd must have a full view in order to be able to detect all contracts present in the environment(s). When storing expectations and capabilities their format is not checked, i.e. Judge Dredd accepts any format.

To be able to perform contract testing globally within an environment, Judge Dredd must be aware of the presence of any service in it. Therefore, it integrates with Kubernetes [1], which is a source of information about environments. Details of how this integration works are in the subsequent sections. Judge Dredd was designed to be extendable. Therefore, any other source which provides environment composition information, like Consul [2], may be used with little effort. Information about the environment and services within it are stored in the database.

Having expectations and capabilities mapped to the name and version of each service along with names and versions of all services within the environment, Judge Dredd is able to verify if any contracts are broken within this environment. After verification is complete, a report is generated to show all interactions between services and whether the contract is broken or not.

Judge Dredd exposes REST APIs to provide an easy way to:

• update information about environments.
• publish contracts.
• validate contracts.

Judge Dredd Agent as a Source of Information About Environments

In order to keep Judge Dredd aware of what services are deployed on the environment under its jurisdiction, it must be periodically notified what services and their versions are present on the environment. This is the main role of the Judge Dredd Agent. It must be deployed on the environment. A general concept of how Judge Dredd Agent works is presented in Picture 4.

Picture 4 – updating information about the environment

Currently, Judge Dredd Agent integrates with Kubernetes. Each predefined period it asks the Kubernetes master about what services are available in the environment. The exact names of services and their versions are obtained from the Docker image name of the service. Subsequently, the Judge Dredd Agent sends the information gathered from the server to Judge Dredd using the REST API mentioned in the previous paragraph.

Based on the information obtained from the agents deployed on each environment in the ecosystem, the server has all the knowledge required to perform contract testing.

How Contracts Are Verified During a Continuous Deployment Pipeline

Typically, after changes are introduced into a service, unit, integration, and functional tests are performed.

Subsequently, each service needs to publish its expectations and capabilities to Judge Dredd. It requires the use of the REST API mentioned in a previous section. Currently, Pact's [3] format of expectations and Swagger's [4] format of capabilities are supported. Both must be prepared by the service being processed in the pipeline and included in REST calls to Judge Dredd.

After contract publishing, the next step is contract verification. This step must happen before deployment as well. Expectations and capabilities of the service must be verified against, respectively, the capabilities and expectations of each service communicating with this service. This done in turns for each environment separately using Judge Dredd's REST API

Picture 5 – Contract publishing and verification

The success of contract publishing and verification means that the service can be deployed and communication is secured.

That's all for Part 1! Tune back in tomorrow for Part 2 when we'll wrap up this two-part article by looking into consumer expectations, provider capabilities, and more. It's available here.

Links

GitHub: https://github.com/HLTech

Maven: https://mvnrepository.com/artifact/com.hltech/pact-gen

Docker: https://hub.docker.com/r/hltech/judge-d

References

[1] https://kubernetes.io/

[2] https://www.consul.io/

[3] https://github.com/pact-foundation/pact-specification

[4] https://swagger.io/

[5] https://github.com/HLTech/pact-gen

[6] https://cloud.spring.io/spring-cloud-netflix/multi/multi_spring-cloud-feign.html

[7] https://spring.io/projects/spring-cloud-contract

[8] http://wiremock.org/

[9] https://docs.pact.io/

[10] https://spring.io/projects/spring-restdocs