Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Contract Testing in HL Tech With Judge Dredd, Part 2

DZone 's Guide to

Contract Testing in HL Tech With Judge Dredd, Part 2

We finish up this two-part article series by exploring Judge Dredd's consumer and provider properties and compare the use of Pact and Swagger files.

· Microservices Zone ·
Free Resource

Welcome back! If you missed Part 1, you can check it out here

Consumer Expectations

Consumer expectations are expressed in Pact format. Pact format specifies sample interactions between a consumer and its provider. All interactions between them must be described separately with sample values, which can be later validated against a Swagger file. In Picture 6, a sample Pact file is presented which was generated based on Judge Dredd Agent.

In this format, the name of the provider, the name of the consumer, and all interactions between them must be provided.

{
  "provider": {
    "name": "judge-d-server"
  },
  "consumer": {
    "name": "judge-d-agent"
  },
  "interactions": [],
  "metadata": {"pactSpecificationVersion": "1.0.0"}
}

Picture 6 – Pact file format

There might be many interactions between services and each of them must be appropriately described. Each interaction contains a description, a request sent to the provider, and the expected response from it, as can be seen in Picture 7.

 "interactions": [
    {
      "description": "publish request; 200 OK response",
      "request": {},
      "response": {}
    }
  ]

Picture 7 – Pact interactions format

A sample request is shown in Picture 8. It must contain a REST request method, in this case, PUT. Path mapping must be specified as well. If there are any parameters placed in the path, they must be provided as well. If the request should contain headers, it is possible to specify them as well. A query string might be specified as well. If a body should be included in the request, it must be put into the proper section.

 "request": {
        "method": "PUT",
        "path": "environments/UAT",
        "headers": {},
        "query": "",
        "body": [{"name": "billing service", "version": "1"}]
      }

Picture 8 – Pact interaction request format

In case there's a response, an HTTP status must be specified. Headers and body are optional.

 "response": {
        "status": "200",
        "headers": {}
      }

Picture 9 – Pact interaction response format

The whole specification of the Pact file format is available in this GitHub repo.

Writing Pact files manually would not be the best idea. Typically, they are created as one of the outcomes of a Pact test. Unfortunately, to generate Pact files, tests for each interaction between service consumer and provider must be written. As the number of services and hence the number of interactions grows, writing such tests becomes a big burden. Automatic generation of Pact files is thus much faster. Additionally, developers don't need to update Pact tests each time interactions between services are changed. Pact Gen [5] is an open source tool used to automatically generate Pact files based on Feign [6] clients.

Provider Capabilities

Provider capabilities are expressed as Swagger files. They provide descriptions of all available endpoints, all possible requests for each endpoint, and a set of possible responses. They also describe the whole REST API of the service. An example is shown in Picture 10 which describes part of Judge Dredd Server's capabilities, which are used by Judge Dredd Agent.

The endpoints are described in the “paths” section. Objects used in the API are described in “definitions” sections. Other sections provide Swagger metadata information like the version of Swagger used to generate the file, etc.

{
  "swagger": "2.0",
  "info": {},
  "host": "localhost",
  "basePath": "/",
  "tags": [],
  "paths": {},
  "definitions": {}
}

Picture 10 – Swagger file format

The “Definitions” section contains a description of all objects and more complex data structures used in the specification of endpoints. A sample is shown in Picture 11. There might be multiple definitions. Each definition contains the type of data structure and its properties. Each property is described by its name and its type.

 "definitions": {
    "ServiceForm": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string"
        },
        "version": {
          "type": "string"
        }
      }
    }
  }

Picture 11 – Swagger definitions format

The “Paths” section contains a description of all endpoints offered by the provider, as shown in Picture 12. Each of them is specified by giving a path under which the endpoint is available. Path parameters might be specified. Their detailed description is provided under the “parameters” section. Apart from this, the HTTP method must be specified, what type of data is consumed by the endpoint, and what is produced as a result of its operation. The “Responses” section specifies all responses provided by the particular endpoint.

 "paths": {
    "/environments/{name}": {
      "put": {
        "tags": [],
        "summary": "Update the environment",
        "operationId": "update environment",
        "consumes": ["application/json"],
        "produces": ["*/*"],
        "parameters": [],
        "responses": {}
      }
    }
  }

Picture 12 - Swagger paths format

The “Parameters” section specifies all parameters used in the path, as a body of the request and any other. A sample is shown in Picture 13. Each of them is described by its name, where is it located, a description might be given as well as a specification of whether this parameter is required or not, and its type. If a parameter is of a complex type, a link to a definition in “definitions” section must be provided.

 "parameters": [
          {
            "name": "name",
            "in": "path",
            "description": "name",
            "required": true,
            "type": "string"
          },
          {
            "in": "body",
            "name": "services",
            "description": "services",
            "required": true,
            "schema": {
              "type": "array",
              "items": {"$ref": "#/definitions/ServiceForm"}
            }
          }
        ]

Picture 13 – Swagger path parameters format

The last part of the Swagger file is the “responses” section, as shown in Picture 14. It contains a description of all possible responses, which are provided by this endpoint. Each response is specified using an HTTP status code and a small description of each status.

 "responses": {
          "200": {
            "description": "Success"
          },
          "404": {
            "description": "Not Found"
          }
        }

Picture 14 – Swagger path responses format

Comparison of Pact and Swagger Files

What Judge Dredd does is give a comparison of Pact files and Swagger files. It is done step by step. The process is shown in Table 1.

Excerpt from Pact file

Excerpt from Swagger file

"method": "PUT",

"put": {

"path": "environments/El2ukVED6J",

"/environments/{name}": {

"path": "environments/El2ukVED6J",

{

    "name": "name",

    "in": "path",

    "description": "name",

    "required": true,

    "type": "string"

},

"body": [

    {

        "name": "billing service",

        "version": "1"

    }

]

{

    "in": "body",

    "name": "services",

    "description": "services",

    "required": true,

    "schema": {

    "type": "array",

    "items": {"$ref":#/definitions/ServiceForm"}

}

"body": [

    {

        "name": "billing service",

        "version": "1"

    }

]

    "ServiceForm": {

        "type": "object",

        "properties": {

        "name": {

            "type": "string"

        },

        "version": {

        "type": "string"

        }

    }

}

"response": {

    "status": "200",

    "headers": {}

}

"200": {

    "description": "Success"

},

Table 1 – Comparison of Pact files and Swagger files

In Row 2, the HTTP methods are verified. Both are PUT methods. The verification result is positive.

In Row 3, the paths are verified. We can see that the structure of the path is the same. The verification result is positive.

In Row 4. the path parameter “name” from the Swagger file is compared against a sample value provided in Pact file. We can see that Swagger specification describes a required parameter of type “string” available in a path. The verification result is successful.

In Row 5, a sample body is given in the Pact file and validated against the Swagger description of the request body. In the Swagger specification, we can see that the body is required and should be an array. In Row 6, there is a further verification of the body. Due to Swagger's specification, it should contain an object with two fields – “name” of type “string” and “version” of type “string.” Sample values provided in the Pact file match this schema. Therefore, verification is successful.

In Row 7, the responses are compared. We can see in the Pact file that for this request an HTTP status of “200 OK” is expected. In the Swagger specification, we can see that “200 OK” is one of possible responses. The verification is successful.

Verification in each row was successful. Judge Dredd judges that the contract is not broken, in this case.

Alternatives to Judge Dredd – Spring Cloud Contract

Spring Cloud Contract [7] is one of available solutions to perform contract testing.

The first required step is to generate Wiremock [8] stubs on the provider side. Each time the provider is changed, tests must be run which create Wiremock stubs of the provider. They must be configured for each interaction with the provider’s endpoints. Subsequently, these stubs must be published to an external server, for example, Nexus. Any change in the provider must result in the creation and publishing of the stubs. Any time consumers run their tests, the provider’s Wiremock stubs must be downloaded from Nexus and the consumer’s tests are run against this stub to check if the contract is not broken.

What are the disadvantages of this approach?

The most important one is that any change in the provider, which breaks the contracts with its consumers, are not detected during execution of the provider’s tests. Any developer introducing a change in the provider, after launching the provider’s tests, must launch all consumers locally and verify their contracts against Wiremock stub, which is kept locally. It is troublesome, especially for many consumers. These tests are manual. Conversely, a tendency is to automate all repetitive processes.

Another problem is that Wiremock stubs of a provider and tests of its consumers must be maintained. In case more endpoints appear, they must be specified on both sides. In case there's any change in the contract, it must be included on both sides. It adds a lot of additional work.

Alternatives to Judge Dredd – Pact Framework

Another solution which has gained attention lately is the Pact framework [9].

The Pact framework offers consumer-driven tests. Pact tests must be provided in a consumer. They are a kind of integration test, but their additional effect is the generation of Pact files, which are used by Judge Dredd and are described in the preceding sections. These tests are run against a mocked provider, generated by the Pact framework. Subsequently, newly generated Pact files are uploaded to the external server. Typically, the Pact Broker is used. It is a tool provided with the Pact framework which offers a place to store Pact's display status of each contract and a map of the dependencies between services. When the provider’s side of contract needs to be verified, the Pact framework mocks the consumer and sends requests specified in the Pact files, which must be downloaded from the Pact Broker, to the actual implementation of the provider.

What are the disadvantages of this approach?

The most important one is that changes in the consumer Pact files, which break contracts, are not visible in the provider. All these changes are, of course, pushed to the Pact Broker, but we would know that the contract is broken no sooner than when the provider’s tests are launched.

Of course, it is possible to specify a hook which will trigger tests on providers, but this approach is much slower. Firstly, the provider must be mocked and integration tests must be run. Later, the provider's integration tests must be triggered which requires the mocking of the consumer. This process must be repeated for all providers of the consumer. So, in order to be sure that everything works fine, we need much more time and some unintuitive workarounds.

Another problem is that the consumer’s Pact tests must be maintained. Any new interaction must be described in tests, and, in case of any changes in any interactions, it must be included in the tests as well.

Roadmap

Swagger is a well-known tool to specify the REST API of a provider. Unfortunately, it is not certain that it describes reality. Swagger annotations are not validated against actual implementations. A scenario is possible that a Swagger file does not correctly describe what a provider can offer. Therefore, a tool which validates Swagger descriptions might be used to describe a Judge Dredd server contract. We are currently investigating the Spring REST Docs [10] framework as a solution to that problem.

We have been working on support for JMS in Judge Dredd with the appropriate expectations and capabilities format.

Judge Dredd has a whole picture of the environment, expectations, and capabilities of the services deployed on it. A good idea would be to use this data to provide a map of dependencies of services under Dredd Jurisdiction with information on where the contracts are broken and where not. This data would be displayed on real-time.

What is worth considering is that Judge Dredd can validate its own contract. Namely, between Judge Dredd Agent and Judge Dredd Server. It would be done by the Judge Dredd Server instance hosted on Heroku.

Links

GitHub: https://github.com/HLTech

Maven: https://mvnrepository.com/artifact/com.hltech/pact-gen

Docker: https://hub.docker.com/r/hltech/judge-d

References

[1] https://kubernetes.io/

[2] https://www.consul.io/

[3] https://github.com/pact-foundation/pact-specification

[4] https://swagger.io/

[5] https://github.com/HLTech/pact-gen

[6] https://cloud.spring.io/spring-cloud-netflix/multi/multi_spring-cloud-feign.html

[7] https://spring.io/projects/spring-cloud-contract

[8] http://wiremock.org/

[9] https://docs.pact.io/

[10] https://spring.io/projects/spring-restdocs

Topics:
microservices ,microservices tutorial ,swagger ,pact ,open source microservices

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}