IBM App Connect Enterprise Unit and Component Tests

Integration flows often interact with multiple external services such as databases, MQ queue managers, CICS regions, etc., and testing the flows has historically required all of the services to be available when running tests. This provides a high degree of confidence that the flows behave correctly for the tested scenarios, but the number of scenarios that can be tested this way is often too small to provide sufficient confidence that the overall solution will behave correctly in all (or even most) circumstances.

Unit testing with mocked services is a common solution to this problem in the application development world, but integration solutions may require an in-between style of testing due to the large number of service interactions and the common re-use patterns seen in the integration world. App Connect Enterprise development started calling these “component tests” some time ago: Unit tests that test slightly larger sections of code and (unlike pure unit tests) are allowed to communicate with external services. This article will attempt to illustrate this approach using a database as an example service.

Motivation

The following flow is an example of the style of flow that might benefit from component testing:

The flow accepts HTTP input, validates it, calls a stored procedure in the database, augments the results using another database table, formats the reply, and sends it back over HTTP. Errors must be handled in various places (which adds to the test requirements) and it is quite likely that some or all of the flow is shared with other flows as part of a library of code in order to aid flow development across the organization.

This flow can be tested end-to-end using HTTP-based test tools, and it is clearly a good idea to have some verification that the flow does actually work when connected to live services. However, unit testing provides much faster feedback, does not require a copy of the database to be accessible, and can be run without needing external test tools, so one possible approach is to use a “mock database” instead of a real one, and create unit tests for the various parts:

This approach is much faster to run (potentially runs in a few seconds), can be applied to library code, and the unit tests can be run anywhere (including developer laptops and in build pipelines), so it works very well as long as the database definitions are not changed by other parties. If the database definitions are changed, then the mocks become out of date, and the tests will pass even though the flows will fail when deployed.

This is quite a common problem, as the integration developers are rarely able to guarantee the stability of the services being called by the flows, and so an additional set of tests is needed. These should use very similar technology to unit tests (to allow re-use of skills) so they can be quick to write and run, but should avoid using mocks:

The term “component test” was used in the ACE product development pipeline for this sort of test to mean “unit tests that use external services,” and is distinct from integration testing because component tests only focus on one service: If a flow uses a database and CICS, then there would be separate tests for database and CICS interactions, with the tests only relying on one service. The name “component test” is unfortunately over-used in the software world, but does at least provide clarity if used consistently.

Test Design Principles

This conceptual split allows some useful design principles, summarized as follows:

• Unit tests should test flow code (flow structure, ESQL, Java, etc.) and must be able to run anywhere without requiring live links to actual services. These tests are fast and can be run on developer laptops, and are (when automated) one of the best ways to ensure the code does what the developer wrote it to do, both at the time of implementation and in the future. Having no service interactions also eliminates the problem of having to remember which unit tests need services and which don’t — no need to think, “Some unit tests run anywhere but others do not, so is my unit test failure a real code problem, or have I just forgotten some setup?”
• Component tests are similar to unit tests and are written using unit test frameworks, but are allowed to invoke external services, and as a result, these might not run on a developer laptop without some setup and/or service provisioning. Because they focus on one piece of a solution (a common library, for example), they can catch defects in that piece efficiently, but were hard to construct before ACE v12 introduced JUnit support for flow testing.
• Integration tests are those where the full set of services are used, covering the interactions between inputs and outputs of multiple services along with flow logic and code. These tests are often driven via external interfaces (REST APIs, queues, etc.) to provide whole-system testing, and connect to many different systems. These tests are essential, but using them to test the internal flow code in detail is similar to x-ray crystallography — firing inputs at the flow from the outside, and trying to deduce the internal code characteristics from the resulting outputs. Unit and component tests provide much better support for querying internal state, and provide complementary test coverage.

Pipeline View

When the different tests are included in a build pipeline, the result would be similar to the following:

The unit tests are expected to be run on developer laptops, but are also run in the pipeline to ensure the code works as expected when multiple developers merge code at the same time or when the server code level is upgraded (among other reasons). Component tests might be run by developers, but are also run in the pipeline to test as many of the external service interactions as possible, and then integration tests cover the rest before the packaged solution is deployed to the next stage (which may include manual testing).

Historical Perspective

In the past, most testing of integration flows would have started on a developer laptop (usually with links to real services) before the pipeline is launched, and then continued in the integration testing phase with all services available; there was little automated unit or component testing. One reason for the emphasis on integration testing is that it closely mirrors how the real flows will run: In many cases, the whole point of the flows is that they connect systems together, so it makes sense to ensure that they actually do their job when given real services.

Another reason for favoring integration testing is the historic difficulty in constructing automated testing for smaller components of flows (subflows, individual nodes, JavaCompute or ESQL methods, etc.) in older product releases. While scaffolding flows could be used to achieve this along with careful code construction, the initial cost and ongoing maintenance of the test structures meant that this was not a common practice.

Despite (and sometimes because of) these reasons, integration testing tends to be slower, and due to the large amount of code being tested it tends to be harder to isolate bugs found at this stage; shifting testing further left to component tests (or unit tests if possible) normally leads to faster feedback when defects are introduced into the code, allowing for shorter development cycles while maintaining quality.

Summary

The term “component test” is not new, but the distinction between that and unit testing is hopefully helpful in explaining what integration developers can expect to run on their laptops without setup, and which tests might be expected to cover different functionality.

See also https://github.com/ot4i/ace-demo-pipeline for an example of unit and component tests in a pipeline, using industry-standard pipeline technology to show how to integrate testing into ACE development.