Functional Increments in Web Services
Functional Increments in Web Services
In this article, learn more about functional increments in web services.
Join the DZone community and get the full member experience.Join For Free
Have you seen code reviews where the code being added is not yet used anywhere in a web service? It's not yet covered with automated tests? It will likely be refactored or deleted in a subsequent pull request (aka PR)? Is a library that is not or only partially used? All of these cases represent non-functional increments to a web service.
A functional increment contains only code that is being used by the service to achieve some functionality. Such code is typically fully covered with automated tests. A functional increment should also be self-contained in a way that it doesn't rely on future changes to expose functionality.
In this article, I will discuss the functional increments approach and demonstrate how it can be used to boost your development productivity as a team, simplify testing and improve the overall quality of your service. While I will mainly discuss Web Services, most of the concepts discussed apply to any software.
You may also like: Creating a REST Web Service With Java and Spring (Part 1)
Layers in a Web Service
A web service is typically composed of three layers:
- The API layer is responsible for receiving an HTTP request and converting it to an in-memory representation that developers can work with, using their favorite programming language. It's also used to send back an Http response to the caller.
- The service layer contains the business logic needed to handle the request.
- The storage layer contains code that is responsible for managing storage (e.g. read/write data to a database).
Building a Web Service Using a Bottom-Up Approach
When building a new Web Service, the most common approach I've seen is a bottom-up approach where the storage layer is implemented first, followed by the service layer and then the API layer. In early stages of development, PRs only contain code that implements the storage layer and corresponding tests. At times, even tests are omitted with an excuse along the lines of: "the tests will come in a separate PR, I'd like to keep the PR small."
Let me first comment briefly on PRs that leave out tests and claim that they will be added in a subsequent PR. Well, it's a terrible idea. Firstly, code is merged to the main repository that doesn't necessarily work. Secondly, it drops code coverage and can consequently cause static analysis tools to fail quality checks. Thirdly, the author can quit his job (or get hit by a bus) and no one is held accountable for adding the tests in question. In the rest of this article, I will assume that we all agree that every PR should include corresponding tests.
Going back to the bottom-up approach (horizontal slices), there are several drawbacks to this approach:
- It's hard to get the storage layer design right up-front and we often end up refactoring it later when we implement the API / Service layer. This means that code that already went through a code review process and was extensively tested is later refactored (and reviewed again ) or thrown away.
- We are often left with dead code because some of the code in lower layers is not eventually used and no one remembered (or dared) to delete it.
- When we implement the storage layer, we often also add integration tests to verify that the storage layer actually works with a real database (or an emulator). We then later implement the Service / API layers and often also add corresponding integration tests to validate end-to-end functionality. This leads to some redundancy in testing the storage layer. In fact, since the API layer integration tests already cover the storage layer, one could argue that the storage layer integration tests are not needed. A counter argument to this is that a class should be self-contained and reusable, which is true in a context of a library but matters less when building a Web Service that is not meant to be shared as code.
In contrast to a bottom-up (or top-bottom) approach, a functional increment typically spans all three layers (a vertical slice), especially when a new API is being added. All changes required to implement the API are added to a single PR. That doesn't make the PR big (which is a common misconception) because it only contains one API (i.e. a thin vertical slice as shown in the image below). For bug fixes or enhancements, the functional increment can span one or two layers.
This approach addresses the drawbacks of the bottom-up approach discussed in the previous section. In fact:
- The API is added in one PR as opposed to multiple PRs. I believe this takes less time to implement, test and code review.
- Since all three layers are implemented at the same time, there is no chance of integration issues between the layers.
- All the code added is used by the API being introduced and as a result, is functional and doesn't need to be revisited in a subsequent PR.
- Reviewers can see the change end-to-end, which makes it much easier to understand the code (as opposed to reviewing code that is not yet in use and trying to guess future integration issues).
- End-to-end integration tests calling the API layer are often sufficient to test all three layers (as opposed to implementing separate integration tests for each layer, which makes some of the tests redundant). This not only reduces development time but also the time it takes to run the tests in a CI build.
That said, in some cases, a bottom-up approach is more appropriate. For instance, at the early stages of a project where the APIs have not been defined yet, developers can start implementing known parts of the system. For instance, if it's known that the service will integrate with a 3rd party service, developers can upfront implement and test the layer that sends requests to that 3rd party service. It's likely that some refactoring will have to be done to that layer at integration time, however, it's still more efficient to implement it up front than to wait until the API is fully defined. There are also scenarios where development work is distributed based on developers' skills (e.g. a developer is more comfortable working with a certain storage technology than other developers), in which case, the development work cannot be divided into vertical slices.
A vertical slice development (i.e. functional increments) speeds up development and code review time, leads to less code, improves quality, and makes testing easier and faster. However, in some cases (e.g. work is distributed based on skills, APIs are not yet fully defined, waterfall design), a bottom-up or even a non-structured approach could be more appropriate.
Published at DZone with permission of Nehme Bilal , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.