How Testing Changes in Component Based Architectures

Imagine building your software like it was ordering furniture from Ikea. You pick out a desk, add a corner piece, raise it up to be a standing desk with some props for the legs, and add small things like organizers to better fit how your work. Things are modular, you can add and remove bits and pieces to build the projects that fits the customer’s needs.

In software projects, this is usually done with a configuration flag that lives in either a system configuration file on the server, or in the database. This flag controls the visibility of all the things in a feature and can be turned on and off as needed. At worst, you might have to restart apache in between. Another way to build components is through a set of APIs. Programmers create the front-end, then mix and match APIs that already exist to create the back-end.

Large software companies use components every day to create experiences for the customer. The Amazon.com homepage, for example, is actually a collection of components pulled into a single logical frame. Google makes its MapReduce algorithm, which can search the web, available as a service to all employees. Combine MapReduce and a street-address-to-latitude-longitude conversion API, and suddenly Google Maps can inform you of not just the fastest route, but what restaurants, hotels, and gas stations will be on your way.

Building with components isn’t just a more powerful approach to software architecture; it also speeds up the whole software delivery process.

Faster Releases Without Back & Forth Between Developers and Testers

In many releases, features or fixes are often added in at the last minute, which could mean one final whole-system check before giving permission to deploy code to production. That also means any previous testing is done without that last piece, which introduces risks of its own. A lot of this is due to features getting to the test group late in the implementation process, or a lot of back-and-forth churn between testers and developers working to fix bugs. If the feature is built as a component, something that can be added or subtracted with bounded risk, the team can flip the feature flag off for the part that is running behind and go ahead and release the parts that are done.

Releasing software faster means you are providing value to the customers faster. It also means that, with a flexible release schedule, you might be able to do away with the classic release death march where a team spends weeks or months discovering and fixing bugs before a product can go out.

The other important thing about quickening the release pace is that if something does go terrible wrong, rather than dealing with rollbacks and database versioning, you can turn the offending feature off. Hotfixes to production under the stress of customers not being able to do anything at all often lead to more hotfixes.

Strategy for Complex End-to-End Testing

In a component architecture, the overall strategy often shifts from simply testing to discover and fix potential problems, to focusing on late detection and reducing exposure to problems when they do happen. There are usually layers of programmatic checks to confirm that the program does what the developers think it should do at the unit level, or the API level, and maybe in the UI as well. But, very little real testing happens.

This strategy can work in some contexts, sometimes, especially when the software is free and the primary users are actually creating data for other people to buy. This is the case with Facebook, Twitter, and to some extent GitHub.

Since there should already be good coverage of lower level functionality, most value can be found in testing end-to-end scenarios. While programmed checks are looking at the same bits of code every time they are run, and are often looking at very simple problems, end-to-end testing performed by a person can explore more complex and varied possibilities that a user might actually do.

End-to-end testing can help discover black swans, those surprising problems with massive implications that most programmatic checks will not find.

Reduced Need for Testing Before Production with Component Architecture

If your strategy has moved from testing and problem discovery to lowering the amount of exposure a problem has, monitoring will become more and more important. Component architectures can reduce the need for pre-release testing, but there has to be a contingency plan in case something goes wrong.

Imagine a scenario where a new type of payment processing is added to a website. The feature is released over night, but night time here is daytime somewhere else and the feature is getting used almost immediately. Somehow, the team missed a critical scenario and when used in a certain version of FireFox, hangs the browser. If you have monitoring set up, notifications will start flying, and the feature can be shut off quickly.

Replicating Customer Configurations for Checking Software Quality

Managing test environments can get tricky with components. Often, there will be a constant stream of commits and deploys to the test environment. In larger companies, this can mean hundreds of people pushing changes all on top of each other all at the same time. Unsurprisingly, this leads to a completely broken environment where it is near impossible to discover which change caused which problem or which two features won’t play well together.

At one client, we had a product where nearly every feature could be turned on or off. Each customer wanted the product configured slightly different. Initially, out strategy was to leave everything turned on all the time but this resulted in problems from features that were not designed to work together. The release schedule was a little slower there, so we were able to replicate key customer configurations for testing.

Organizations that are constantly pushing to a test environment might compromise by creating a special environment using a cut of the latest release version. Testing in production is a cliché at this point, but occasionally it is the only choice.

Why – and How – to Get Started

There are plenty of reasons you might want to switch to a component-based architecture. You may want to recover from disaster more quickly, reduce regression test time, or eliminate costly hardening and coordination time. How to start begins with defining a single service – just one – that will be reusable, along with one consumer for that API. The API should be new development. It rarely makes sense to rewrite old code from scratch, but new feature development is already funded. Once that API is in place the organization can begin the conversation about the registration of APIs, signature, and versions. For now, start with one API, learn something, and see how test strategy changes.

Conclusion

The climb is hard one, but the view from the top is worth it.

Component based architectures are the exact opposite of a monolith. Rather than one large, usually dangerous to change product, you get pieces of a puzzle that can be added and removed. These pieces make for a safer development environment, new styles of testing, and hopefully a better user experience.

How has working with component-based architecture affected your work? We would love to hear your story.