What Makes Test Automation Successful?
We've all heard that automating software testing is crucial in an Agile environment. But how exactly do you get to that point? Read on to find out.
Join the DZone community and get the full member experience.Join For Free
an important but often underestimated part of software development is testing. testing is, by definition, challenging. if bugs were easy to find, they wouldn't be there in the first place (although it should be noted that early in the sldc various trivial bugs might show up as well of course). a tester has to think outside the box to find the bugs that others have missed. in many cases, understanding the business domain of an application is more crucial for effective testing, as is detailed knowledge of the application itself.
in open source projects, quality is typically addressed by contributors and coordinating architects. tests of units, components, and services are often done effectively and automated well. this allows a project to move forward even when many contributions are made. comprehensive automated testing with sufficient range and depth helps keep the product stable.
while some open source projects develop from accumulating contributions of dispersed people, devops oriented projects may follow a scrum or kanban approach that includes simultaneous development and release. this process also relies heavily on the comprehensiveness of tests and their seamless automation. whenever there is a new version (this can be as small as a check-in of a single source file), tests should be able to verify that the system didn't break. at the same time, those tests shouldn't break themselves either, which for ui-based tests is not trivial.
the testing pyramid, proposed by mike cohn in his book, succeeding with agile , positions the ui as the smallest part of testing. most of the testing should focus at the unit and service or component levels. it makes it easier to design tests well, and automation at unit or component/service level tends to be easier and more stable.
i agree that this is a good strategy. however, from what i've observed on various projects, ui testing remains an important part. in the web world, for example, the availability of tools like ajax and angular allows designers to create interesting and highly interactive user experiences in which many parts of the application come together under test. an ultimate example of ui-right web applications is single-page applications , where all or much of the application functionality is presented to users on a single page. the complexity of a ui can rival that of the more traditional client-server applications.
i, therefore, like to leave some more room in the top of the picture, making it look like this.
even for ui automation, the technical side can be fairly straightforward. there are simple open source tools like selenium that can take care of interfacing with the ui, mimicking the user's behavior toward the application under test. tests through the ui are often mixed with non-ui operations as well, such as service calls, command line commands, and sql queries.
the problems with ui tests come in maintenance. a small change in a ui design or ui behavior can knock out large amounts of the automated tests interacting with them. common causes are interface elements that can no longer be found or unexpected waiting times for the ui to respond to operations. ui automation is then avoided for the wrong reason: the inability to make it work well.
let me describe a couple of steps you can take to alleviate these problems. a good basis for success automation is test design. how you design your tests has a big impact on their automation. in other words, successful test automation is not as much a technical challenge as it is a test design challenge. as i see it, there are two major levels that come together in a good test design:
- the overall structure of the tests.
- design of individual test cases.
for the structure of tests, we follow a modularized approach, which is a similar approach to how applications are designed. tests cases are organized in test modules. think of them like the chapters in a book. we have some detailed templates for how to do that, but, at the very minimum, you should try to distinguish between business tests and interaction tests. the business tests look at the business objects and business flows, hiding any ui (or api) navigation details. interaction tests look at whether users or other systems can interact with the application under test and consequently care about ui details. the key goal is to avoid mixing interaction tests and business tests since the detailed level of interaction tests will make them hard to understand and maintain.
once the test modules have been determined, they can be developed whenever it is convenient. typically, business tests can be developed early, because they depend more on business rules and transactions than on how an application implements them. interaction tests can be created when a team is defining the uis and apis.
one other major factor to define automation success is known as testability. your application should facilitate testing as a key feature. agile teams are particularly well suited to achieving this, since product owners, developers, qa people, and automation engineers cooperate. however, open source projects do not necessarily have such teams, and the product owner(s) will have to define testability.
Published at DZone with permission of Hans Buwalda, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.