At the StarEAST software testing conference in 2017, Isabel walked onto the stage during the popular “Lightning Strikes the Keynotes” session and delivered a presentation on ancient Scottish sheep farming practices. The talk, which could only last five minutes, was informative, witty, profound, and extremely relevant to software testing — all at the same time.
That is Isabel Evans for you. A highly regarded speaker in the conference circuit and luminary in the software testing world, she approaches the challenges of quality assurance with deep insight. Combine that with her thirty years in the IT sector and you get a rare, tour-de-force perspective — one that can tackle the daily challenges faced in QA from a broader, “big picture” position.
If you want to see Isabel speak in person (and you really should!), check out her upcoming events and master classes here.
Chelsea: Can you give a quick overview of your background and areas of interest?
Isabel: I have been in the IT industry since the 1970s. I was a developer in the 70s and moved into testing and quality assurance in the 80s. Then, through the late 80s and 90s, I moved into quality management and began looking at organizational quality as opposed to just software quality.
Since then, I have developed several interests. I am interested in user experience, UX design, and testing, and about how you can take the engineering qualities in software and see how they merge into UX attributes like trust and flow.
Another area that I have worked in is organizational quality, where I explored organization excellence and improvement. IT is really just a tiny part of the picture; it is just a service into the rest of the organization. So how do we, as an organization, improve and build better things?
Within that, I developed an interest in team work and how people interrelate. That was partially driven by the fact that I am not particularly good at teamwork and networking myself. I realized that there are many other people in the IT industry who are not particularly comfortable with that either. It’s a bit of a stereotype, but sometimes those stereotypes are there because they fit.
So, I started looking at ways in which you can work with other people even if you are not necessarily comfortable doing that. A lot of the techniques companies put into place (for example, formal reviews) are about getting people to communicate and giving them a structure within which to do it. How do you develop those structures so that they are flexible and encourage people to communicate in ways that are positive?
All of that has come together into my main interest at the moment: The UX and usability of testing tools for testers. There is a big focus in the industry right now on the fact that test tools are not necessarily easy to manipulate. For a while, people have been saying that testers need to become more technical so that they can fit the demands of the tools they are trying to use and leverage them more easily.
Chelsea: I see what you mean. What would it look like if tools developed to fit the demands of the testers that are using them? What if the tools meet the testers where they are?
Isabel: Exactly. What needs to happen to the toolset to make it more usable? I have noticed that there is research going on around the developer toolkit (saying that the developer toolkit isn’t easy to use). Suddenly, we have this phenomenon where it is acknowledged that highly technical and intelligent developers are attempting to use tools that they don’t understand. For many of these tools, 80% of the features go unused because the developers simply don’t know how to use them.
So, it’s not an issue with testers only; it’s a whole UX problem within the software industry.
Chelsea: It often strikes me how much testing comes down to people. In my observation, technology is all about philosophy and psychology. We get caught up talking about the technical details, but at the end of the day, it is psychology that drives how we interact with those details. It is amazing how even when we are talking about machines, everything is so human driven.
Isabel: In the 70s or 80s, someone famously wrote, “Don’t talk about computer interfaces; all interfaces are human interfaces.” The point was that even if you are talking about something that is embedded into a piece of firmware, humans had to create that interface, and they gave their interpretation to how it should work. That is even truer when it comes to the actual UX of a product.
Chelsea: So if we, as humans, are evolving, then testing must be evolving, too. How have you seen testing change in your career?
Isabel: It is sort of going around in a circle. When I first started in the testing industry, there weren’t many specialized testers. I didn’t encounter specialized test groups for a while. When I first became involved in a specialized testing group, the focus was on exploring the software and then translating those tests into automated tests. We did a lot of automated testing. Then, in another organization, through the 80s and 90s into the 2000s, there was a loss of that automation because it was perceived as too difficult to do. Suddenly, an emphasis on manual scripting and technique arose. Now, we come back into the twenty-first century, when people are starting to talk more about automation and exploratory testing, and less about manual scripting.
At the same time, development has gone from small focused teams working on a specific problem through to big projects with silo working and now coming back to people saying they need more frequent deliveries — essentially, the rise of Agile and DevOps.
I find it quite fascinating that it feels like we are just cycling.
It seems to me like there is a tension between a desire to create in a completely free sandbox and the need to control that space out of a desire to “become serious about it.” The problem is that if you control it too much, you damp down the creativity and people will eventually feel like they need to burst through into the “next thing” open sandbox. It’s a sort of evolution in how people deal with ideas.
With each of these stages that we go through, there is an acceleration — a change in how we do it. People are talking about the fourth industrial automation revolution going on now. As a society, we have gone through four industrial revolutions, and each time, people say this is the time that the machinery will overtake peoples’ craft skills and everyone will lose their jobs.
Much of that fear now revolves around artificial intelligence, but the truth is that it still looks like we are nowhere near being able to relinquish the running of the world over to machines. At some point, if that arises, we may be in a new reality of perpetual leisure. But at the moment, all the automation we see requires human interaction or human direction. How long will that last?
When I was first in the industry, studying computer science, my final thesis was about whether machine intelligence would ever be possible. I focused on the subtopic of natural language processing. Back then, the idea that anything like Siri or Alexa would exist was laughable. It was not that long ago that people thought that kind of technology was simply impossible.
Chelsea: On a small scale, say the next two to three years, how much do you think artificial intelligence is actually going to supplement our testing?
Isabel: I wouldn’t like to make a call on that. When I hear people talking about that, it seems so close to the research and so far from what I see in most people’s experiences. While most people are struggling to do automation at all, artificial intelligence seems far off. And yet I have a feeling that there will be a breakthrough and it will suddenly flip — like smart phones. There was a long period when people didn’t think there would be a market for it, and then suddenly it becomes unthinkable to not have one. When artificial intelligence does become mainstream, I think it will be like flipping a coin. Until that point, it will probably look quite unlikely to most people.
Chelsea: It seems to me that innovation works in a cyclical manner like testing — that you suddenly have this cloudburst of brilliance and breakthrough in a certain place that lasts for a period, and then there is a space, and then there is another burst of ideas. It also seems that the space between those bursts of innovation and creativity has become so much shorter.
Isabel: When ideas move from being crazy to being mainstream, suddenly everyone can do them. Prior to Roger Bannister’s running the four-minute mile, people said that it was impossible. But once he had done it, lots of people began accomplishing the same thing. It’s because they suddenly realized that it is possible, and removing that mental block allowed them to accomplish greater things.
It’s the same with things like DevOps and continuous delivery.
Chelsea: So, when it comes to DevOps and continuous delivery, while the tools and resources are important, a large part of the problem is actually that mental block.
Isabel: For any idea, technical or otherwise, you have early adopters. There is a chasm between the early adopters and the mainstream. Once the idea crosses that chasm and more people are using it, it turns into the norm. On one end of the curve, you have the people who just want the new ideas, regardless of whether it is buggy, and on the other end, you have the people who are never going to change. But once the idea crosses the chasm, that change is quite rapid. With any new idea, the challenge is getting the idea over the chasm. When you see that change happen, you hear the language change from “this is new,this is untested, and dangerous” to “this is proven and everyone is doing it.”
Chelsea: Is this where you see Agile and DevOps being now?
Isabel: I think Agile has crossed the chasm as a concept, and DevOps is just doing that now. Agile is maybe a third of the way through the “innovation curve.” (Editor’s note: See Geoffrey Moore‘s work on Crossing the Chasm for more information on the innovation curve.)
A vast number of people are still doing Waterfall. They are hearing about Agile and DevOps and are getting worried that maybe they should be doing it, but they have not yet adopted it. It may not be appropriate for them, or they may not accept it culturally.
The funny thing is that some people who were the earliest adopters of Agile are now saying that it has morphed into something that is not what they intended. As it becomes more widely adopted, it becomes standardized, there are certificates associated with it... it’s not new and exciting anymore. They are pushing things in place to make it safer, which makes it less appealing to the early adopters who will begin looking for something new. Agile needs to change and adapt in order to be agile.
So when we talk about that tension between the free sandbox and the need to be “serious” about doing something, we can identify the same tension in Agile and DevOps.
Chelsea: In regards to communication and teamwork, do you think Agile and DevOps has been improving that as of late, hindering it, is it just the same-old all over again?
Isabel: One of the things that is interesting about the way that Agile has been implemented in some places is that you become forced to communicate with people on a frequent basis. There are lots of good things about that, but for some people, it is absolutely exhausting. So, do Agile and DevOps increase the frequency of communication? Yes. Do they improve the quality of that communication? Not necessarily. Do they make communication easier? For some people, yes, but for others, they make it more difficult.
When you set up an agile co-located team, you will certainly have shared spaces, but it is also beneficial to have “caves.” Once you go into your cave, you can be left alone and work on something without needing to communicate. That gives you the rest you need to be able to deal with things in a common space. Remembering those human factors is very important because it is just as bad to force everyone to be in constant communication and interaction as it is to force people into silos.
Chelsea: Do you think that, when it comes to DevOps, communication is one of the biggest challenges to making it work?
Isabel: I think it is. DevOps is more than sitting the Development and the Operations teams in the same room. And yet some organizations believe that is all it takes to “do DevOps”.
Here is another question you need to consider in regards to whether communication is improved or not. Suppose you have set requirements that are stable and very complex. The best way to communicate those requirements could be to write them down and then study them very carefully. And if you are dealing with something that is stable and complicated and you have the time to do it, a formal review might just be a better solution than attempting to understand the requirements via Agile story cards.
There can be a “baby and the bathwater” issue with this sort of thing. The principle of Agile is that you are flexible to change how you do things based on the project or requirements. Not all the practices of Waterfall are bad; sometimes, they are the best approach, and it should be within the scope of Agile to realize and implement that appropriately. The challenge is recognizing what is most applicable to the project.
You don’t have to take a standard operation like Waterfall and make it bureaucratic for everyone. And you don’t have to take Agile and say that now we can never have documentation. You can blend it; you can empower people to make sensible choices for the situations that you are in. That is the idea of Agile.