Cloud load testing continues to be an increasingly discussed and important topic. Nearly everywhere I go, questions arise as to what, why, how, and when these capabilities are important. Perhaps you are looking for a quick read to understand how to start defining a Cloud Load Testing strategy for your enterprise. That is what I am setting out to do in this blog.
1. What Are You Delivering to Your End Users?
As an end user, I always wonder, why is it that organizations want me to find their defects? I'm not sure if you have had this same experience, but surely you have, and I am sure your experience is equally as frustrating, if not more.
So as an organization, we need to think about "What are we delivering to our end users?" because if we don't, we can be sure our competitors are. Most successful organizations with longevity have a best intent to deliver valuable products and services to their end users at a fair price.
Thinking of and recognizing the results they are delivering against this "best intent" is often not done. This limits the continuous assessment and feedback of the market and performance of their products/services in the hands of their end users.
So, what are you delivering, how do you get feedback, and how are you continuously improving your end users experience? The reasons I ask this question is simple; what if you had a way to capture these results from production, then replay them across your pre-production environments, getting results you could use to optimize your applications for maximum value to your end users? Now won't that be really good for the business?
2. Why Do You Spend Cycles Proactively Ensuring it Works?
Consider your position in the market, what the customers are saying, and how your competitors are constantly improving. Do you even stand a chance if your strategy is status quo? I think not, and now is the time to figure out how to pivot and proactively make a difference, or else your book will end at Chapter 11.
With the Olympics and holiday shopping as two large upcoming events, many organizations see these as make or break and are seeing this as a reason to spend cycles proactively ensuring their systems are working. Of course, with these events (and others like them), volumes are going to be extraordinarily high; which if your business can hold up to the increased demand, there is much to be gained in a number of ways. Do you measure any of these ways today?
Perhaps you have ways to correlate flows, transactions speeds, downtime, user sentiment, employee satisfaction, and the like to that of revenue, brand value, competitive advantage, customer acquisition and retention, employee satisfaction, and user sentiment. In today's organizations, this is typically not the case, and it's an area that can result in exponential returns.
Of course, perhaps you are thinking, "Yeah right, Todd, easier said than done." Wouldn't be the first time I have heard that, by the way. My suggestion, let's start with one flow; perhaps measuring the correlation between transaction response time of a specific page on your site or app, and that of revenue related to that system. I believe when you start to look at the trend and find quick ways to improve that transaction response time, you will be shocked to see the change in revenue — even if it's only 1%, what does that mean annualized for the organization and you? (That answer is often measured in millions.)
3. How Are You Maximizing Value, Efficiency, Cost, and Time?
Everyone opens bananas differently. Is there really a right or wrong way? The objective is to get to the edible fruit and nurture your body with vitamins and other goodness. Whether you start with the stem or the stern, the shared goal is the same; there is an easier and better way — how are you doing it?
In this metaphorical reference to proactively working to maximize value, efficiency, cost, and time, there is a very strong relationship. Again, most, if not all, organizations with paying customers are thinking about these factors and working proactively to bring the best end user experience.
Along this path, whether primate or human; we are learning by doing, observing the results, then repeating. Organizations and individuals are evaluated on how well they maximize the value of investments, along with efficiencies being built-in to lightweight processes, reducing the costs to an appropriate level, then spending only the amount of time needed to maximize the overall equation.
In a recent poll, 58% of individuals opened the banana at the bottom. This is the same behavior observed by primates. Not saying there is a right or wrong, just a majority. Where do you count yourself, and why? Is it because of how you are doing something that could be done better another way?
4. When Do You Focus on These Capabilities?
"Always, at the beginning and throughout."
That’s what I wish I heard.
Sadly, when the topic of working proactively to ensure a good performing experience for your end users arises, it is often something raised after a massive production incident of some sort.
Another common topic that is brought up has to do with "highly skilled resources" and/or "scheduling to have a scenario run" as excuses as to why this has been deprioritized and not done. Seems this is another huge limiting factor to when people can run these types of tests and get the results they need so as to improve the end user experience proactively and earlier.
I don't know what your personal experience has been, but when "it’s hit the fan," most CIOs and business leaders are NOT saying, "Glad we didn't waste time and effort thinking about performance for our end users." It is often, "How will you prevent this from happening again?!" When this happens, it is a great opportunity to have a prepared response so you can appropriately address both the immediate concern and the future for the organization and your end users.
In short, this consideration is something that should be built-in from the beginning with accountability and responsibility across the technology and business teams to the end users. Making this a reality is a bit of a cultural adaptation, but having visibility on the impact this has on your organization enables you to leverage this tipping point for the moment for the beginning of a transformation.
In the four (4) above points, we can see why the capabilities of cloud load testing would be important to ensure that the end user experience is optimum. There are a number of reasons why organizations find cloud load testing as an easy to adopt and having high value. I have outlined five (5) of these below.
- Burst: Leveraging cloud load testing as a way to supplement your existing load and/or performance testing capabilities. For example, perhaps you typically run a 1,000-virtual-user test, but for an upcoming event, you want to ensure you can handle and tune for a 5,000-virtual-user scenario. Rather than purchasing an additional 4,000 virtual users, simply leverage cloud load testing to drive additional traffic to your application under test for only the required period.
- No Infrastructure to maintain: By using the capabilities of cloud load testing, you are leveraging the cloud to host your load generators and engines, so maintaining a separate infrastructure for this capability is no longer required. Build and maintenance of these core infrastructure components are the responsibility of the vendor you are using, as they are responsible for delivering this as a service to you. In some examples, this is a significant savings of capital and operational expense.
- Integrations: As this capability has been further adopted broadly over the years, so has the demand across the varied stakeholders. As a result, there has been the flurry of integrations, which now makes it very simple to schedule, execute, and report on cloud load testing results. This is because now it can be triggered from any number of locations, including IDEs, build systems, quality reporting systems, load and performance tools, and more.
- Run concurrently with existing load tests: Scheduling and shared resource models have increased the complexity and reduced the availability of load- and performance-testing resources (people and tools) in the past. Now with cloud load testing, this capability is available exponentially, and can be triggered by anyone provided with access to do so. This capability, along with lifecycle virtualization, has enabled teams and organizations to introduce continuous testing scenarios with an extremely reduced costs model.
- Adding on supplemental testing types: When driving fast and adding value, often others will turn to you for considerations on how to help solve for other challenges. One of these I have often run into is, "If you can do this with performance, can you also add: functional, security, usability, etc.?" Well, the answer is yes. And, quite frankly, I wish more people would think this way. It makes you ask some hard questions about what your production environment is like, and how your end users will experience life with your capabilities. Just imagine what it would mean for your organization and business if you could add performance, functional, security, and usability all as capabilities under a single test with a single result that runs continuously. The time is now!
Blog: How Cloud Load Testing helps you deliver amazing software
Trial: StormRunner Load (trial)
eBook: 2015-16 State of Performance Engineering
Book: Effective Performance Engineering Book