High Availability (HA) is the ability of a system to survive during component failure.  HA can range from a very expensive zero downtime solution with massive physical redundancy to a less expensive solution that accepts some downtime possibly involving manual fail-over of components.  The right solution depends on the industry and its needs and is defined by the ‘class of nines’.  Here are some things to consider around a HA implementation:

Know Your History

Do you have a good handle on what your user base looks like in terms of volume and geography and when peak load occurs on your system? Have you planned for extreme scenarios based on your current peak load numbers and future projections? Do you have good Root Cause Analysis processes in place that triage downtime issues and help you learn from your failures? A good Application Performance Management (APM) tool can help. To determine where best to spend your HA dollars, it’s important to know where the limitations of the current system are.

Set Expectations

The time and money spent on the HA solution should be a joint IT and Business decision. Business wants the best possible solution they can get for the money and it’s the IT team’s job to educate the business on that solution. If a certain downtime is needed during times of failure, inform the business on what it is so they can do the cost benefit analysis of lost revenue or poor perception versus investment in a more robust solution.

Understand Potential Impacts of Social Media

Social media can drive large volumes of traffic to your site based on things like promotions or other viral campaigns. Keep in touch with your marketing department and do your homework early to estimate what the impact of a campaign on your site could be and then build for that traffic. Test for chaos. There is nothing worse than a successful ad campaign that drives  people to the site only to be turned away at the door.

Adopt a DevOps Culture

DevOps is a model that breaks the traditional barrier between IT and Application Development. Modern APM tools allow an IT organization to gather application specific details (in addition to system health) and Application Development experts are needed to analyze that data. Understanding points of failure in the system from an application development perspective can help design a better solution and often this software approach is more efficient than throwing hardware at the problem.

Publish the Application ‘Safety Zone’

Conduct performance tests on your system to understand its current capacity and publish that ‘safety zone’ to support, business and other key groups. The ‘safety zone’ can be expressed in terms of number of concurrent users that the system can support,  number of transactions it can process in a given time or the volume of data that can be loaded without impacting end users. The measures are specific to the system but should be clearly articulated. Once understood, there should be proactive monitoring of the system against the ‘safety zone’ to allow for quick intervention during traffic or usage spikes or other signs of trouble.

Understand Your Infrastructure and Architecture

While an obvious statement, large complex systems are not often fully understood even by the most senior members of the team. Each person on the team might specialize in a certain part of the system with no one having an overall picture. APM tools can help map the system architecture. This view is needed to understand the weak links in the system and the potential points of failure so they can be addressed from a HA perspective.

Download a PDF of the checklist.