Software Quality: A Three-Dimensional View

In the world of consumerism, the primary goal is to satisfy one's needs and desires. These needs find fulfillment in the satisfaction derived from the products or services we receive. Since human wants are diverse and unique, the nature and quality of the products or services are shaped accordingly. Quality, in this context, is determined by the level of human satisfaction it provides. Achieving this quality necessitates a continuous process of monitoring and improvement in the delivery of products or services.

In the software industry, software quality can be categorized into three essential aspects: decision quality, process quality, and product quality. These three dimensions collectively determine whether the software will ultimately satisfy both customers and end-users. Below, we'll delve into each dimension of quality and outline how each is monitored and improved.

Figure 1: A three-dimensional view of software quality 

Decision Quality 

In our lives, we frequently find ourselves at crossroads, faced with a multitude of decisions to make, each with its own set of considerations and potential consequences. These decisions become the building blocks of our life's journey. However, when working within a team or an organization, the process of decision-making takes on a collaborative dimension, where the choices made collectively have a profound impact on the outcomes achieved.

The Three Pillars of Decision-Making Described

In the software industry, decision-making features three key pillars: business decisions, management decisions, and technical decisions. Each of these pillars plays a crucial role in the success of software projects and, by extension, the organizations involved.

Business Decisions

Business decisions require a full understanding of market dynamics, customer needs, and the overarching goals of the organization.

Management Decisions

Management decisions involve evaluating available resources, setting priorities, and charting a clear roadmap for project execution.

Technical Decisions

Technical decisions include the selection of the technology stack, architectural design, and coding practices, among others. 

There are several ways to ensure the quality of our decisions.

The Three Pillars of Decision-Making in Action 

Proper Framing

The key to proper decision-making is framing. Framing helps us to define precisely and accurately the problem we aim to solve. Without proper framing, we may end up addressing the wrong problem. 

To develop an appropriate frame, you can use the following decision hierarchy as a tool (Figure 2). This pyramid structure helps define the scope of the questions to be answered, providing focus and preventing information overload:

Decisions to take as given: These decisions have already been made and need to be carried out.

Decisions to focus on: These are the decisions currently in focus.

Decisions to make later: These can be made later or separately.

In the example that follows, when we first start planning a new software development project, we need to determine the deployment methodology for this particular project. To make this decision, it's crucial that our discussion is framed properly.

“Selecting the best software deployment methodology is essential, especially considering that the project has a licensing period spanning the next five to ten years.”

Figure 2: Decision hierarchy for selecting the best software deployment methodology

In the decision hierarchy outlined above, the pyramid structure guides the decision-making process with distinct levels of focus.

Top Level: Take as Given

• Ensure software supports production traffic for five to ten years.

• Can practice the current deployment methodology (manual deployment) or new methodology: continuous integration / deployment (CI / CD) , serverless deployment, or another methodology.

• Make the decision during the initial development phase.

• Involve all stakeholders (business and technical) in the decision-making process.

Middle Level: Focus On

• Choose a deployment methodology for long-term service (five to ten years).

• Opt for manual deployment with partial automation or embrace a new approach?

Decision Later

At the base level, decisions can be deferred for later stages:

• Improve the security of the chosen deployment method.

• Evaluate the existing resource knowledge level.

• Ensure software sustainability.

Identifying Creative Alternatives

Having a variety of creative alternatives is crucial for making sound decisions. Your list of alternatives should be extensive and diverse to encompass a wide range of possibilities. Good alternatives possess the following characteristics:

• They are novel in some way, significant or minor. 

• They differ significantly from one another.

• They offer a broad range of choices.

• They are feasible.

To identify alternatives effectively, you can use a strategy table as illustrated below (Table 1). Each decision within the problem's focus category becomes a column heading, while the left side lists the strategies associated with each decision. It is the responsibility of decision-makers to define a high-quality set of alternatives before proceeding with further evaluation.

Example 

There was a situation where management made a decision to initiate a new project, resulting in a change to the current deployment strategy and the introduction of a new CI/CD deployment strategy. This decision aligns with the structured decision hierarchy outlined below, characterized by distinct levels of focus.

Top Level: Take as Given

• Adopt CI / CD as the new deployment methodology  

Middle Level: Focus On

• Plan for necessary internal practices renewal.

• Organize knowledge sharing sessions for current employees.

• Introduce the technologies and tools involved. 

Base Level: Decision Later

• Enhance security measures with the new deployment methodology.

• Establish software support level practices 

Figure 3: Decision hierarchy for adopting CI/CD for deployment

Each decision in the problem's focus category becomes a column heading across the top of the table. These column headings are derived from the middle section of the decision hierarchy.

These strategies are associated with adopting CI/CD as the new deployment methodology: adopting CI/CD in-house, considering a third-party deployment service, and contemplating a hybrid approach. 

• If the strategy of adopting CI/CD in-house is chosen, a coherent CI/CD practice needs to be documented, a training plan needs to be created, and the tools need to be identified. 

• When considering the third-party deployment service as an alternative, there is no need for training and tool selection. 

• With the hybrid approach, there won't be significant changes in the current practices and tools. 

In summary, if we can come up with a set of sound alternative strategies for the decisions we need to make, it will help us come to a good overall decision.

Table 1: Strategy table for adaptation to CI/CD deployment

Structuring Relevant Information

Many decisions pertain to the future. To make informed judgments in such cases, structuring relevant information becomes crucial. Decisions about something that hasn’t happened yet often involve assessing probability and possibility. Decision trees (Figure 4) serve as a valuable tool for structuring these types of decisions.

Decision trees represent the sequence of decisions and uncertainties, illustrating possible outcomes and associated probabilities.

Example 

Reducing the count of Customer Reported Bugs (CRBs) and implementing an effective regression testing mechanism, which our organization had previously carried out manually, necessitated several decisions.

To make informed decisions in this regard, it's essential to gather information, including estimates of future costs and revenues, to understand the value of this opportunity. In the scenario below, we need to consider person days (PDs) required for implementing appropriate automation practices as well as potential CRB increases on the project — both of which carry uncertainty. Predicting these uncertainties and their outcomes accurately is crucial. 

A decision tree, such as the one presented below, proves to be an effective tool for conveying decision-related information. It lays out a clear sequence of possibilities, including the entire path of cause and effect, along with associated probabilities.

Figure 4: Decision tree: Probability of success in automating regression tests  

In the decision tree diagram shown above, we contemplate whether to opt for manual regression testing or establish a team for implementing an automated regression suite. 

To arrive at good decisions, practice is essential. In the software industry, where multiple decisions contribute to producing quality products, it's imperative that these decisions point to clear and achievable goals.

Process Quality 

Any mature organization should follow defined processes, monitor their effectiveness, and provide for continuous improvement. Several standards and models have been developed for monitoring processes, including the CMMI Model, Six Sigma Model, ISO 9001 Model, and others.

In the software industry, processes are typically classified as follows:

• Software development process

• Software operation process

• Software maintenance process

• Management process

• Subprocesses within departments such as HR, sales and marketing, purchasing, office administration, and so on

All of the processes mentioned above need to be well-defined in a software development organization and subject to regular audits. The best approach to ensuring solid processes is to conduct quarterly internal audits. Audits help identify areas for improvement within the aforementioned process classifications, and the necessary changes can then be incorporated. 

The Plan-Do-Check-Act (PDCA) cycle is a useful mechanism for achieving this continuous improvement.

Figure 5: PDCA Cycle

The PDCA cycle enables an organization to ensure that its processes are adequately resourced and managed, and that opportunities for improvement are determined and acted on. (ISO 9001:2015 Section: Introduction)

Product Quality

Product quality in the software industry is defined as the degree to which the software meets customer or end-user needs or expectations. Quality is evaluated through the perspectives of the user, the builder, and the manager. In actual user experience, software quality depends on the following characteristics:

• Functionality

• Reliability

• Usability

• Efficiency

• Maintainability

• Portability

In the software development life cycle (SDLC), it is imperative to monitor and continuously update the following:

• Clear requirements

• Architecture design quality (peer review / team review)

• Code quality (peer code review / tools)

• Secure implementation at server and application levels

• UI / UX feedback

• Clear maintenance and operational documentation

To ensure that standards and best practices are applied, we conduct reviews and use tools. Subsequently, we employ the following monitoring metrics to gauge improvement on a quarterly basis.

Table 2:  Monitoring matrix

In a software organization, we typically receive issues and feedback for delivered projects. It is essential to align these updates with the categories mentioned above. Conducting an annual review allows us to evaluate and determine the standards and practices we follow at each stage of the SDLC. This review is crucial for improving code quality. If we are using tools for this purpose, there might be occasions when upgrading existing tools or exploring new ones becomes necessary.

To Wrap Up

We’ve discussed the importance of quality in the software industry and how it can be categorized into three essential aspects: Decision quality, process quality, and product quality. Overall, we emphasize the significance of quality in decision-making, processes, and software products, highlighting the need for continuous improvement in each of these areas to meet customer and user satisfaction.

"Quality is fitness for use." -Joseph Juran