Functional vs. Non-Functional Testing: Can You Have One Without the Other?

Functional and non-functional tests are the most popular approach to categorizing the different types of software testing. These two categories refer to the very essence of the testing process and what exactly is being tested. There are two things to know about functional and non-functional testing if you’ve never dug deep into these two testing categories before. 

One, the division between non-functional and functional testing is not set in stone, and for some testing types, categorizing them is no easy feat. Two, both functional and non-functional testing are essential for the success of your software testing project, albeit in different ways. Today, we will take a closer look at the difference between functional and non-functional requirements, where these two types of testing stand in the software testing process, and how they influence the cost of testing.

What Is Functional Testing?

According to the ISTQB, functional testing is a type of testing that checks the functionality of a component or a whole system. To put it simply, functionality testing helps us make sure that the specific function of a software product is working exactly as intended. For example, if we compare a software product to a vehicle, a functional requirement would be its ability to travel from point A to point B. In other words, this is what the product is expected to do flawlessly every time you want to use it and the reason why you are using the product at all.

What Is Non-functional Testing?

ISTQB defines non-functional testing as a type of testing that deals with the components of the system that are not related to its functionality. The scope of non-functional testing can be downright endless and very much depends on the specifics of the product. Non-functional requirements have more to do with the way the product works for the end-user rather than the expected result. To continue our car analogy, non-functional requirements can be anything from the maximum speed a vehicle can achieve to the number of steps it takes to find the necessary station on the car radio.

Why Is There a Distinction Between Functional and Non-functional Testing?

The distinction between functional and non-functional testing is there not because these two types of testing exist independently from each other. It’s also not there because it matters too much to the QA engineers. For most of them, what matters is the result of testing, a software product free of bugs, vulnerabilities, and inconsistencies.

“Since we are testing products intended for the end user, we care a lot about the way the product is received and interacted with by the user, which, in turn, gives the client the desired results.”

Igor Kovalenko, QA Team Lead

The biggest reason there is any distinction between functional and non-functional testing is the client’s convenience. With a clear approach to categories and groups, it’s easier for the clients to opt for the types of testing that are essential for their project and opt out of the ones that are not particularly relevant to their needs.

Functional Testing vs. Non-functional Testing: Key Differences

Which Types of Functional and Non-functional Testing Are the Most Vital Ones and Why?

There are dozens of functional and non-functional testing types, and each one is important in its own right. We will not go out on a limb and say that some types of testing are more important for a project than others because there is no universal set of tests applicable to every single project. Still, several types of functional and non-functional tests are performed especially often and, when executed right, have a particularly noticeable effect on the quality of the final product. Here are some of those testing types.

Functional Tests

Functional testing techniques help the QA team understand whether the product conforms to the developers’ initial requirements set for it. There are multiple types of functional tests performed every day, but these are three of the most common ones.

• UI testing. In most cases, the graphical user interface is the only way for the audience of your product to interact with it. A faulty interface is a fatal obstacle that prevents the users from accessing certain features of the product or even using the product altogether. There are dozens of different UI aspects that need to be tested, from functional validation to spelling.
• Unit testing. With the help of unit testing, a QA engineer can get as precise as possible when testing a newly created or recently updated software product. Unit testing deals with the smallest code fragments and can be manual for maximum precision or automated for maximum efficiency.
• Security testing. Security in a software system has always been important, but its importance reached an all-time high with the start of the COVID-19 pandemic. With stakes at an all-time high, companies cannot afford any security risks in terms of data safety and brand loyalty. Some companies specialize in security testing, but it can also be done as part of a functional testing package.

Non-functional Tests

Non-functional testing deals with the way the system operates as a whole, not the specific functions it contains. Of course, every non-functional aspect of the system is important in its own right, but some types of non-functional testing are appropriate for any software project:

• UX testing. This type of testing helps the developers understand whether the product is easy for the end-user to operate. UX testing is often mentioned together with UI testing, but they operate on different levels of the user story. While UI testing makes sure that all aspects of the interface are there and work properly, UX testing ensures that they all make for a satisfying user journey.

“UX and UI testing are different types of testing and they are not used together. UX is a non-functional type of testing while UI is functional. There is no type of product where you can skip these types of testing. Even when the product doesn’t have a user interface, the end user is still going to interact with it in some way, which is why both UX and UI testing is essential. And they are not interchangeable either. For example, an app can have a button that looks normal from the UI standpoint, but it appears too small on certain screens. And when the end user is mostly going to use the product on those screens, the UX is going to suffer.”

Andrii Nikitenko, QA Team Lead

• Performance testing. Performance testing is both a type of testing, which analyzes the application size, speed, and reliability of a software product, and a group of functional tests, which may include stress testing, load testing, configuration testing, spike testing, scalability testing, and several other types of testing. These non-functional tests ensure stable performance under any circumstances.
• Compatibility testing. Given the variety of platforms your software product can be used on, compatibility testing is one of the most essential types of non-functional testing to consider for a project. The QA team will test your application against different types of hardware, software, mobile devices, operating systems, and their versions, browsers, and networks to ensure consistent performances throughout them all.

Types of Testing That Are Sometimes Overlooked but Are Still Very Important

In our experience as a software testing company, there are two primary types of clients who request QA services. The first type depends entirely on the expertise of the QA team at every stage of the process, including the choice of testing types and the scope of testing. The second type has a set of specific requirements and testing types that need to be done. However, there is often a risk of some essential types of testing missing from the test plan in the second case. These are the three types of testing that are sometimes overlooked but don’t deserve to be:

• Installation testing. A lot of attention is paid to the way the user interacts with the product, but it starts way before the application is launched for the first time. To be precise, it starts when the application or another type of software product is installed on the user’s device. And the fewer obstacles the user will encounter along the way, the more satisfied they are going to be with the product.
• Localization testing. It is very rare that a software product, such as a website or an application, is intended for use only within a single country or region. The more parts of the world you are going to cover with the release of your product, the more you need to care about proper localization. It involves anything from the correct time and date format and currency settings to complying with each state’s laws and regulations.
• Integration testing. In the hierarchy of system testing, unit testing comes first, then comes integration testing, and system testing comes last. Of course, no one is going to overlook unit testing to test the individual components of the product or system testing to see how they all interact with each other. Still, it’s integration testing, which deals with software modules pulled into groups, that is sometimes mistakenly treated as an afterthought.

Functional Testing and Non-functional Testing: Where Do They Stand in the Testing Process?

Creating a detailed test plan and designing a testing process is one of the ways to ensure that the whole testing endeavor goes over smoothly and that no functional and non-functional aspect of the system goes untested. But where do functional and non-functional testing stand in relation to the testing process?

One of the key things to know about the testing process, in general, is that there is no universal approach that fits every project and scenario. Instead, in every new testing project, the engineers take into account dozens of different factors, from the scale of the project to its target audience.

In our 20+ years of experience as a software testing company, we begin every testing project by reviewing the documentation that came along with the code. The documentation alone should tell us enough to help us design the testing process correctly and make sure that it’s not missing anything important.

However, in some cases, the documentation is missing or is insufficient. In that case, our QA engineers will perform exploratory testing first with the help of exploratory testing tours. It is typically viewed as a standalone testing technique and is not included in the functional or non-functional testing categories. It helps the engineer map out the testing process and decide on the product’s functional and non-functional requirements that need to be tested. Alternatively, we can perform checklist-based testing when the software product has relatively standard specifications and architecture.

Once exploratory testing yields sizable results, the engineer can see the big picture and the specifics of the upcoming testing process. Typically, we will test the functionality of the product first and then move on to its non-functional aspects. However, that is also not the only correct way of doing things. It is possible to test some non-functional aspects of a product during functional testing or in the exploratory phase. For example, an experienced QA engineer can already spot usability and UX issues of a product while still defining the scope of testing.

Change-Related Testing and Its Importance

While the most common approach to categorizing software testing is dividing all testing types into functional and non-functional, many professional QA engineers now distinguish a third category of software testing — change-related tests.

As evidenced by their name, change-related software tests take place only after many changes to the code. This includes smoke testing, sanity testing, regression testing, and retesting. However, from the perspective of the software testing process, change-related testing can take place at any stage, both after changes to functional and non-functional components of the product.

Performing change-related testing — most importantly, regression testing — can potentially delay the release of your product for another few weeks. However, it’s one of the most crucial aspects of a well-tested product. Change-related testing helps you ensure that neither functional nor non-functional aspects of your product have been negatively affected by the recent changes to the code and that no old bugs have reappeared without being detected by timely regression testing.

Functional and Non-functional Testing: the Aspect of the Cost

Whether you are developing a software project from scratch and will need to test it eventually, or you are getting ready to release a recently completed product and view software testing as the final frontier between an unfinished and finished project, the cost is important.

How Do Functional and Non-functional Testing Factor Into the Cost?

For most software product owners, the cost of the testing project is the next big concern after the quality of the testing services. With so many resources being poured into the development process, most developers and product owners cannot afford similar expenses when it comes to testing. This is why they often wonder about the effect of functional and non-functional testing on the overall cost of the testing project.

The important thing to understand here is that the budget for your testing project is not defined as much by the functional and non-functional tests that need to be performed as much by the cooperation model you’ve chosen. With the exception of a few standout types of testing, including security testing, UX testing, and performance testing, QA engineers don’t charge more for specific types of tests. That is why there shouldn’t be a distinction between functional and non-functional testing when budgeting your project.

Factors Influencing the Cost of Testing

Besides the most obvious factors affecting the cost of the testing project, such as the cooperation model you decide on and the scale of the project, there are a few other factors that directly influence the overall price. These are the five of them featured in one way or another in every testing project.

• Size, seniority, and location of the team. The choice of a QA team is one of the most crucial aspects of setting up a new testing project, including its cost. The logic behind the team size and level of seniority is simple, but the team’s location is also very important. For example, hiring a QA team in Ukraine will cost you considerably less than the same team in the US or Western Europe.
• Functionality complexity. You don’t need to be a software engineer to understand that the more features an application has, the more resources it requires for thorough testing. For example, performing testing on an e-commerce website with thousands of products will always be more expensive than testing an application with a single core feature, such as an alarm clock.
• The complexity of UX and UI. Your product may have the most groundbreaking functionality, but it’s not worth much when paired with an interface that is impossible to navigate, riddled with bugs, or looks and feels outdated. The more UI elements your product has and the more ways there are for a user to interact with the product, the more complex and expensive this part of testing will be.
• Amount of manual and automated testing. Both the amounts of manual and automated testing influence the cost of a software testing project, but in different ways. The consensus among many QA engineers is that while testing automation is more expensive to set up, it typically proves to be more cost-effective in the long run since the tests can be reused. In contrast, manual testing needs to be performed by hand, so its cost doesn’t change much as the project progresses.
• Amount of functional and non-functional testing. The scope of functional testing includes way more than testing the product’s functionality. The amount of unit testing, system testing, and user acceptance testing will affect the cost of the project. The same goes for non-functional testing. The more thorough you want to be with security, performance, compatibility, and other types of non-functional testing, the more your project is going to cost.

“The duration of the project depends on the product complexity, on the clarity of the requirements, on the level of communication between the team, the readiness of the client to answer questions, project size, technology scope, variety of platforms, and the number of screens. The cost of the project depends on similar factors.”

Andrii Nikitenko, QA Team Lead

ROI of Functional and Non-functional Testing

Return on investment is somewhat of a controversial topic in the software testing community. Some say that there is no definitive way to calculate the ROI of software testing, while others go as far as to claim that ROI in software testing is a myth.

As is often the case, the truth is somewhere in the middle. Unfortunately, calculating the return on investment of your software testing efforts is incredibly difficult: you can know for sure how much your testing project costs, but you can’t know exactly how it influenced your profit.

On its own, neither functional nor non-functional software testing does not generate profit. But with the help of timely testing, you can increase your profit by preventing software bugs, security issues, usability problems, and other factors negatively affecting your product’s position in the market.

So is there a difference between functional and non-functional testing in terms of ROI? If we look at the place of these two types of testing in the testing process, we will see that functional testing often takes place earlier than non-functional testing. And we know from Boehm’s law that the earlier bugs are spotted, the fewer resources are spent on fixing them. Plus, functional bugs are often easier to locate, especially when there is enough documentation, to begin with. All of this makes functional testing noticeably cost-effective.

Non-functional testing, on the other hand, typically happens at the final stages of the software product development or sometimes even after its release. At this point, fixing a bug takes more time and money significantly. Moreover, non-functional bugs are often harder to spot because they deal with the finer aspects of the software product’s operation. 

However, you should always keep in mind that non-functional testing should not be viewed as secondary to functional testing. These testing types directly impact the reception of your software product, so even in situations where the ROI is different from your expectations, it’s not a reason to limit your testing efforts.

Can You Skip Some Types of Testing to Save Money?

There is no standardized set of functional or non-functional tests that need to be performed on the product. In every situation, the scope of testing is defined by the factors we have talked about earlier in this article. However, QA engineers run both functional and non-functional tests on a software product in most cases. This is done to make the testing process comprehensive and effective, which will result in a thoroughly tested product with a minimum probability of failure.

If we are talking about an abstract software product, then yes, it’s sometimes possible to scale down the scope of testing if you are strapped for resources. However, your testing project still needs to include both functional and non-functional tests. 

You can only skip non-functional testing entirely when you are developing a product that is not intended to be interacted with by regular users, such as a utility software product with no user interface. And even then, you will likely need to test its performance, installation, ability to withstand different loads, and other qualities that factor into a software product’s success.

Why Non-Functional Testing Is as Important as Functional Testing When It Comes to Investing Your Time and Money

The importance of functional testing is usually not a subject of much debate since it deals directly with the product’s intended functionality — the exact things it was designed to do. However, for those who are new to software development and testing, non-functional testing may not seem as essential. Moreover, since non-functional testing tends to be resource-intensive, product owners often wonder whether it’s a worthwhile investment.

The bottom line here is simple: a software product with insufficient or missing non-functional testing will never perform as well in the market as a product where every non-functional aspect has been tested down to the smallest possible detail. So your job here is not to try to cut down on non-functional testing in an attempt to spend fewer resources but to find a QA team that is a perfect fit for your needs and budgets.

“In this day and age, there are hardly any industries or types of products where you could treat non-functional testing as an afterthought because every new feature of the product is designed with the end user in mind.”

Igor Kovalenko, QA Team Lead

Final Thoughts

Dividing tests into functional and non-functional is only one of the possible approaches to categorizing software testing, but it’s a pretty important distinction. If there is one thing we tried to achieve with this article, it’s to show you that functional and non-functional testing is crucial for the success of your software project. 

If you want your product to work according to the functional requirements and be well-received by the audience, skipping either type of testing or minimizing their importance is a decision that is eventually going to backfire. On the other hand, designing a comprehensive test plan tailored both to your project’s specifications and your company’s internal resources is a strategy that will always pay off.