How Do You Test Your Tests
Use processes like test-driven development, low-code testing, and mutation testing to ensure tests are robust enough to catch errors.
Join the DZone community and get the full member experience.Join For Free
Software today has shifted far from the monoliths of the past towards microservices. Microservices have many benefits over monoliths. They are more flexible, scalable and generally enable developers to add features more quickly. Unfortunately, testing cloud platform microservices is more complicated than it was in the monolithic ancestors.
For cloud services, functional, integration, and unit tests are performed by creating a lower-level version of your function and plugging in inputs. The outputs from the function can show how your logic and infrastructure performed during the test.
Are Your Tests Robust Enough?
Each type of testing needed for proper cloud integrations comes with its own set of challenges. One challenge is knowing that your test is robust enough to catch errors or missing logic. A robust test will check that everything that is supposed to be completed in a flow occurs and that the data flows as expected.
Let’s take the following flow as an example. In this set of microservices, an input from an app hits an API Gateway to collect some data. The cloud platform will log the event into an S3 bucket, fetch the required data from an SQL database and return it to the app.
This use case is relatively simple, with only a few moving parts. To produce an integration test, we must check that the bucket write occurs and the data returned through the API gateway is valid for valid input. Developers must write multiple tests to ensure the function handles errors appropriately and that security is set up correctly on the API Gateway.
Just having tests pass is not enough to know they are robust. For example, your test could check that the result of the API gateway request is correct. If the data pulled from the database and passed back is correct, your test will pass. However, this does not mean that the data is written to S3 correctly. The passing test misses part of the compute functionality.
How Do You Know Your Tests Work?
If tests passing alone is not enough to ensure that your tests catch all cases, how can you know they work? The short answer to this is you can’t. However, there are techniques you can utilize to mitigate this risk. Since testing is a requirement for stable platform code, many techniques have been developed over the years to give company’s the satisfaction that what they are shipping to customers works.
Mutation testing is a method of testing your tests by updating the source code. The updates to the source code should cause errors in your system that are found by your tests. If you change how your code functions, your tests should catch the change and fail.
In our example, a mutation could be made to the source code to stop the write to S3 from occurring. If the write to S3 is not checked in the tests, everything would still pass even after the mutation.
Test-driven development (TDD) is an agile software development process. In TDD, coding, testing, and design are tightly coupled steps in building your software solutions.
To develop using TDD, developers first write a unit test for their function. Running this test fails at first because there is no code to satisfy the test requirements. Next, developers write code to make this test pass. Developers repeat these steps until they have built an entire function that meets the stakeholder’s needs requiring the development.
TDD works well for some teams, but others may fall into issues that add to development time rather than reduce it. These include when a team does not run tests frequently enough to spot ongoing issues in their code, when developers write too many tests at once, or when written tests are too trivial.
In the example above, developers could first write a test that ensures that AWS Aurora and AWS S3 are updated when our function runs. Further tests could check for the validity of the data sent to each source, and so on. Writing the tests before and only implementing code to pass tests can ensure that the code meets the requirements in the delivered software solution.
Low-code testing is an approach to tests that use minimal coding. Typically developers knowledgeable in test languages such as Jest or Mocha are required to create a test. Low-code testing uses other methods such as graphical interfaces to test code. Third-party platforms can provide a graphical interface to facilitate low-code testing.
Using a click-and-drag box to represent a general functional need means anyone who understands design can build a test. Testers do not need to understand the languages behind the interface to create a test. Reducing the technical knowledge required to produce a test can improve development efficiency because more people on a team can contribute to testing.
Low code tests also reduce errors in writing tests because they use consistent and repeatable building blocks. When writing a test script, developers can introduce errors into the test code that may not show up. A test with an error could be passing, in which case developers would not find it until an error arose in your production environment. Low-code testing’s building blocks reduce the risk of this error because developers would reuse the same, well-tested building blocks.
In the example above, the tester would create a block testing writing to the Aurora database and another testing writing to S3. The graphical interface would easily indicate to the developer what this function is required to do.
Developing production-grade software is required by any company in the software game. The goal is to get that software as quickly as possible with no issues. Testing software is critical to ensure that issues will not arise in front of clients. Methods and frameworks around how to test have evolved along with software to ensure its stability.
To ensure your tests are as stable as your software, developers can use techniques to validate their tests. Mutation testing and test-driven development both help to ensure tests are working for the software. Developers can use low code testing to ensure bugs do not creep into the tests themselves by using the same building blocks across your entire cloud platform. Mutation testing and TDD are policies you can implement across your software teams. Services can provide low-code testing frameworks to use out of the box.
Opinions expressed by DZone contributors are their own.