The Top 6 Barriers to Continuous Testing

continuous testing (ct) is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. ct is an integral part of continuous delivery (cd) and devops practices.

successful continuous testing is a competitive advantage, enabling businesses to deliver value to  their customers with higher quality faster. however, making the jump to ct isn’t easy, and if you’re not aware of the barriers to continuous testing, you may  find yourself headed for disaster.

here are the top 6 barriers to continuous testing:

start-up cost

the initial effort required to pick, setup and configure the tools and frameworks to get to a running automation process can be daunting. the heavy investment for research time and substantial financial investment coupled with the added possibly of interrupting your day-to-day operations have been a big deterrent for many organizations.

test automation coverage is low

in agile development, you should not write more code than you can test with automation. an inability to automate enough tests at speed results in fewer features being developed.

tests maintenance is unmanageable

creating a few dozen automated test cases is easy. maintaining them over time and scaling your test suite to hundreds or even thousands of tests is sometimes, excessive and expensive. maintenance remains the most challenging issue in test automation, especially when app changes happen frequently.

lack of coordinated end-to-end testing strategy

tests can be automated at many layers: unit, service/api integration or gui. unit and service/api integration tests are closer to the code while the gui-based tests validate the system from the user's perspective. not knowing how many tests should be done at what level prevents teams from going fast while minimizing risks.

lack of unified platform for quality reporting and collaboration

not having a unified platform like an alm system makes it very hard to collaborate within a cross-functional team. in qa, a lack of consolidated reports that provide actionable insights means there’s no control over the product quality. ultimately this holds teams up because continuous delivery requires real-time quality assessments to objectively decide go/no-go for a production push.

the myth “you can’t do continuous testing with legacy systems”

many legacy systems weren’t designed with the testability in mind. many features with complex business logics have evolved over the years. another added issue is that different system’s layers chain with each other. all these reasons make automated testing at the code level for legacy systems almost impossible. while unit testing and integration testing can be added incrementally, aggressively building up the full regression test suite from ui level would help the legacy system retain its quality from user’s perspective, but an added level of expertise is needed to implement this.