As many companies make the transition to automated testing, the way they manage the growth of testing data needs to scale as well. Spreadsheets and inbox analytic from testing tools aren’t enough to report trends or support quick test reruns anymore, enterprises need more and they need it as quickly as possible. The ability to scale your testing automation within an agile environment will allow your enterprise further growth without every change costing you a disproportionate amount of time. This is how bol.com, a major online retailer in the Netherlands and Belgium, did it.
With over 9 million products and more than 1 million visitors per day, bol.com is a large company that needs the ability to scale quickly and effectively. The functionality of the website is built by a 35 person development team, and in order to safeguard the quality of the site, they build out more than 6,000 automated functional tests in FitNesse to continuously verify correct software operations. On the performance testing side, bol.com uses JMeter.
The bol.com Test Automation (TA) Team manages the quality and functionality of the automated tests, as well as the testing infrastructure, the tests themselves and the reporting. When tests are failing or the test duration increases, the team fulfills a role that signals the responsible development teams. To facilitate this process, the TA team developed an in-house solution that acted as a short-term fix to a long term problem. The poor scalability and manageability of their in-house solution ended up costing them an immense amount of time and money for every change and expansion they needed. With increasing growth and an unscalable solution, bol.com needed a test management system that could quickly integrate with their existing tools and allow them to leverage new tools in the future.
Bol.com adopted XL TestView to provide real-time quality awareness, visualization and test management by handling complex test management functions
- Organizes the existing large test suite into multiple parts that are run heavily in parallel using multiple test slaves.
- Identifies the tests that have an above-average duration and need speed-up actions.
- Provides quick support for re-execution of tests, reporting on a per-team level, and links to other relevant information about the test specification runs.
- Offers extensive reporting and dash-boarding capabilities with trends across teams, applications, and components.
XL TestView is the first test management and analysis tool that enables you to manage complex test setup scenarios by defining, configuring, and executing tests across a full spectrum of test tools. It offers an aggregated overview across all test results by seamlessly integrating with your already existing test tools.
IT Manager at bol.com, Ralph Hopman says “In a smooth and painless process, XebiaLabs replaced our legacy code with a modern solution to run, monitor and analyze our tests. XL TestView fits perfectly in our existing technology stack and allows us to leverage new tools and technologies without having to worry about test management.”
Bol.com’s open source technology stack integrates easily with XL TestView, providing assurance that future technical or usage needs will be supported without any problems – either out of the box or with minimum effort through the tool’s extensibility support.
Finally Hopman added, “The current growth figures of bol.com require flexible support in test automation execution, management and analytics. XL TestView provides just that and paves the way to align continuous delivery with further growth.”