If you didn’t get a chance to join us for our webinar on Wednesday, here’s a recap and some insights on how you can detect app issues during production instead of during acceptance testing.
Paul Bruce, Developer Evangelist, and Nick Sanjines, Systems Engineer, both with Perfecto, took the time to talk about how the industry is approaching app development and testing. There were some technical glitches with the presentation platform but they were able to talk in-depth (and show live demos) about the mandate to fix bugs earlier in the development cycle.
How Much Time Do You Spend Fixing Bugs During Development?
Paul began with a survey asking, “How much time do you think developers spend fixing production bugs?”
Honestly, I was a little surprised (in a good way) that the majority of development teams are spending over 20% of their time fixing issues in-cycle. (Ignore that orange check. That is my vote and I'm in Marketing.) Paul actually cited that a ClusterHQ Survey said that 43% of developers spend 10-25% of their time fixing bugs during production.
The takeaway: If you aren’t testing in-cycle yet, begin your shift left so that you can release better apps faster.
Continue the conversation using hashtag #beyondshiftleft.
Paul reminded us of a recent Facebook glitch that caused devices running a certain version of iOS to reboot. The fact of the matter is that #defectshappen and it’s your job to do your part in ensuring app quality for every user. It’s also your job to keep your app fresh and innovative. That’s why, Paul reminded us, if your developers are only fixing defects from your last release, they are spending less time working on new stories. Fixing defects will slow you down and it won’t allow you to innovate.
Spend Less Time Fixing and More Time Creating
Before diving into the guts of how you can detect app issues earlier, it helps to repeat some of the ways defects are introduced in the first place. Some of these include unclear goals, technical misalignment, fire fighting and lack of visibility and measurement. Tackling these issues upfront will help you improve user satisfaction, make better decisions and give you confidence that you are delivering a solid digital experience.
Let’s face it: time is critical. If you are addressing these issues and fixing defects early, you will save time and you won’t have to pay attention to re-work.
5 Ways to Detect App Issues Earlier
1. Speak the Same Language
2. Write Stable Tests
OK, I admit that “duh” came out of my mouth, but what Paul really wanted to stress is that you can’t take shortcuts. It’s not enough to write code and think you’re done – check out a recent post by Paul: Eliminating Flakey UI Tests to Stabilize Continuous Delivery. Writing stable, AUTOMATED scripts allows you to reduce gaps in object identification, clean set up and tear down procedures, and “soak” new tests before checking them in.
3. Break Monoliths Apart
Big is not better. The goal here is to catch defects earlier and reduce test regression time. Your tests should not define what your development and testing cadence is. However, how do you break up tests? Paul showed a matrix of how you might perform different types of tests, perhaps categorized "job" and by the time it takes to run them. He then itemized the types of tests, such as Unit, API, Integration, to run. Inside Intellij, Nick showed a great code sample of how you can break automated tests up. He broke one big test into six to seven tests in “one class.” You could even go further as your app matures and break down them further. To me, this architecture would make it easier for a developer to perform a unit or smoke test on a certain function or “story” during CI.
4. Test More in Every Build
Better (and faster) feedback not only comes with better tests, but Paul reminds us to consider the array of digital platforms and all the different ways people are using your app. We think of this as real-user condition testing. Basically, they recommended that you perform parallel testing across the devices and browsers that are most important to your users. The important thing here is that they weren’t talking about creating or running MORE tests, it was about running one script across multiple devices such as an Android device running Marshmallow vs an Android device running Nougat. Side-by-side testing will also allow you to test unique conditions such as network speed, what apps are running in the background, and how your app handles interruptions.Nick showed us how you could leverage a cloud-based QA test lab that runs inside your IDE. These tests can easily be run during CI (by a developer), not just during acceptance testing.
5. Provide Context With Results
Get reproducible results. Nothing will have meaning or value if you can’t easily see the results of your tests, or if you can’t share those results with other team members. Make sure you are able to dig into each test and make sure it passed on every platform.
Paul and Nick ran out of time so we didn’t get to hear some of the Q&A, but I have no doubt that Paul will share some of that in future posts.
The webinar offered some great insight into how you can find defects earlier, improve your feedback loop and release high-quality apps. It also showed off the Perfecto CQ Lab – check it out for yourself if you haven’t already.