DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workkloads.

Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Agile Testing: Blending Shift-Left, Automation, and Collaborative Testing Strategies
  • Scriptless Testing vs Scripted Testing: Which One Is for You?
  • Test Automation in QA
  • In-Sprint Software Automation: Revolutionizing Agile Development

Trending

  • How Large Tech Companies Architect Resilient Systems for Millions of Users
  • Unlocking AI Coding Assistants Part 4: Generate Spring Boot Application
  • Java’s Next Act: Native Speed for a Cloud-Native World
  • A Guide to Container Runtimes
  1. DZone
  2. Testing, Deployment, and Maintenance
  3. Testing, Tools, and Frameworks
  4. Five Questions CTOs Ask Me About QA

Five Questions CTOs Ask Me About QA

Five burning QA questions get answered!

By 
Stacy Kirk user avatar
Stacy Kirk
·
Aug. 19, 19 · Opinion
Likes (4)
Comment
Save
Tweet
Share
15.3K Views

Join the DZone community and get the full member experience.

Join For Free

Over the past 10 years, I have consulted with dozens of tech organizations on how to architect high-quality web and mobile applications. I’ve been fortunate to talk with a lot of CTOs, VPs, and Directors of Engineering who have experienced testing challenges and who have asked my insights on their QA function within their organization. These are the top 5 questions asked or sentiments shared:

1. Why Does QA Give Me Such a Headache

Only one CTO was blunt enough to say “Dealing with QA gives me a migraine” but from numerous strained voices of frustration and the rubbing of temples that occur when CTOs explain product quality or testing delays, I think it’s fair to say he’s not the only one with this feeling. So why the headache?

My answer is “Baggage”. If you’ve ever dealt with a person with a lot of baggage, you’ve probably had a few massive headaches. QA is usually one of the last stages before delivery to Production, it carries along with the baggage of all the earlier stages: dependencies, missing requirements, costly setups to mimic Production environment, schedule delays, unhappy teams feeling overwhelmed and undervalued…bag…bag…bag.

2. Do I Need QA or Can Development Do the Testing?

There has been a trend over the past ten years to reduce the number of manual testers and put the ownership of testing on the developers that write the code. CTOs have been very passionate about the level of QA needed to deliver products. The question is, do I need QA if we are a test-driven development organization?

My answer is “It depends on the readiness and willingness of your development team; most teams are not there yet.” It’s like deciding between a new tech stack for Development. You look at how it will make your team more effective, how it relates to the product being built, and the success and willingness of your current team in being able to ramp up on the technologies. That same level of diligence is necessary before making the decision to cut QA.

We have all seen organizations that have decreed, “Developers will be the testers” and quickly cut their QA departments. Well, I’ve seen it work and I’ve seen it fail. I’ve seen QA teams cut and then ramped back up two years later. Just because it works at Facebook doesn’t mean it will work with your organization. There is a cultural transformation to testing that cannot be overlooked.

My recommendation: Do an internal assessment. In my work with several clients who faced this same dilemma, my first step is always to guide them through a QA/Agile assessment which helps them to find a solution that best suits their needs and the organization’s priorities and goals.

One way to kick this off is to ask your development team these questions: On a scale from 1-5 (1 being “Hell No… Not I” and 5 being “Absolutely, I’ve been game since last year”),

  • Do you feel comfortable with fully testing your code?
  • What is your interest in learning?
  • Do you think testing should be done by Dev?

If your team scores three or lower on those questions, you need to either keep your QA team or create a test transition plan at the same level of diligence required to move your organization to a new tech stack.

The transition plan should include identifying QA Champions (from Dev and QA), gaining cultural and technical training & coaching, and pair testing from your QA leaders or consultancy that specializes in these types of testing transformations.

3. What Is the Best Way to Hire Top-Notch Sdets/Automation Developers?

hiring top notch developersHiring top-notch experienced automation developers are hard. Very hard. There are very few that make it more than a couple of years before they are called into the ocean by those attractive mermaids called “Dev”, inevitably leaving their career of QA to become a developer. I was coaxed many times…“Stacy, you are such a good developer, why waste your time with QA?” As I grew my consultancy firm of test-driven automation developers, I experienced the same retention problem.

So what have I recommended? My answer is Build Top Notch.

My Recommendation: Hire brilliant developers with only a few years of Dev experience and pair them with great test automation developers. This will provide them with a mentor excited by a career path in QA; because they are brilliant, they will ramp up faster than the time it will take to fill that Senior Test Automation.

If you don’t have experienced test automation developers in house, I suggest bringing on 3-6 months of consultancy to help ramp up their skills and to build out a strong QA Test Automation Architecture for your team. Many consultancy firms are open to having a part-time ongoing relationship for continued mentorship.

If you are determined to hire a Senior SDETs, understand that their world is a lonely one. Sweeten your benefits to include time off for conferences and training, to attend relevant meetups, and provide subsidized membership fees for test automation groups.

4. The Test Reports Returned Good Results, Why Are There so Many Bugs in Production?

When I hear this question, there is definitely a headache forming, mostly because someone has just called or knocked on the CTO’s door screaming “What’s going on? Our Customer is pissed! Why was this missed? How could this be missed, it is so obvious?”

My answer: It’s complicated. Okay, before you roll your eyes (yes that happens with this answer), oftentimes it’s not the simple answer “Tester Jo didn’t test the feature properly”. There are many nuances from Production bugs from environmental differences, deployment misconfigurations, to inconsistent reproducibility, or missing requirements covered in heavy assumptions.

My Recommendation: Without exception, for every high severity bug missed, there should be a retrospective to carry out root cause analysis of the issue (not just assumption). The investigation team should consist of Dev, QA, Operations, and Product (not just QA). Once the root cause is identified, the team should build a solution and add the required work to the technical debt backlog. As a CTO, you have to ensure that those improvements get priority. This exercise will be uncomfortable at first (but bugs in Production make us all uncomfortable) so please ensure that the team understands it’s not a “blame game” exercise for you. It’s about building better customer experiences, which is everyone’s responsibility.

5. How Do I Raise the Bar of Our QA Capability?

Some QA teams were initially hired to only execute manually. According to PractiTest’s State of Testing Report 2019, over 74% of testers are doing some form of test automation or scripting within their organization. The challenge comes in ensuring that manual test teams can transition to an automation-driven testing model. Other teams have automation but leadership isn’t clear if this saves time, identifies bugs, or increases coverage. As technologies change, many CTOs feel that their QA team is not keeping up or has a clear strategy to improve their technical capabilities to keep pace with their development team. How is this solved?

My answer: It starts with investing in an assessment and road mapping activity.

My Recommendation: The fastest and most comprehensive way is to hire an external firm to come in and conduct an assessment team that will offer comparative data and a suggested improvement roadmap. For example, QualityWorks offers an Agile Test and DevOps Maturity Assessment (ATDMA) that provides gap analysis, industry comparison scoring, and a maturity development roadmap inclusive of process, tools, and training. Here’s a sample of our comparison chart that shows how a company ranks against industry standards based on the results of an assessment:

QualityWorks ATDM

If there’s no budget for an external assessment, you can work with your current team to answer the following question: Where are the technical, process, visibility, and cultural gaps within the QA organization? Once identified, they should prepare a roadmap for improvement.

I’m about to make a generalization about QA…they like to find bugs. Not just in code, but in life. Therefore, if you task them with figuring out the bugs in the QA organization, architecture, and process, you’ll be amazed at what they can present, especially if you have a more experienced team. When my team and I assess organizations, we are always blown away by the readiness of the QA organization to share their own internal bugs and to recommend areas of improvement. They share because they know their information is anonymous. So as their leader, you will need to provide a platform that allows them to be open to share without fear of retaliation or judgment.

Still in Need of an Advil?

I hope this the first step to help relieve a few CTO headaches and to provide techniques for continuous improvements and visibility within the organization.

Question answering agile Test automation dev Testing

Opinions expressed by DZone contributors are their own.

Related

  • Agile Testing: Blending Shift-Left, Automation, and Collaborative Testing Strategies
  • Scriptless Testing vs Scripted Testing: Which One Is for You?
  • Test Automation in QA
  • In-Sprint Software Automation: Revolutionizing Agile Development

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!