Why Engineering Leaders Must Rethink Testing to Build Reliable Software
Engineering leaders (not just managers) hold immense power to shape how software is built, tested, and delivered. Some of the most impactful conversations I’ve had with engineering leaders have revolved around shifting the narrative on testing and software quality. These discussions often reveal surprising misconceptions and expose gaps in understanding that can make or break a team’s ability to deliver reliable software.
Below, I share some of the most common conversations I have with other engineering leaders, from CTOs to ICs, that tackle important truths about testing and software quality—truths that can help your organization move in the right direction, one step at a time.
The Myths Holding Testing Back
One of the most harmful misconceptions in software engineering is the division of testing into “manual” and “automated,” as if they are in opposition. Testing isn’t about choosing one over the other—it’s about using the right tools for the right job. Think of it this way: A chef isn’t called an “automated chef” because they use a food processor; they’re just a chef leveraging the right tools to get the job done. Similarly, testers use automation as a tool—it doesn’t replace their critical thinking, creativity, or investigative skills.
Why Automation Isn’t the Silver Bullet
The obsession with automating everything has led to an industry-wide anti-pattern: teams with extensive test suites but declining software quality. While automation is excellent for repetitive tasks, not all tests should be automated. Over-automating creates hidden costs, from the time it takes to run large test suites to the effort needed for maintenance. Automation is valuable, but it must be balanced with pragmatic decision-making. Just as no chef cooks every dish with an Instant Pot, no team should automate every test case.
Metrics That Matter (And Those That Don’t)
Misused metrics are another common pitfall in testing. Metrics like code coverage or the number of automated tests often lead teams astray, focusing on quantity over quality. For instance, 100% code coverage doesn’t mean your software is robust or bug-free. Instead, leaders should focus on metrics that truly reflect quality and productivity. Those are hard to find but are worth looking for. I like using DORA’s Change Failure Rate and Bugs Found in Production as tools to guide decisions, not absolute quality indicators. Ultimately, only your customers can define whether your software is of high quality.
Testers as Innovators: Advocating for Users and Preventing Bugs
Testers and testing should not be afterthoughts in the development process. Testers are investigators, experimenters, and user advocates. Their work goes far beyond running test cases—it’s about uncovering risks, improving usability, and ensuring the software delivers value. To help testers succeed, leaders must:
Provide robust tools for testing (for all testing, not just automation).
Encourage a culture of respect for their expertise.
Involve them early in the development process to prevent bugs, not just find them.
Quality Starts with Leadership and Culture
Here’s a hard truth: despite what some may think, testing doesn’t create quality—it identifies where quality might be lacking. Quality is everyone’s responsibility, and it starts with leadership. Leaders must actively foster a culture where quality is prioritized at every stage, from requirements to production. How do you create this culture? By aligning teams around a quality-first mindset:
Ensure all roles—developers, testers, product managers—share ownership of quality.
Make quality a central theme in sprint planning, retrospectives, and team discussions.
Celebrate successes like preventing bugs, not just fixing them.
Now it’s your turn: What strategies have worked for you to foster a quality-first culture? Have you struggled with testing myths or misconceptions in your organization? I’d love to hear your thoughts.