2017 was the year bugs ate our computers. While we pondered the implications of machine learning, multiple software bug incidents reminded us that software programs are only as good as the people who code them. And, computers only do the things we define for them. Newsworthy software “fails” have a negative impact on customer satisfaction, public reputation, and the bottom line. In 2017, Tricentis studied a collection of 606 software bugs from 314 companies. They found that those 606 bugs impacted half of the world’s population and caused $1.7 trillion in losses.
In a previous post on requirements based testing, we discussed the dollar cost of inadequate defect management. Today, I’d like to address another aspect of software defects, the intangible — yet highly impactful — issue of brand perception and consumer loyalty.
Software Failure Case Studies
Equifax probably wishes it had spent some time doing defect management when attackers hacked its system in mid-May through a web-application vulnerability. Data, including partial driver’s license numbers and other personal identifiers, was scraped from 143 million Americans by who-knows-who for who-knows-what.
The thing is this, the breach was caused by a bug in the Equifax server code, the Apache Struts web-app, that had published a patch 2 months before the breach. As of March 2018, the number of consumers affected had risen to a staggering 147.9 million people, or just over half of the US population. The giant credit-reporting company suffered a huge hit to user perception of trustworthiness, with shares falling 28% on public knowledge of the breach. Can you say, “brand trust destroyed”?
Case: Nest Thermostat
The beloved Nest thermostat, which allows users to monitor and adjust their home’s environment from a smartphone, left users in the cold — on one of the coldest days of the year. Users vented their frustrations on social media while co-founder and VP of Engineering, Matt Rogers, blamed a software update. “We had a bug that was introduced in the software update that didn’t show up for about two weeks.” he apologized. A small glitch created a huge problem for Nest users, and the reason the problem code wasn’t detected before deployment? Inadequate testing.
Not all bugs are as egregious as these examples, but all will impact your brand. The cost to brand perception is frequently greater than the cost of finding and fixing a bug during development. Users have become notoriously fickle; they expect sites and apps to be responsive, intuitive — and error-free. After all, the only thing that takes less time than downloading software is deleting it.
When software goes awry the resulting poor UX creates brand hostility. But, what constitutes ‘going awry’? Awry could be anything from OS version incompatibility to input field glitches. (Let’s not even talk about slow-load times on restricted networks or downtime due to code errors!)
- 88% of Americans form a negative opinion of a brand if it has a poorly performing app
- 79% of users are less likely to continue purchasing from a poorly performing platform
- 48% of users think poor software performance means the authoring company doesn’t care about their users
- 34% of users will move to a competitor if they’re not happy with your software
Additionally, the amount of personal data collected by today’s apps mean any sort of hack (hello, Equifax!) exposes your customer to a bevy of potential woes, including identity theft and financial loss.
Irritated customers are all too likely to complain on social media for the world to see. From large to small, bugs not found in testing will come back to bite you, literally. So, what’s the SQA equivalent of bug-spray?
Proactive Defect Management Solutions
1. Implement a Quality First Mindset
To combat buggy releases, stop asking, “Are we done testing” and instead ask “Does this release have an acceptable level of risk?” Check out our post on risk-based testing strategy for background. Beyond assessing risk, your dev process (from ideation to deployment) should use a quality first approach. Simply deploying to deploy isn’t the solution.
The trick to creating a ‘quality first’ mindset is to engage all stakeholders (users, indirect users, operations and support staff, developers working on systems that integrate or interact with the one under development, or maintenance professionals potentially affected by the development and/or deployment) in the value of comprehensive testing. Some ways to do this include:
- Work with your stakeholders to identify what they think they want, produce a product which reflects that understanding, then get feedback from them and update to reflect the new and improved understanding
- After release, share stats with your dev team. How many users do you have? What’s the current media status? This doesn’t have to be quarterly stock reports, just a real-time view of how your product is viewed publicly.
2. Involving Quality Assurance Early in the Development Process
Agile’s incremental approach allows for more feedback, flexibility, and of course, testing opportunities. Each time a feature, fix, or function is added or changed, it’s checked for bugs. (Hello, Nest?) While today’s methodologies ask for a shift-left rush to deploy, don’t neglect the myriad test opportunities these methodologies offer.
3. Hiring an Outside Software Testing Resource
Depending on the structure of your company, it’s possible the best defect management solution is hiring an outside team who specializes in software quality assurance (SQA). If your team is heavy on development but light on testing, or you’ve got multiple projects in your dev. pipeline creating a time crunch, it might be worth your time to consider an outsourced software test company. Contracting QA to an external software testing resource allows your team to focus on core competencies while mitigating a shortage of build-specific test expertise or managing deadlines. Read more about software test outsourcing considerations here.
Pay Attention to Software Quality
The truth is that a software bug will cause both direct losses in business revenue as well as indirect losses due to declines in customer loyalty and brand reputation. In the long run, the downstream costs of a hit to brand reputation and customer loyalty often exceed direct costs. Check your metrics, if your subscriber or user numbers are falling more often than rising, you may need to reassess your company’s testing paradigm. Discovering bugs through negative social media reviews or declining subscription numbers is truly a worst-case scenario.