5 Myths About Software Testing & QA That Need to Die
One of the definitions of the word myth is ‘an unproved or false collective belief that is used to justify a social institution.’ As such, the word has become synonymous with the notion of a false idea that we tell ourselves to avoid confronting a more complex truth.
Unfortunately, this is a common aspect of the industry’s concept of software quality.
Software quality has two major components that are both lumped under the term ‘quality assurance’. One of these is quality control, testing code to see that it performs as intended without defects and unwanted side effects. The other is quality assurance (QA) which is honing the development process to avoid creating defects in the first place.
Given this elemental confluence of two different functions under one term, there are some additional ‘myths’ that need examination.
Myth #1 – Only Software Testers are Responsible for Product Quality
Decades ago when software development and quality assurance were performed by two strictly separated business entities, code was created by one group and tested by the other. These two functions did not mix. There was even a popular phrase of ‘throw it over the wall’ denoting transfer of a code system from one of these groups to the other. Separation of testing from coding was an intended and much sought-after goal.
The truth is, that quality has always been everyone’s job. The ‘assurance’ in QA happens at the level of writing each line of code. Each developer needs to be writing code with goals of both functionality and testability. Software test personnel and code developers should be working side by side in continuous communication with each other. With the advent of Agile and DevOps, has come a desire for ‘quality is everyone’s job’ to become a reality rather than a cliché.
Myth #2 – Bug-free Software
Like most value judgements, software quality falls across a spectrum. Trying to reduce it to a binary ‘good’ (totally bug free) or ‘bad’ (any defect at all) does a disservice to the development and testing processes. Like all work activities, testing has associated costs and they increase with the number of hours consumed. A simple code system might be rendered bug free by enough testing and repair but the cost could well exceed the profitability of any possible system sales.
Microsoft created the concept of the escape rate. This is the number of defects per thousand lines of code that are detected in a released product version. The very concept is an admission that perfection is an unachievable goal. A useful goal to strive for is to reduce that number as much as possible, to drive it toward the ‘good’ end of the spectrum.
Leaning into the assurance of QA will provide focus on the process of software creation and inevitably reduce the escape rate. This is what Agile and DevOps push for in ‘driving QA to the left’ side of the development flow chart.
Myth #3 – Automate All Tests – Manual Testing is Obsolete
Test automation is another operational concept that has suffered from too much binary conversation. It has been hailed as the solution to the bottleneck of functional verification and condemned as an endless sinkhole for development talent that could be producing value-generating code. Once again, there is a spectrum of value to be gained from test automation.
There are test processes that are repetitive, require large scale environment setups and have many minutely detailed steps that beg for automation. Exploratory tests where each test result is used to design and guide the next test are where manual testing shines and automation costs way more than it is worth. While DevOps depends on automating as much test work as possible, user acceptance testing requires a human perspective that cannot yet be economically automated.
Myth #4 – Anyone Can Be a Software Tester
There is a belief in the software industry that, given a sufficiently detailed test plan, anyone can execute it to perform effective test work. This concept also exists upon a spectrum. A test plan that is composed of binary test cases where every test has a simple yes or no outcome can usually be performed by unskilled or low skilled testers. Such a test plan would be voluminous, expensive to create, and it would need endless maintenance as features and fixes were added to the system.
Most software engineers view themselves as creative. They want to be making something useful from lines of computer code. This has led the development side of software generation to look down on the testing side, hence the derogatory term ‘tester’ versus ‘software engineer’. In truth, some of the most valuable talent in any software house are the software engineers who have a quality perspective. These are people who can look at a design for what can go wrong and see that as clearly as they see the process for making it go right. A software quality engineer who can understand the code, its intentions, its design and see where its likely pitfalls are is invaluable.
Myth #5 – Software Testing Adds No Value
Quality is an expense. Development is a profit center. This has become a given in the business accounting world. The effect is that justification of quality expenditures on tools, training and such is much more difficult than it is for development. This idea that quality doesn’t add value comes out of a failure to realize that, without it, the lack of product quality will result in a significant loss of product revenue.
Development’s creation of code can be looked at as producing raw material and assembling a product with quality performing the finishing touches. With the rise of app stores where a product succeeds or fails on user ratings that change by the moment, product quality has never been more crucial or deserving of investment.
What is Software Quality Worth?
All five of these issues come down to a question of what value a software organization should place on quality. Viewed as an expense, it tends to be short-changed. In reality, quality is probably the most valuable product a company can create.
Download our “What Should Quality Cost” whitepaper