The Truth About End-to-End Testing
Much thought and planning has gone into how to structure software quality tests and a great deal, if not most of it, concerns the balance of testing costs versus results.
One focus of this controversy is end-to-end testing. End-to-end testing is the verification of a fully functional system’s ability to respond to its user interface with appropriate activity at all the extremities of its connections.
For example, an e-commerce customer makes a name change to their account. End-to-end testing will verify the name change filters from the website form through the middleware and into the necessary records in the system database.
Thorough, effective testing of end-to-end operations involves complex test scenario planning. This typically involves installing the system on an isolated server, creating specific data and configuration conditions that can be readily restored, and having a detailed, fully documented test process to follow.
If all this sounds expensive and time-consuming, it is.
The real question is, does end-to-end testing ultimately pay for itself in the reduction of defects that escape into the delivered system?
Why You Need End-to-End Testing
In theory, a carefully thought out, thorough array of unit and integration tests followed up by a system verification against its user interface specifications should assure overall system functionality. If you look carefully at all the parts, logic says that the whole of their assembly should perform as expected. So why do operational issues appear when the parts are exercised as a system?
In a word, communication.
Software systems — applications, service platforms, e-commerce sites, and the like — are composed of a multitude of functional modules that must communicate command and data information to each other. Even if the interfaces into and out of a code module have been verified to meet their specifications, they may encounter content and timing issues in real-world usage.
The human equivalent is called the telephone game where a circle of perfectly capable individuals whisper a message from one to the other until the message completes a trip around the circle. Without fail, the message given to the last member of the circle bears little resemblance to the original text.
A string of middle-ware modules connecting a website to a database bears much resemblance to this game and, too often, has similar results.
End-to-end tests verify the use of the system controls to retrieve, manipulate, and store data results in the intended activities at the database and API points of the system. In particular, they test message content integrity (does the correct information get transferred throughout the chain) and operational timing (did content pieces/control directives get where they needed to be in the proper order). This becomes vital when third-party services such as credit card authorizations are performed through APIs that require multiple calls to complete a transaction.
A complete end-to-end test is the final verification that the product design made it into the reality of a ready-to-go software system.
The Downside of End-to-End Testing
As with the need for end-to-end testing, its downside can be summed up in a single word, cost. With its extreme setup and overall control and monitoring requirements, this is easily the most expensive and time-consuming software quality test method.
It requires a staging server or server network with a prepared database and configuration file contents that can be restored and directly altered for each test. It requires personnel who are aware of the system’s design and who have a software quality perspective to guide them in prodding all the system’s weak points and probable failure modes.
React Native End-to-End Testing
End-to-end testing is a valuable tool for ensuring the quality and reliability of your React Native apps. By testing your apps end-to-end, you can simulate the user experience and find bugs early on. This can save you time and money in the long run, and it can help you to improve the quality of your apps for your users.
End-to-End Testing in an Agile World
Clearly, while necessary, a full-up end-to-end test is not a casual undertaking. This leaves the question of how to support Agile Continuous Release (CR) code development. End-to-end testing can be automated but the effort to do this is both extensive and transitory. Writing test scripts to perform a thorough test is typically as expensive as simply spending the hours to manually perform it and system changes (bug fixes, new features) require script rewrites to match.
The best solution appears to lie in sanity checks. These are automated tests that touch on every functional aspect of the overall system. They don’t test in detail, they verify that each function is present and responsive to a limited set of inputs. Any oddity in sanity check results should be immediately investigated.
The Lowdown on End-to-End
Many product managers have decried end-to-end tests as a cost they can’t live with and can’t live without. Periodic full tests have to be performed but sanity checks for each release help mitigate the gaps. The frequency of end-to-end testing should be a point of constant risk analysis.