Is Scriptless Test Automation All It’s Cracked Up to Be?

The Agile software development methodology has compressed the traditional months-long software development cycle into days. To support that kind of release speed requires QA testing to be performed concurrently with development, often with test planning done even before the first line of system code is written. This headlong plunge into continuous code release has not removed the need for testing. It has made software QA an absolute imperative.

Test automation has evolved into two basic aspects. In one, the QA engineer (all but indistinguishable from a code engineer) must write test scripts that are programs themselves and maintain them accordingly. In the other, test scripts are created via the test framework itself either by recording user activity or using keyword designs that become a meta-test language allowing the script structure to be separated from its content. This second version is called by the misleading term scriptless test automation.

The Evolution of Scriptless Test Automation

The pursuit of faster and faster release turnover has driven a need for testing processes that can keep up and test automation appears, at first glance, to be a perfect fit. Automation is viewed as offering the ability to have the computer test its own software via test scripts that can be continuously reused on command. As good as it seems at first glance, two issues have arisen with this perspective.

Test automation scripts have to be written in the first place and, once written, they have to be maintained because they must change with the modifications to the code that they test. Worst of all, these generation and upkeep processes tend to consume the very talent resources that are desperately needed to write system code in the first place.

The Robots Are Coming!

Scriptless automation is an attempt to render the script creation and maintenance processes as painless as possible. In one of its incarnations, scriptless automation is performed by having the automation framework watch the system’s operation and record functional actions to be replayed at the user interface. Such tools are commonly used to facilitate a reduction in the size of the manual test force that is typically charged with operating all the system controls according to a test plan. Simply take each test and record its operation once then replay it as needed.

The second version goes beyond the first in that it allows the test designer to frame the test process as a series of keywords and write scripts as near-English sentences. These keywords are associated with test code fragments that the test framework links together and executes by following their use in the higher-level scripts. A series of test scripts can be described in a comma separated values (CSV) spreadsheet and executed directly from it.

Why Use Scriptless Test Automation?

Scriptless test automation is commonly promoted as QA’s future, especially by the companies that make automation test tools. Like all powerful tools, it has its strengths and weaknesses.

Its single greatest draw is avoiding having to hire QA engineers who have the programming skills to code and maintain the test scripts. This goes beyond simply saving on personnel costs. There is great value in being able to push the design of system tests back up the chain into the hands of the people who have specified the parameters of the product in the first place.

The keyword scripting process allows the system tests to be generated by those who know exactly what the system is supposed to do. It also facilitates test design before code is written because it is specifying the structure rather than the exact content of the test scripts.

For organizations that are using Design To Test (DTT) methodologies, this means that the test scripts can become the design documents for the system itself. The system code will be written to perform the desired functions because it will be written to pass tests that explicitly describe those functions. As the system code evolves, the test code blocks that the keywords address will evolve with it and propagate those changes across the scripts as they are integrated into the system.

The Pitfalls the Scriptless Automation

This works extremely well for high level tests, but what happens when the test inevitably must get into the guts of the system software to exercise critical functions? Here, the maintenance aspect rears its head again. Tests of middleware business functions can be written with high level keywords, but API tests can be much more difficult, especially when negative testing is required to test error trapping.

When the function to be tested gets down to the most granular level of the system code, the need to write very specific test code begins to obviate the value of keyword scripting and negates use recording altogether. Scriptless automation has a lot going for it, but it does have its limits.

Contact Us