Ten Commandments of Automated Test Development

This article will describe commandments we should keep while developing automated tests.

Commandment 1: automation should not be based on current test cases only

There is a common misconception that the entire test automation should be based on test cases only.

Automated testers operate only available test cases and transform them into automated scripts.

This approach makes sense but if we take other testing methods, they can be also useful.

If we broaden the term “automation” beyond the limits “test case — tool — test script” to the phrase “proper usage of tools, aimed at helping testers to perform their everyday tasks”, we can use the technical capabilities of a computer for tasks that need this, and testers may do the rest tasks.

Hopefully, when people work properly, the technology always fails and vice versa.

Commandment 2: automation development is the same thing as software development

Automation development is exactly a process of developing certain software. Even if a tester simply uses a simple graphical interface for dragging the objects from one pop-up to another one, there is software code that stands behind these processes.

Approaching automation in the same way as software development means that an automated tester should take into account numerous processes and types of professional activities, needed for software development.

For example:

1. Design — we need to decide what should be present in a product and what — not;
2. Implementation — we need to write code;
3. Storing — code should be stored somewhere;
4. Testing — automation should be tested, in the same way as we test software;
5. Defects — bugs are and will be here and there;
6. Logs — if we don’t understand their nature, we will have problems with understanding what automation created by us is responsible for.

Commandment 3: using development standards and idioms

In addition to approaching automation in the same way as software development, we should also not forget about embedding popular ideas on its further implementation in its structure.

Every software and programming language have their personal idioms and details but common methods of design and implementation are always useful.

Commandment 4: remember about maintenance and upkeep

Software can’t be absolutely perfect and completed. And automation is not an exception. It will always contain bugs.

When an application that is being tested is updated, automation should be also updated. We can’t prevent this but can try to develop software in such a way that it will decrease costs spent on its maintenance.

Commandment 5: automation scripts should not depend on each other

The development of a test script that depends on the results of another test is commonly a strong anti-pattern.

If test scripts depend on each other, you should not run them separately and this complexifies automation debugging and fixing of software bugs.

Apart from that, it’s not recommended to run scripts with the ones they depend on.

Of course, there is a borderline case when all scripts depend on the main one. This single script can configure the test environment and verify test data.

It’s a very rare case, thanks to today’s conditions of automation and constant deployment but this script can be useful when frameworks used before are unavailable or are simply not suitable for an actual automation task.

Commandment 6: use only proper logs and test reports

Proper usage of logs, results, and bug reports is crucial for understanding and maintaining a level of automation on each project separately.

Automation logs are in some way a detailed example of how to run each automation test separately: what and when was launched, what failed and what succeeded, what conclusions can be made from test results, and so on.

Of course, if logging was properly used in an automation stage, we can get the necessary information on results and finally analyze strong and weak spots of using certain automation basics.

Commandment 7: influence testability and automatability

Testability and automatability are not common tasks of testers but testers should influence them. And this influence is a primary task.

Sometimes developers don’t know what testers need for software testing and what should be automated.

Commandment 8: beware of sunk cost fallacy

Sometimes we all make errors. Sometimes this leads to appearance of critical bugs.

We should take everything from information available but it frequently may lead to not what we have expected.

If something goes wrong, we instinctively try to solve the problem and will do this until we reach a proper result.

But sometimes we should start from the beginning since it’s not practical to waste time and other people’s money.

This is called “sunk cost fallacy”. In short, a certain amount of money has been spent on a project, we need to spend more for recovery, to benefit from money already spent and sunk.

Commandment 9: don’t use crafty tools

Don’t even try to develop things that are hard to maintain, in comparison with manual tasks.

Commandment 10: test data should not depend on transient data

Don’t use transient data since one day a test script you will use will simply fail and this can result in failure of the bigger part of automation.

And cases may be different so fixing such a triviality as time may lead to receiving complaints from managers and dissatisfaction from a client.