Today every QA engineer is familiar with the popular theoretical limitations of software testing. These tasks need spending too much time and strength.

Still, we all know that it’s unreasonable to completely forget about software testing.

Other methods of validation (verification of software operability) such as analysis, model checking, and tests obviously give us great potential opportunities. Anyway, they are not perfect tools that can completely replace testing as a more popular activity.

Therefore, we should understand [highlight dark=”no”]the scope and limitations of software testing and the way we should execute testing.[/highlight]

All concepts and principles of testing that will be analyzed later in this article are based on theoretical and practical analysis of the functionality of web products and their related components.

Principle 1 – definition

To test software, you should make it work improperly.

This principle describes the main task of software testing – find the bugs by making software work improperly.

All inferences on software quality help us understand that testing is not directly connected with defect fixing, in contrast to a debugging process. It’s based only on catching them.

Testing and technical specifications

While developing software that depends on testing that is popular due to certain methods of flexible development (Agile), we should perform testing since tests are considered more important than requirements of the specifications. But it’s an incorrect judgment.

Tests are just good examples, even if their number is high. They have no abstraction that is initially established in specifications.

Principle 2 – testing or specification

Tests can’t replace specifications.

There is a dangerous logical misconception that tests are the structure of specification. This has been shown by numerous glitches and crashes that happened because nobody predicted the possibility of exceptional situations.

Though specifications can comprise not all cases, they are an example of certain generalization.

Sometimes specifications can be used for creating a required number of tests, even automatically. But a reverse process isn’t possible without human interaction.

Regression tests

The specificity of testing, if we analyze the peculiarities of software development, is a constant possibility that already fixed bugs will appear again. It’s like hydra’s heads that grow again after being cut off.

This thing is called regression and makes a user perform regression testing, in other words, verify that the things that have been already fixed still work smoothly. Consequently, a user should always remember the defects that have been found on the project.

Principle 3 – regression testing

Incorrect execution of any action should lead to creating a test case that will be a permanent element of a test package on a project.

This principle can be applied to every bug that arises during software testing and software development. This requires using the tools that help to prevent failure in the execution of test cases.

Predictions

Work with a test is useful only when you are absolutely confident whether it has been executed or no. This criterion is called a test prediction.

If a user has hundreds of tests or more, he/she can check them separately. But this will be almost absolutely impossible when their number rises.

Such a task will require software test automation skills.

Principle 4 – Using predictions

You should define test passage/failure automatically.

Such a definition leaves this issue open in terms of a form of such predictions. Sometimes predictions are specified separately.

They are built in some tests since the software-in-test should consist of contracts by default, and the tests serve here as predictions.

Test cases that are analyzed manually or automatically

More than half of the test cases are executed in a manual mode. QA specialists make up interesting scripts and prepare corresponding tests themselves.

This category includes tests that are based on the third principle as a result of incorrect execution, they were not initially supposed to be used during product testing.

Today we can frequently complement these two categories with automated tests that appear with the help of specification and a generator of automated tests.

Activity that is limited to such manual tests doesn’t completely use the capabilities of modern PCs and their related programs.

Principle 5 – test cases that are executed manually or automatically

Every process of software testing should consist of the test cases that can be checked both manually and with the help of automation tools.

The advantage of manual tests is their depth. They can reflect the way a developer understands a current cycle of issues and a data structure.

The benefit of automation tests is their complete width: they can execute complete testing of a big range of values such as extreme ones that can be missed by users.

Testing technique

Let’s move from testing to studying new software testing techniques that are in some way dangerous for a thinking process.

One may generate an idea that is likely to be improved and is verified by his/her intuition.

Software testing is an extremely complex process and not all ideas can be useful after being analyzed.

A good example of this is random testing. We may feel that all strategies that used the product data should be completely useful, in comparison with random data entry. But correct values such as a number of bugs found will prove the fact that random testing sometimes exceeds reasonable limits.

Principle 6 – empirical assessment of a testing tactic

You should always assess any testing method. No matter how interesting it seems to be, look at it only in an objective way, by using exact criteria while performing bug searching.

While applying this principle, you should answer one very important question: what criterion should be used.

Thematic literature on testing gives the following description: “a number of tests that show software failure”. For practical specialists, such a value is not useful at all.

A tester should find all bugs, not one bug, even the most important one.

Let’s imagine that using the “first failure” criterion will be correct and it will be used several times. But all future failures can be of absolutely different nature and an automation process should help to find as many bugs as possible, not focusing on the first found bug.

A number of tests are also not useful either for a project manager or for end-users that can assess a number of found bugs.

The time needed for finding the defects is considered a better value.

However, there is a risk to choose a strategy that will help to rapidly find the bugs but only after searching them for a long time and this will increase the time needed for testing software.

Principle 7 – criteria of evaluation

An important peculiarity of a testing strategy is a number of bugs found as the function of time.

The function of searching or a number of bugs is an extremely important thing.

Using software for finding common defects will help you to easily assess the strategy by analyzing the number of defects that can be found in software for a certain amount of time.

Conclusion

As we can see, all principles are interconnected and are based on one general logical sequence: the first principle says that tests are the result of technical failure and the last one tells about numeric proof of this point of view that is also based on all common principles.