Proper Establishment of a Testing Process

Huge companies focused on simultaneous production and testing of numerous programs sometimes utilize numerous tools and methods in their everyday work.

This article will highlight the process of testing related to test projecting and test integration, that efficiently complement an overall software development process.

Common problems of testing process organization

While organizing a correct testing process inside a huge company, we usually face the following issues:

• A company has no one center of testing competence, it often utilizes random tools, there is no documented description of current testing processes;
• Common methods and testing techniques, useful for all QA engineers, are not used. Frequently, all production and testing departments use their personal technologies and practices of testing: developers can store everything at GitLab and testers – at popular trackers such as Jira, YouTrack and TFS;
• Various testing frameworks with popular programming languages or their personal accomplishments;
• Random types of test reports: someone prefers moving of CI system GitLab CI and someone wishes to save all test runs in a build log of TeamCity;
• Various types of testing components and installers on test boards: we can install everything manually or we can use versatile meta runners in TeamCity or GitLab CI;
• Unique types of tests: one big meta test is used on one project, with the further integration into a test framework and on another one, smoke tests are grouped into a separate group of tests (with the possible division into load and functional tests).

In order to solve the problem of numerous testing technologies, we should unite them into one group, summarize a structure of the skills used in the field of everyday activity of a QA department of a company. In order to reach similar aims, it engages testers who have the task to search for optimal approaches and solutions that help to share efficient practices of software testing everywhere.

We may try to create a company knowledge base which will include a special place, especially for software testing.

Creating a QA knowledge base inside a company

So to understand the amount of work that must be done, we can start from a primary evaluation of the method of testing inside a particular company. For example, if a company has 2-3 QA engineers and each of them has the newcomers to train, they (QA managers) can make a list of questions, the answers to which will go to a summary table.

The questions may be of such a type:

• What is being tested: what parts of the software or installer are being tested currently?
• How are the tests saved?
• Does a team use any frameworks?
• How are the test reports structured?
• Types of deployment: how do testers prepare an application for the process of testing?
• Current testing process: a short description of the software testing type.

After we gathered the detailed information, a company director together with a QA engineer can analyze the possibility of using a versatile list of tools. After this, they can work on a detailed plan to gradually integrate the testers into a new process (methodology and philosophy) of testing.

Stages of Software Testing

What stages of software testing must be commonly included?

At the stage of software testing, it is highly recommended to follow a general philosophy of sequence of performed actions.

Particularly, it means the following steps:

1. During software development, you should launch exclusively unit tests. In such small tests, we can test the methods and classes of the basic functions of the software. A partial change of unit tests into a code review or adding the new tests is possible.
2. When the deployment and software development inside the test environment have been completed, we can proceed with manual testing. For this, a previously established checklist, which contains the overall functionality of the software, is used. This process can be executed apart from automation;
3. Manual tests can be launched alongside the BVT tests. These automated tests can help to test a current accuracy of a build: if the software structure contains all necessary files and settings for correct work of the software;
4. When Build Verification testing has been completed, smoke tests are launched, in order to efficiently test the correct functioning of software architecture. Frequently, it’s a huge test run which doesn’t need much time and money;
5. Successful completion of smoke testing gives a way to measured execution of all the following tests: from functional tests to integration tests and testing the updates.
   

   Correct Software Validation Process Integration

1 type of tests – 1 type of a report

After the completion of any type of testing, we should prepare an analytical report, which contains the main outcomes and outputs.

We can actually use the following structure:

• Console logs;
• Uploading the testing results to the test board (for example, to ReportPortal and TestRail);
• Report to Allure Report, which can be further uploaded to the interface of any analytical framework:
• Uploading to a company database, for example to Grafana / InfluxDB;
• Written reports in a form of Excel file which is familiar to every tester.

Typical features of correctly structured tests

Further, we will analyze some popular types of testing, analyze their proper place in a general pipeline, mention main technical features: requirements to testing, what happens with software before and after testing, what reports can be formed, etc.

Everything mentioned below can serve as a template to the process of software testing

Unit tests

Unit tests are automated tests that start when software development begins and tests exceptionally methods and classes which are compatible with product features.

The peculiarities of unit tests:

1. They are executed when the product development had been completed;
2. They analyze the product software code;
3. At the end of tests, a list of passed and failed tests is formed;
4. Test report generation with their following saving in the logs and integrated systems of data statistics.

Manual tests

Manual tests are the processes of installation and optimization of a testing object, that are executed manually, and also step-by-step test reviews. Commonly, they are formed in a form of user-friendly instructions – in other words, in the form of a classic checklist.

The peculiarities of manual tests:

• Unit tests have been successfully completed, and the test environment has been successfully deployed in a virtual test board;
• Step-by-step testing plan with the detailed description of all test scripts that must be executed by a tester;
• A list of passed ad failed tests;
•  Test report generation: can be prepared manually or in a form of virtual integration to used trackers.

Build Verification testing

It’s a form of automated tests aimed at testing the correctness of test build functioning: whether all files and components are in the software architecture, whether there is localization to popular languages, whether the files with previously established settings are here, etc.

The peculiarities of the tests:

• The code of automated tests;
• A list of passed and failed tests;
• The possibility to form test reports in console logs, with the integration to CI-system reports, export to TestRail or ReportPortal.

Smoke testing

A group of automated tests which help a tester to rapidly test the basic functionality of the software. Commonly, such tests are limited in terms of money and time and also can be used together with those functions, whose improper work may result in the impossibility of future usage of the software (in other words, the tests are meaningless).

The peculiarities of the tests:

• The application has passed unit tests, everything goes to a virtual test board. A group of used Build Verification testing has been successfully executed;
• A list of passed and failed tests;
• A possibility of forming the test reports in console logs, with the integration to CI-system reports, export to TestRail or ReportPortal.

Functional tests

It’s a group of tests aimed at testing the business logic of the developed application.

The peculiarities of the tests:

• The application has passed unit tests, everything goes to a virtual test board. A group of used Build Verification testing has been successfully executed. A report on all steps of smoke testing is formed;
• A list of passed and failed tests;
• A possibility of forming the test reports in console logs, with the integration to CI-system reports, export to TestRail or ReportPortal.

Load tests

The tests that help to validate the software functioning under a certain system load and detect a moment when there will be a denial of service (DoS). Additionally, the test aimed to find a peak load are executed.

The peculiarities of tests:

• The application has passed unit tests, everything goes to a virtual test board. A group of used Build Verification testing has been successfully executed. A report on all steps of smoke testing has been formed. All previously validated functional tests have been checked;
• A list of passed and failed tests;
• A possibility of forming the test reports in console logs, with the integration to CI-system reports, export to TestRail or ReportPortal.

Instead of conclusion

This article highlights the most up-to-date problems of the correct organization of software testing into one company pipeline. To completely solve the issues of testing (lack of tools, incorrect testing, lack of control of task execution), you can use the above-mentioned scheme of software testing.

Obviously, every quality assurance company uses its personal specific of testing but still, general schemes of the process description are a little bit similar in every case.