Usually, people get confused about the meaning of smoke testing and sanity testing. First of all, these two concepts are very different and QA engineer executes them during different stages of a testing cycle.

Smoke Testing

Smoke testing – is a special kind of software testing which means a thorough review of developed web product performance, especially the most important and critical moments. We use the results of such tests in order to understand if we can consider the created software as completely resilient build for future testing.

This type of testing is performed when the QA department in the quality assurance companies gets a new software build, considering it as unfinished and defective. During such a test you have to be sure that all important parameters of the tested product work properly according to expectations.

Philosophy of this kind of testing is based on a fact that QA specialists should find the source and causes of bugs appearing as soon as possible. Also, they reject the tested build to the programmers for further development.

Smoke tests are extremely important because they allow avoiding difficult tests at the stage of official release, so defective software won’t go into production.

The basic task of such tests is quick verification of software performance and stability without the need to execute thorough tests in the future.

Main Advantages of Smoke Tests

• They allow finding problems at the stage of software integration into the real functional environment;
• Testing allows tracking bugs at the early stages of web components development;
• It makes sure that software changes didn’t harm the main system structures;
• Critical test failure immediately leads to rejection of pre-verified build version.

Sanity Testing

Sanity testing – is a very specific way to check the performance of certain functional elements, systems, web architectures and evaluating. Its main goal is to give a complete guaranty that the developed systems works properly.

[highlight dark=”no”]You can perform this kind of testing before all regression tests and after smoke testing.[/highlight]

Traditionally QA engineer executes sanity tests when minor system defect was fixed or software logic was changed partly.

As soon as developers change the code, new software build is going to QA department. After the build installation, QA specialists can perform an effective check for changed functionality instead of full software regression which takes much more time.

If fixed bugs and changes of the code work improperly, the tester cannot take the build for testing. The remarkable thing is that any defects that were found at such an early stage can save a lot of time on unfinished build verification.

Main Peculiarities of Sanity Testing

1. Basically, it is superficial testing which is concentrated on a detailed check of the selected functionality.
2. It is a kind of regression testing.
3. QA engineer perform this test when he/she doesn’t have time for thorough build verification.
4. The testing process isn’t recorded.
5. It is quick testing to make sure that changes in software configurations do not work as expected.
6. QA engineers execute these tests in order to check that minor defects in software performance were fixed and didn’t cause big configuration errors.

Advantages of Sanity Testing

• It significantly saves time and technological resource because of focus on several functionalities only;
• It helps to find related and missing objects;
• You can use this test to check the correct performance of some part of the product after small changes.

Its Disadvantages

• The main focus is on certain functional parts.
• It doesn’t affect the total software level; it’s hard to know for developer team how to edit bugs found during the performance testing.
• You use it during testing for some functionalities. So if there’re problems with other functions, they won’t be found beforehand.
• Usually, the tests aren’t documented, thereby similar tests can be missed in the future.

Smoke Testing Versus Sanity Testing

Smoke testing	Sanity testing
It is performed at early stages of software development before regression testing	It includes builds which passed smoke tests and which have to pass the whole cycle of regression checks
Build can be either stable or not stable	The build has the most stable efficiency
Its aim is an analysis of new software versions	The main goal is testing for efficiency of the configuration work for further thorough testing
Critical test failures immediately cause rejection of this software build usage	Found bugs allow replacing the build in the rejected list which will be worked on later
Both QA specialists and developers can perform these tests	Traditionally it’s tester’s duty
It includes certain documentation and work based on pre-established scenarios	There’s no strict work based on scripts
You can perform it manually and with the help of programmed tests	It is executed only manually

Examples of Such Tests Usage

We will analyze real examples of similar kinds of testing based on real cases.

Example of Smoke Testing

Let’s imagine that tested software has such 5 modules as:

• Registration in the system;
• Client’s account;
• Users browsing;
• Client creating;
• Client tasks creation.

It means that inside such modules QA engineer performs smoke testing and checks the main functionality of these modules specifically:

• Entering the system with third-party client data;
• The client cannot log in with invalid data;
• You can create a new user after authorization;
• If it’s possible to use client account after its creation.

Example of Sanity Testing

Imagine that we have 5 modules again:

1. Authorization in a system;
2. Viewing the user’s page;
3. His/her account;
4. Creation of a new user;
5. Tasks creation.

QA engineers found a defect on authorization page during testing. For example, you can enter only 6 symbols in the password field. And that is opposite to technical assignment which indicates that password can consist of more than 4 symbols.

Tester told the developer about this defect. Programmers fix the bug and module returns to QA specialists for testing. In addition, QA engineers must check the productivity of the other 4 modules in order to be sure that testing of the fixed bug didn’t affect the correct functionality of other systems and logic.

It’s important to understand that [highlight dark=”no”]such an approach tests only extreme module functionality, there’s no test of details because of time poverty. It’s exactly the sanity testing.[/highlight]

QA engineers  execute such tests only when software build passed smoke tests and validation for further checks.