Software Quality Criteria in QA Services

According to the generally accepted definition, software quality in QA services determines how far the program product meets its requirements. Nevertheless, sometimes it is hard to understand: when exactly a product fully satisfies its requirements? How to understand that a product reached the required quality? Adoption of product quality criteria helps to solve this task not only for software testing companies but also for users.

In the given article, we singled out the main criteria that at least will help answer the question: whereby the conclusions about a high quality of a product were drawn?

Quality criteria of a program product should be measurable, meaning that there should be a possibility to introduce them in a quantitative concept. These measurable quality criteria help to reconcile with a customer a desired product state for the release. Besides, measuring quality criteria by a dedicated testing team while working at a project can help in predicting some possible deviations from a planned date of release.

Program product ability to perform its functions (conformance to the requirements)

It is the most important quality criterion because users pay money exactly for software functional. This criterion can be measured by two options:

1.  The number of opened functional errors in a product. There shouldn’t be critical and major defects in the functional. All the essential defects that couldn’t be fixed should be described in customer documentation as a product limitations (or as a known issue).
2.  The number of passed test cases if they cover all the software functional requirements. In other words, all the functional test cases must pass to make sure the product meets the requirements.

System stability

It is the ability of a software product to function correctly during an extended use under certain load. This is one the key software characteristics, but it’s not always clear how to measure it and, what’s important, how to define an aimed value.

In order to make sure that a product is stable, one can simulate conditions in which the product will work for the user. For example, if it is known that, on average, a product is used continuously for 10 hours, it is necessary to check its operability during 15 hours, as there can be deviations from the average time towards “toughening”.

Another example – a web-application, an average expected number of users of which is not more than 5000 at one time. In this case, it’s necessary to work for stable operation with 7500 users at one time, as there can be deviations from average value towards increasing.

Thus, stability testing of a program product should be performed under tougher conditions than the expected use conditions. Correspondingly, while determining an aimed value it’s necessary to indicate working time and volume load that will exceed the expected ones under normal conditions. Besides, one can indicate resources consumption level of the application in aimed values. For example, consumption of memory space shouldn’t exceed while an application is running.

Program product performance

It is an execution speed of product’s standard functional operations. In this case, quality criteria can be or an average execution time of analogous operations in a competitive products, or index can be defined as “not worse than in the previous versions” (in respect to a product, that was released).

Supported platforms (configurations)

All the basic functional tests are performed on all supported platforms. It’s not always possible to conduct every available test on all configurations, declared as supported. It takes a lot of time to perform such tests and allows proving that in the product there is no specific for the platform errors. In such a test case one should include only that tests, the success of which depends on the platform type.

The number of incidents on a sold copy (per user)

This is the number of requests to a help desk after the product release. As indicator values can be measured only after release, one can use it exclusively for monitoring (not as a condition for release). In other words, it’s impossible to reach a target before release, but one can establish goals on its declining in next releases. Using of such a target will help to fix the majority of the problems that occur when users work with a product.

Of course, achievement of the planned indexes can’t ensure that a product won’t have any errors and crashes, but it allows drawing the conclusion that a given product has quality, reconciled with a customer.