Methods of Testing the Antiviruses

Permanent development of digital communications has caused that a number of users of global Internet has increased almost several times.

Today, before going to the world wide web, everyone considers installing a special antivirus software.

A wide range of antivirus software which, according to their developers, is really reliable, practical and simple to use, is on the market for special utilities and components. It is in this context that a process of testing antivirus software and components becomes very popular, in order to efficiently find the most suitable web product.

Methods of Testing Antivirus Software and Applications

Methods for Testing Antiviruses

Today there are 4 types of methods in the field of testing software and similar products:

• Static;
• Dynamic;
• Reaction rate tests;
• Retrospective.

Let’s thoroughly analyze every mentioned type.

Static Testing

It’s the easiest and the clearest way of testing the functionality of antivirus software and similar products. Its sense lies in obligatory scanning at a technical request, which is reproduced on the basis of malware.

In order to get particular results from a static test, you should use a massive collection of malware, which structure has more than a thousand files and similar documents.

In the net, you can find special collections of test organizations (AV-Test and AV-Comparatives) which have more than a thousand possible software and files, which allow performing successful testing. Sometimes its structure can comprise up to 1000 000 files.

Of course, mentioned tests have their advantages and disadvantages.

An advantage is that the whole process of testing is performed on numerous collections of available malware, which include the most popular types of malware content.

A disadvantage is that such range of collections usually includes only “fresh” releases of malware files.

Experience has shown that types, which live not more than six months, are very popular.

Additionally, a disadvantage is also that such testing helps to emulate hardware at your request, while in a reality a user gets malware files and elements by personal email or by direct downloading from a world wide web. It’s very important to find such files in a moment when they appeared on a client’s personal computer.

Dynamic Testing

Its sense is to perform reproducing of real client’s environment, in which we actively test a necessary product, which is responsible for the security of user environment, with the help of the most available number of possible virtual tools. Such testing becomes more and more popular due to the quick development of new tools and methods, which can’t be completely performed inside traditional testing environment.

For example, in order to analyze the current efficiency of used antivirus protection, PC Pro testers not only reproduced a virus collection at their request but directly downloaded the emails with the infected files or emulated downloading of “dangerous” files from the Internet, on the basis of written scripts.

Such test can be regarded as the most similar to a real situation and, as it became clear, a possibility to resist malicious software and components in most client products is much lower than a number of found viruses in the moment of testing infected documents and other files at your request.

Testing a Reaction Rate

Though such a method is not very used and popular, such type of testing must be thoroughly described and decoded.

Testing a reaction rate was performed almost every day in time when world wide web suffered from “a plaque of mail worms”. We can even recall the names of such programs – Sobig, Bagle, Mydoom, Sober.

In contrast to the static type of testing, testing a reaction rate uses a quite small set of necessary patterns.

To exactly identify a reaction rate of the tested antivirus program, firstly turn attention to the speed of finding the last modification and build of malware content. By the way, while performing such testing, the advantage is given to those antiviruses, which databases are most often renewed and are updated with new information and protection files.

Retrospective Testing

In contrast to classical methods and ways of testing, retrospective testing is performed to test different versions of antivirus protection at the moment but in past.

Usually, such a moment is very far from a date of testing, for the most possible number of computer viruses, with which you need to work, will have been developed. It means that the results of retrospective testing allow checking a real protection of tested antivirus software product.

Testing Antiviruses by Labs

Analyzing the results of testing by a specialized lab, firstly thoroughly study a tested product and also its vendor.

As it usually happens, different tests can test 2 same products developed by one developer in a different way. That’s why we highly recommend to complexly study the technologies of a developer of antivirus content.

A detailed report of such highly-specialized labs as ICSA and West Coast Labs on testing of a vendor’s product is always published after testing finishes with a positive point.

These points of the labs in the last time start to play important role in the process of choosing tested antivirus software but their absence doesn’t mean that a product is not qualitative or second-rate. The developers just didn’t want to take part in the process of testing.

Testing Antiviruses with Virus Bulletin

Virus Bulletin lab uses original method of testing the functionality of antivirus programs on the basis of the logic of finding malware components from well-known popular collection WildList.

Moreover, it tests not only the ability of the utility to find a virus but also the complex functioning of the whole software, especially its quick reaction and influence on hidden dangers.

Additionally, it calculates a sum of fake detections, which is a very important component for any antivirus program.

All antiviruses, which are successfully tested and get points from 95 to 100 % and in case of zero fake response, get corporate right to use a special logo VB 100%.

A specific feature of this lab is that for testing they use only a list of malicious components released 2-3 weeks before the start of testing. Alongside with work of finding the viruses, components are additionally tested on empty files and data, to show a possibility of fake work of protection.

All companies which sent their software components for testing are not aware, will their product be successfully tested or no.

Principles of Testing Antiviruses

Importance of Testing the Antiviruses

In the last years, it becomes very important to resist the dangers which were not found. In order to foresee a level of software protection from the viruses which are not yet released, you should use particular methods of testing. Classical tests won’t be very helpful, as they all are directed to resist the dangers which were found before by someone else.

On the one hand, we can completely switch off a base signature in the tested product and analyze how it could find malware components from a modern list “In The Wild”.

But we must point out that such method won’t help to get a necessary result: the nature of any antivirus requires work together with a signature base.

A defensive component with switched-off bases of such type is an absolutely different product, which doesn’t need to be tested.

Of course, we can test the ability of antivirus to find future released viruses, testing current virus base, but only in case of testing antivirus with a signature base, which term is not less than six months.

It’s logical to suppose that viruses that are now in a list of ITW, didn’t exist half a year or a year before. Antivirus component should resist the dangers which don’t yet exist on the world wide web.

By the way, such testing is performed by specialized resource av-comparatives.com, where every user can find the results of retrospective testing, which help to choose the most suitable antivirus and also will help to follow a tendency in the development of protection programs and components.

Universal Test Plan (Test Case) of Testing Antivirus Programs

Antivirus Component Validation Logic

Basic criteria for testing:

• An indicator of detecting virus base;
• An indicator of detecting, according to the “in wild” list;
• A number of fake operations;
• Eristic analysis;
• A process of emulation;
• Healing infected component.

Let’s analyze each of them.

An indicator for detecting a virus base means that in the process of testing it performs scanning at the request, with a big number of infected types. A level of infection is directly measured in the percentage of a number of infected components against a total number of tested files.

An indicator of detecting according to list “in wild” means testing files and components from mentioned list ITW. It is measured by the percentage of found malware programs against a total number of objects.

A number of fake operations means that a test uses a big number of possible files, which are not really dangerous. Testing measures a number of fake operations against a total number of elements.

An eristic analysis is a special method, completely based on signatures and eristic, which allows to efficiently find modified versions of viruses when a signature is not completely identical with a body of unknown utility and suspicious program has all features of virus component.

By the way, this technology is rarely used in modern tests, as it can easily increase a percentage of fake operations while testing.

An emulator is a huge testing of a component with a signature scanner, on a suspicion that it consists malware element.

Healing is a method of identifying the possibilities of antivirus components to “heal” the objects when malware environment has reached files of a user and actively resists its direct deleting.

To conclude, we may make a clear conclusion that today there are not 100% efficient software component which will completely fast and directly “kill and destroy  ” malware components and data.

But a complete understanding of possible dangers and the right choice of methods of antivirus protection can help to hugely decrease a level of possible infection and properly build a process of testing and analysis of particular protection component. It means that your quality assurance team should responsibly approach testing antivirus components and with a clear understanding of performed actions.