Testing Flutter Applications

This article describes possible ways of testing cross-platform and native applications, analyzes the reasons why Flutter can be extremely useful during day-to-day testing, and highlights the issues that may be met during its usage.

Using Flutter for testing native web applications

When a project’s approach to a development process changes rapidly or a new method (technology) appears, it’s very important to not damage testing specifics.

It’s great if testers continue using a common combination of technologies and tools that proved their efficiency in the past, during further interaction with a new language or a framework.

Obviously, you can’t do without automated tests in today’s realities of software development, especially if there is constant regression, and if you don’t use proxy, this leads to a big decrease of technical variation of an application and coverage of certain crucial test cases.

Automation with Flutter

Flutter allows performing both native/cross-platform automated testing.

There is an option to create tests in an application and they will function on both platforms simultaneously.

When a user has access to the overall functionality of a project, he/she can add missing ids or even find and fix new defects.

The product supports a BDD approach (Behavior Driven Development). It’s frequently used for creation and further usage in UI tests.

We recommend using Gherkin as a development language since its functionality allows using feature files, creating scripts either in Russian or English.

This language is very clear, contains a clear implementation of test steps of a script without any additional arguments, parameters to customize the start of test automation: for example, you can run some types of scripts by using special tags, not complete tests, and this will greatly save the time needed for testing.

If you decide to use Gherkin together with Flutter, you should use the flutter_gherkin framework (https://github.com/jonsamwell/flutter_gherkin)

Now let’s analyze the differences and similarities of Calabash and Dart+Gherkin and choose the most efficient tool to perform web testing.

Calabash vs Dart+Gherkin in combination with Flutter

Feature files are the same for both approaches.

For example, there is a nominal script to perform authorization with a pin code and it will be properly perceived both on Dart and Ruby in combination with Calabash.

Scenario: Correct first-time authorization in the application (a short code)
When I run the application
And I see a short code input screen
Then I enter the correct short code.
And I see the home screen.

Both technologies work properly in Russian, English, and some other popular languages.

The steps are different during direct implementation.

So Dart+Gherkin:

Dart+Gherkin

Calabash:

Calabash

We can’t say for sure what tool is better.

Anyway, a technical structure inside every technology is not modified and it’s very good.

Flutter uses an additional .dart file to configure tests and interact with them. You won’t see such a file while working with Calabash.

It’s not correct to say that it is a big disadvantage of Flutter or Calabash — it’s probably specifics of functioning with certain technologies and tools.

While working with automated tests in Calabash, you should take into account the fact that ids should be used on both platforms of the applications you are testing.

Dart allows entering unique ids while writing a set of automated tests, not needing to upload separate files for iOS/Android once again and this will greatly save the time spent on testing.

A short conclusion: automated tests created on Dart are as good as automated tests based on a Calabash framework.

Basic issues in third-party libraries that may be met during testing of Flutter applications

We’d like to admit that unique flaws of Flutter applications occur both in native and cross-platform products.

Further, we’ll highlight what should be primarily tested while performing testing of Flutter applications.

Defects of a Flutter framework:

1. One of the common bugs are bugs with displaying and editing of fonts on iOS: for example, the spacing between letters on iOS is much wider than the one on Android. This results in numerous visual bugs;
2. If you develop an application for iOS 11-12, you have chances to meet a flaw in third-party libraries. For example, you can find a defect when notional permission to access a notification appears right after you have launched an application, not after you have clicked on a special button, that is present there according to design (or there may be any other graphical element instead of a button).

These bugs may occur either on native or cross-platform web products. They can be fixed by a local team or with the help of library authors.

Now let’s analyze the interaction with expected native behavior.

If you use and test native applications on Android and iOS for a long time, you can predict user’s expectations from different ways of product’s behavior.

For example, returning a back-swipe function on iOS is a common action. But still, you don’t need to do this for Android devices.

System dialogs on Android and iOS are completely different.

iOS always requires permission to access push notifications and Android allows doing this by default.

Such features of a mobile operating system should be refined manually.

Or sometimes, they may be even removed if an expected iOS behavior is reproduced on Android, in the same way as a back-swipe.

To conclude, we’d like to say that Flutter is an important step forward.

You can easily test cross-platform application functionality.

Sometimes it’s even more comfortable and pleasant than interacting with native products.

All possible difficulties that may arise during development and testing can’t spoil all benefits of a Flutter framework.