Basic BDD: Software Automation Frameworks

Any modern programming language has BDD automation frameworks (sometimes even more than one).

We will try to examine the most current and important BDD with a foundation of various structural principles of frameworks that exist nowadays. By the way, for each of them, there’s certain online documentation, which tells specificity of its usage. So, let’s start!

Basic Frameworks

More than half of BDD frameworks is some kind of standard version of Cucumber that comes from basic framework JBehave, which was described in the manner of its creator Dan North or non-Gherkin spek-runners. Some specialists arrange behavioral scripts in separate files on purpose, meanwhile, others put them inside the written code.

C# and Microsoft .NET

Perhaps, only SpecFlow is the most popular BDD framework for the Microsoft .NET computer languages. Its slogan can be interpreted as “Cucumber for .NET”, which means that it is fully compatible with Gherkin product. The basic set of the framework is completely free and open-source, but the license for some SpecFlow+ add-ins costs money. Also, there’re extensions for Microsoft Visual Studio, which helps to make development easier.

Nowadays there’re a lot of BDD frameworks for C# and .NET. Someone considers xBehave.net as an alternative, which functions perfectly with the xUnit.net. The main difference of xUnit.net is that all step-by-step scripts are written directly in the code, but not in the separate text files.

LightBDD can be considered as more light-weight framework, which uses some methods while interacting with partial classes for maximal readability of the code. NSpec is very similar to RSpec, and actively uses lambda expressions.

Concordion provides common usage of original decisions for specs creating. It’s not necessary to consider NBehave anymore since, from the beginning of 2014, no specific changes were introduced to its functionality, and many developers regard it as a “dead” framework.

Java and JVM Languages

The main competitors in java are such frameworks as JBehave and Cucumber-JVM. So, Cucumber-JVM is an official version of Java program language and third-party JVM languages (Scala, Groovy). It’s maximally compatible with Gherkin and can generate good and informative reports. Driver Cucumber-JVM can be easily customized under its technical preferences.

Jbehave is one of the most widespread frameworks in the BDD market. But unfortunately, it doesn’t have Gherkin’s features. It was created in order to use an original framework on Java.
Both frameworks are very popular, has original plugins in a lot of IDE and are easily assembled on the base of Maven packages.

Also, the Java programming language has other BDD frameworks. For example, JGiven uses fluent API for writing scripts, and bright HTML reports appear with test results. Spock and JDave are unified spec-frameworks, but at the same time, JDave is developing in its way very fast. Scalatest framework was created only for Scala program language and is totally based on spec components. BDD Concordion also has its own original solution for Java.

JavaScript Frameworks

90% of Java frameworks function on Node.js base. Mocha framework is a popular solution that can transfer phrases into English and put them in specification code. Jasmine framework is very similar to Mocha, but the problem is that there’s little information on the Internet on interacting with such a tool.

PHP Frameworks

There are two basic BDD frameworks for a programming language – Behat and Codeception. First one is an official version of Cucumber for PHP, and among developers, it’s considered as the “purest” BDD framework for software automation. Codeception framework can be regarded as a programming utility for creating different tests.

Obviously, both frameworks are good ones, but most PHP programmers believe that Codeception is a more flexible and advanced decision.

Python

Python programming language has a lot of testing frameworks and many of them are actually BDD. The most popular and famous are lettuce and behave.

Framework behave is fully compatible with Gherkin, but lettuce doesn’t have so many language components in its technical functionality yet. Both these frameworks have their plugins for IDE.

Radish is another framework which helps to expend Gherkin with the opportunity to use ready scripts, cycles, and variables. All these three frameworks can write ready scripts in feature files. Also, they can deploy some steps like special functions instead of classes, therefore it makes the creation of new steps of check maximally easy and allows to manage them more independently. Moreover, it helps to avoid unnecessary object constructions.

There are other frameworks for Python programming language. For example, pyspecs – an original spec-oriented framework.

Ruby Frameworks

Cucumber, that is truly considered as a fundamental standard of quality in the field of BDD frameworks, was firstly launched in the programming language Ruby system. Cucumber interacts easily with the official program language Gherkin, and every version of Cucumber was created on the base of original Ruby’s syntax.

Framework Spinach is described as the simplest Cucumber, with maximally finalized functionality of encapsulation of user steps. Rspec is a spec-oriented framework which doesn’t interact with Gherkin.

So What Framework Will Be Better to Choose?

Unfortunately, there’s no straight answer to this question. The best BDD framework is one that maximally satisfies your technical needs on the project.

When you choose the appropriate one, it’ll be better to follow such rules:

• What programming language exactly do you want to use for automation process?
• Is a framework, that you’ve chosen, popular?
• Is its technical component supported actively?
• Is its specification language fully compatible with Gherkin?
• What types of tests do you want to use it for?
• What restriction is there in comparison with other decisions?

All BDD frameworks that can detach a script text from program code are very good at the deployment of shift-left checks. Such BDD frameworks that determine the script text directly into program code are better to be used for white-box testing, but they can cause some troubles when less experienced developers will be working with them.

Absolute leaders of framework automation market for now are: behave, Cucumber-JVM, SpecFlow.

For the maximum effect of interchangeability, it’s recommended to use a framework that’s compatible with Gherkin.

Bonus! Short Classification of BDD Frameworks