Filed under: Cucumber, — Tags: Agile testing pyramid, Behaviour-Driven Development, Polyglot, Testing iceberg — Thomas Sundberg — 2017-07-25
When implementing verification of an application using Cucumber, you have a choice of the language to use. Being able to choose is a good thing. But it also forces you to take a decision.
Cucumber is implemented in lots of different languages. The Cucumber site lists these implementations:
That's a lot of options! What should you use? The consultant answer is: "It depends".
If your only point of automation is done through the user interface or through a public web service API, then any of the options above will be sufficient. All you need to be able to do is to fire up a browser and use the application. This can be done using tools like Capybara or Selenium. It is even easier if you verify through a public web service API.
Only testing your application from the outside will place your testing at the top of the agile testing pyramid. This is a well known anti pattern with these properties:
Only testing through the user interface is not a good idea.
The cure for testing through the user interface is to allow testing to be done at more than one level in the agile testing pyramid. This means that you will create parts of your application and test that part. This can be at a very low level, perhaps only one class. Or at a higher level, instantiating a few classes and verifying that the expected behaviour is working properly when they are used together. Mixing levels and granularity for what is tested will blur the line between acceptance tests implemented using tools like Cucumber and unit tests implemented using tools like JUnit. Blurring this line raises questions on where to use which tool.
A common question for people new to Cucumber is, "When should the verification be implemented using Cucumber and when should it be implemented using unit tests?" The answer to this question can be found in the testing iceberg introduced by Matt Wynne and Seb Rose.
The litmus test is "Is this example important and interesting for the stakeholders?" If the answer is yes, use Cucumber. If it is no, use your unit test framework.
The testing iceberg tells us that some of the testing should be done at levels below the user interface. This means that you have to be able to instantiate objects from the production code. This is only possible if you use the same language for the Cucumber steps as for the production code. It is impossible to create and work with a Java object using Ruby.
It is a good idea to use the same language when implementing steps in Cucumber as the rest of the application is written in.
If you use the same language, you are able to implement steps that can operate on any level of your application. It has these benefits:
I would like to thank Malin Ekholm for proof reading.
RSS