Introduction
Last month wrapped up my Cucumber posts for a while. I’ve covered the full gamut of topics for Cucumber, and I don’t believe I have too many more nuggets of knowledge to offer without putting myself out of a job.
While working with Cucumber over the past year I’ve done a lot of research. One of the major things I’ve seen is that there is a lot of hate directed at Behavior Driven Development. While people do generally back up their complaints, I believe most of it is unjustified and stems from either not understanding how to properly test software, or misuse of the tools. I wanted to address some of the complaints I’ve seen most often and hopefully provide a little insight into how Cucumber can assist in the BDD process and help an organization when used properly. In this post we’ll clear up misconceptions such as:
BDD Isn’t Testing
That is correct, BDD isn’t testing. Notice there is no T in the acronym. BDD is a software development process which is based on TDD. Behavior-driven development was developed in response to the issues encountered in test-driven development. Cucumber’s intention is to bridge the gap between BDD and testing, and allow these requirements written in a behavioral manner to be used within the testing environment. The best way to accomplish that is to follow some of the guidelines from my previous post. Additionally, some processes can also help. Ensure your domain expert is writing the gherkin scripts. Keep these simple, and again, make sure no how’s are addressed. Once the scripts are written, let the testers turn these into actual tests. Turn each Scenario into a Scenario outline by replacing values with variables, and building Example tables. Look at edge cases, and appropriate content, focusing on good test data. This will turn a functional design requirement into an actual test, and will pull in each team members desired skill set.
Cucumber Requires A Separate Testing Environment
This complaint comes from people deep in unit and integration testing. BDD doesn’t fit well into their maven or ant builds, can’t be kicked off nicely using surefire and failsafe, and requires more maintenance. Well guess what, that is going to be true for almost all functional testing setups. Even coding your functional tests in xUnit style, will still require another environment as functional tests need the application up and accessible to run. This means plugging in additional values such as deploy information and configuration. When done properly, this isn’t a large deal, but yes, another source repository should probably be used to manage these. The amount gained from having these styles of tests, greatly outweighs the small toll on the price of additional upkeep.
Increased Wordiness
This is something I’ve heard from developers and others as a complaint about BDD in general. The extra level of abstraction provided by the human readable file creates more work, and more upkeep. If this is something that plagues your work, you might want to consider why you are using BDD. BDD isn’t necessary to effectively accomplish TDD, but it can help when requirements are being handed down by product owners, and domain knowledge experts, who have little programming background. Cucumber allows non technical staff to develop testing scenarios before development starts, and also to pass back up the results to potentially less technical managers and others interested in browsing through reports. If everyone is technical, and comfortable with xUnit style results, then yes, Cucumber or another BDD tool may not be for you. Don’t just consider your requirements when choosing a tool or testing framework, consider your audience as well.
Cucumber Breaks IDEs
This is an unfortunate inevitability due to the nature of most BDD tools. They rely on regular expressions to do the parsing of test steps, which becomes a complete bear for IDEs to interpret. For this reason, Robot, and several other testing frameworks have their own IDEs for assembling test steps into a test case. Well, I’ve been working on one for Cucumber, and you can try it out for yourself. It is still under development, but currently there is java code to grab all needed step definitions, and an html file to build test cases. Feel free to create a fork help contribute, or just suggest improvements or find bugs.
Step Organization w/ Reuse becomes Unmanageable
This I’ve read online multiple times, but I have seen or run into it in person. I believe this is because the projects I’m working on are very careful with how they design and structure their tests. As with ALL automated tests, Cucumber tests should be treated as code. If organized smartly, steps are simple to find, update, and reuse. The best approach I’ve seen to this is to create a main step class, and have multiple additional classes extend off of them, having a separate class for each area of functionality. By ensuring that each test class extends a base class, you’re allowing for cross over in test steps. Instantiating important classes or objects in your base class will give access to each sub class, which allows for greater re-use of test steps. If, for example, you need to login for each test case, create a login model instantiated in your base class so that each extended class can use information in it for each test, without having to duplicate supplied information or test steps.
Another complaint I’ve heard in this area is that after a certain point, it’s hard to keep track of all of the test steps out there. This adds to the re-use issue, as people may be replicating functionality. This again should be treated as it is in the software development world, with test (code) reviews to familiarize everyone with the test steps out there. Unfortunately, in the development world you are deep in the code, and see most of the methods you could be calling. Often with Cucumber, you have different people writing the tests than you do implementing the steps, and so the test writers may not be familiar with all of the available implementations out there. This can be somewhat abated by test reviews, but still pose a challenge. Use of the Cucumber IDE I referenced can definitely help to remedy this. Having all of the test steps available from a menu, and help filling them in, will ensure that reuse becomes as common as possible.
Conclusion
Following these suggestions, you can implement a robust TDD process using Cucumber for testing. As I mentioned before, I don’t plan on making any more Cucumber related posts, but we’ll see what direction my work takes me. When I inevitably do some more work with Cucumber, I’ll be sure to post anything new/helpful that I come across.
Happy Testing!
