NBehave VS2010 Plugin: New Builds

 

We’ve been hard at work improving the plug-in, and now is a good time to make it available. Please bear in mind this is still work in progress, and we really need your feedback both on stability and features.

There’s a new front-page on the bitbucket repository: http://nbehave.bitbucket.org/

New alpha builds will be published on the bitbucket download page: http://bitbucket.org/naeem.khedarun/nbehave/downloads

The wiki is also getting some love: http://bitbucket.org/naeem.khedarun/nbehave/wiki/Getting_Started

There are two new major features in the latest build:

Syntax Highlighting

1

Execute single scenarios

2

New releases will be published at @NaeemKhedarun, and major features will be blogged about here, I hope you enjoy the new features. And please, we need feedback!

Introducing the NBehave text based scenario VS2010 runner

 

I’ve been using NBehave for a long time now on various projects, usually using it’s fluent syntax. I wanted to move to using text based scenarios but unfortunately unless you like dropping to the command line to run your tests you were out of luck.

Now there is the wonderful looking SpecFlow, and I would be doing a disservice to the community if I didn’t mention it. It’s a very complete and increasingly popular framework for running gherkin compatible text based scenarios. However it accomplishes its task in a very different way to NBehave, and the differences are worth looking at in a separate post.

Initially John and myself took a look at making a ReSharper plug-in, however its API was driven towards running code not text and getting things working was not immediately obvious. So instead we decided on testing out the visual studio API which we had far more success with.

So let’s take a look at the new VS2010 runner and how it can make life a little easier. We will need to get the installer from the build server as the plug-in is quite new and has not made an official release yet. I would recommend getting the latest executable artefact from the following link:

http://teamcity.codebetter.com/project.html?projectId=project30

image

Now we can run the installer and ensure the plug-in is ticked, this will deploy the VSIX package into the appropriate location.

image

Assuming you have a project already using text based scenarios, or have follow Johns blog post then we should be ready to go. I already have a solution ready so let’s take a look:

image

We can see the plug-in is loaded and ready to go, I already have a feature file and it’s associated step file set up:

image

If we right click on the feature file then we will find some new context menu options:

image

Picking Run Scenario will run the scenarios in a separate process and published the results in the output window:

image

The debug option does as you would expect, and both options currently run all the scenarios in the selected file. This is the first version and is quite light on features, however there are some planned:

  1. Run a single scenario.
  2. Run from solution explorer.
  3. Syntax highlighting and completion for gherkin files.
  4. Full featured results window instead of output window.
  5. Go to step definition from scenario.
  6. Keyboard shortcuts.

I hope you find it useful, if you have any suggestions or bugs you can get me on twitter at @naeemkhedarun or on CodePlex at http://nbehave.codeplex.com

Happy coding!

NBehave: Executing a Scenario

NBehave can initially very strange to developers … after all, the idea of “executing” a text file is simply foreign.  So here’s a brief run-down of how NBehave works. Let's suppose we have the following scenario file:

Given I have entered 1 into the calculator
And I have entered 2 into the calculator
When I add the numbers
Then the sum should be 3

NBehave processes this line-by-line.  So it picks up "Given I have entered 1 into the calculator" and will try to match this against a suitable method.  Classes that can provide such methods are attributed with [ActionSteps].  Let’s suppose the NBehave locates the following class:

[ActionSteps]
public class AddNumbers
{
    private Calculator _calculator;

    [BeforeScenario]
    public void SetUp_scenario()
    {
        _calculator = new Calculator();
    }

    [Given(@"I have entered $number into the calculator")]
    public void Enter_number(int number)
    {
        _calculator.Enter(number);
    }

    [When(@"I add the numbers")]
    public void Add()
    {
        _calculator.Add();
    }

    [Then(@"the sum should be $result")]
    public void Result(int result)
    {
        _calculator.Value().ShouldEqual(result);
    }
}

So NBehave matches our line of text to the following method, using the regular expression in the [Given] attribute:

[Given(@"I have entered $number into the calculator")]
public void Enter_number(int number)
{
    _calculator.Enter(number);
}

The value of 1 is captured by the regex from the text string, so NBehave calls "Enter_number(1)".  Well actually, since this is the very first step in the scenario NBehave first looks for a [BeforeScenario] attribute and so "SetUp_scenario()" will be executed before "Enter_number(1)".

The next line of the scenario "And I have entered 2 into the calculator" is matched to the same method ("And" is treated as equivalent to "Given" in this case) and so "Enter_number(2)" gets executed.

And so on and so on.  So overall, the scenario:

Given I have entered 1 into the calculator
And I have entered 2 into the calculator
When I add the numbers
Then the sum should be 3

will get translated into:

var calc = new AddNumbers();
calc.SetUp_scenario();
calc.Enter_number(1);
calc.Enter_number(2);
calc.Add();
calc.Result(3);

The final result, which I hope is now clear, is that the scenario file is really just a human-readable way of expressing code!

April 24 2010

NBehave: Different ways to drive BDD tests / specs

My last post introduced NBehave and BDD as a natural evolution of unit testing.  Here I will show you the different ways in which you can express a spec using NBehave.

1. With Inline Lambdas

This is how I finished my previous post:

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     // CreateFeature omitted for clarity
   5:  
   6:     [TestMethod]
   7:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
   8:     {
   9:         Calculator calc = null;
  10:  
  11:         Feature.AddScenario()
  12:             .Given("a calculator",        () => calc = new Calculator())
  13:             .And("I have added 10",       () => calc.Add(10))
  14:             .And("I have added 10 again", () => calc.Add(10))
  15:             .When("I add another 10",     () => calc.Add(10))
  16:             .Then("the sum should be 30", () => Assert.AreEqual(30, calc.Result));
  17:     }
  18: }

This syntax is bearable for small lamba expressions, although you do usually end up fighting Visual Studio and Resharper to maintain the nice code layout.  This style also does not really promote reuse of the step definitions.

2. With the Steps in Suitably Named Methods

Here we pull the code out of lambda expressions and elevate it into methods.  Methods are located via reflecting on the object passed in on line 12 below.

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     // CreateFeature omitted for clarity
   5:  
   6:     [TestMethod]
   7:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
   8:     {
   9:         Calculator calc = null;
  10:  
  11:         Feature.AddScenario()
  12:             .WithHelperObject<CalculatorSpecificationSteps>()
  13:             .Given("a calculator")
  14:             .And("I have added 10")
  15:             .And("I have added 10")
  16:             .When("I add another 10")
  17:             .Then("the sum should be 30");
  18:     }
  19: }
  20:  
  21: public class CalculatorSpecificationSteps
  22: {
  23:     private Calculator _calculator;
  24:  
  25:     protected void Given_a_calculator()
  26:     {
  27:         _calculator = new Calculator();
  28:     }
  29:     
  30:     protected void And_I_have_added_10()
  31:     {
  32:         _calculator.Add(10);
  33:     }
  34:     
  35:     protected void When_I_add_another_10()
  36:     {
  37:         _calculator.Add(10);
  38:     }
  39:     
  40:     protected void Then_the_sum_should_be_30()
  41:     {
  42:         Assert.AreEqual(30, calc.Result);
  43:     }    
  44: }

The helper class can be the same class that holds your spec, or a base class, or a completely separate class altogether.  This style has better readability of the spec and also allows you to build up a “vocabulary” of reusable steps.  On the downside, it does constrain your method names.

3. With the Steps in Attributed Methods

We all know that loose coupling is a good thing, and our code may be better if we could differentiate our method names from their longer description.  We can do this by using attributes that contain regular expressions:

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     // CreateFeature omitted for clarity
   5:  
   6:     [TestMethod]
   7:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
   8:     {
   9:         Calculator calc = null;
  10:  
  11:         Feature.AddScenario()
  12:             .WithHelperObject<CalculatorSpecificationSteps>()
  13:             .Given("a calculator")
  14:             .And("I have added 10")
  15:             .And("I have added 10")
  16:             .When("I add another 10")
  17:             .Then("the sum should be 30");
  18:     }
  19: }
  20:  
  21: [ActionSteps]
  22: public class CalculatorSpecificationSteps
  23: {
  24:     private Calculator _calculator;
  25:  
  26:     [Given(@"a calculator")]
  27:     protected void SetupCalculator()
  28:     {
  29:         _calculator = new Calculator();
  30:     }
  31:     
  32:     [Given(@"I have added $number")]
  33:     [When(@"I add another $number")]
  34:     protected void Add(int number)
  35:     {
  36:         _calculator.Add(number);
  37:     }
  38:     
  39:     [Then(@"the sum should be $result")]
  40:     protected void ValidateResult(int result)
  41:     {
  42:         Assert.AreEqual(result, calc.Result);
  43:     }    
  44: }

So here I’ve used the same method for my given and my when clauses by applying two attributes.  I’ve also made use of NBehave’s ability to use regex captures to extract parameter values.  Very cool IMHO.

4. With the Scenario in an External File

In this style of spec, we put the following into a separate file (typically with a .scenario extension) and we still have the CalculatorSpecificationSteps class above:

   1: Feature: arithmetic addition
   2:  
   3: Scenario: Add numbers
   4:     Given a calculator
   5:     And I have added 10
   6:     And I have added 10
   7:     When I add another 10
   8:     Then the sum should be 30

We can then use the NBehave runner to execute this spec.  By this stage, you have probably worked out that this involves line-by-line parsing of the scenario and matching against attributed methods.

With this style of spec we have the best possible separation of intent and implementation and the readability of the specification is superb.  There is no code, funky punctuation, “dagger operators” or other cruft to try and ignore.  The downside is the need for a custom test runner (although NAnt and MSBuild tasks are provided) and a current lack of integration with TestDriven.Net and the Resharper test runner.

5. With a Table-based Scenario

The best way to think of this is as a data-driven specification.  And the best way to explain is by an example.  We take the external file above and modify as follows:

   1: Feature: arithmetic addition
   2:  
   3: Scenario: Add numbers
   4:     Given a calculator
   5:     And I have added [num1]
   6:     And I have added [num2]
   7:     When I add another [num3]
   8:     Then the sum should be [result]
   9:  
  10: Examples:
  11: |num1|num2|num3|result|
  12: |10|10|10|30|
  13: |20|20|20|60|
  14: |15|8|3|26|

This file actually represents 3 scenarios, and each scenario will get executed separately.  The values in the “Examples” section will get substituted into the scenario steps and eventually passed into the methods on the CalculatorSpecificationSteps class.

This approach allows us to very quickly and easily create a number of scenarios that differ only in the values being used, although perhaps at the expense of some readability.

Which one to use?

General advice is, of course, generally wrong and so I’m going to try hard not to give any.  However, if you develop an attributed ActionSteps class then it can be used in styles #3, #4 and #5.  Styles #4 and #5 also provide some ready-made human-readable documentation about your application and it’s possible that htye can be authored by a non-dev.  If there are reasons why you can’t (or don’t want to) use a custom test runner, then style #3 will is entirely compatible with  your test framework of choice (NUnit, MBUnit, Xunit or MSTest).

PS. only style #4 is available in NBehave v0.45.  The rest are available currently on the trunk and will be included in NBehave v0.5.

April 8 2010
Newer Posts Older Posts