NBehave: Executing a Scenario

NBehave can initially very strange to developers … after all, the idea of “executing” a text file is simply foreign.  So here’s a brief run-down of how NBehave works. Let's suppose we have the following scenario file:

Given I have entered 1 into the calculator
And I have entered 2 into the calculator
When I add the numbers
Then the sum should be 3

NBehave processes this line-by-line.  So it picks up "Given I have entered 1 into the calculator" and will try to match this against a suitable method.  Classes that can provide such methods are attributed with [ActionSteps].  Let’s suppose the NBehave locates the following class:

[ActionSteps]
public class AddNumbers
{
    private Calculator _calculator;

    [BeforeScenario]
    public void SetUp_scenario()
    {
        _calculator = new Calculator();
    }

    [Given(@"I have entered $number into the calculator")]
    public void Enter_number(int number)
    {
        _calculator.Enter(number);
    }

    [When(@"I add the numbers")]
    public void Add()
    {
        _calculator.Add();
    }

    [Then(@"the sum should be $result")]
    public void Result(int result)
    {
        _calculator.Value().ShouldEqual(result);
    }
}

So NBehave matches our line of text to the following method, using the regular expression in the [Given] attribute:

[Given(@"I have entered $number into the calculator")]
public void Enter_number(int number)
{
    _calculator.Enter(number);
}

The value of 1 is captured by the regex from the text string, so NBehave calls "Enter_number(1)".  Well actually, since this is the very first step in the scenario NBehave first looks for a [BeforeScenario] attribute and so "SetUp_scenario()" will be executed before "Enter_number(1)".

The next line of the scenario "And I have entered 2 into the calculator" is matched to the same method ("And" is treated as equivalent to "Given" in this case) and so "Enter_number(2)" gets executed.

And so on and so on.  So overall, the scenario:

Given I have entered 1 into the calculator
And I have entered 2 into the calculator
When I add the numbers
Then the sum should be 3

will get translated into:

var calc = new AddNumbers();
calc.SetUp_scenario();
calc.Enter_number(1);
calc.Enter_number(2);
calc.Add();
calc.Result(3);

The final result, which I hope is now clear, is that the scenario file is really just a human-readable way of expressing code!

April 24 2010

NBehave: Different ways to drive BDD tests / specs

My last post introduced NBehave and BDD as a natural evolution of unit testing.  Here I will show you the different ways in which you can express a spec using NBehave.

1. With Inline Lambdas

This is how I finished my previous post:

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     // CreateFeature omitted for clarity
   5:  
   6:     [TestMethod]
   7:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
   8:     {
   9:         Calculator calc = null;
  10:  
  11:         Feature.AddScenario()
  12:             .Given("a calculator",        () => calc = new Calculator())
  13:             .And("I have added 10",       () => calc.Add(10))
  14:             .And("I have added 10 again", () => calc.Add(10))
  15:             .When("I add another 10",     () => calc.Add(10))
  16:             .Then("the sum should be 30", () => Assert.AreEqual(30, calc.Result));
  17:     }
  18: }

This syntax is bearable for small lamba expressions, although you do usually end up fighting Visual Studio and Resharper to maintain the nice code layout.  This style also does not really promote reuse of the step definitions.

2. With the Steps in Suitably Named Methods

Here we pull the code out of lambda expressions and elevate it into methods.  Methods are located via reflecting on the object passed in on line 12 below.

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     // CreateFeature omitted for clarity
   5:  
   6:     [TestMethod]
   7:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
   8:     {
   9:         Calculator calc = null;
  10:  
  11:         Feature.AddScenario()
  12:             .WithHelperObject<CalculatorSpecificationSteps>()
  13:             .Given("a calculator")
  14:             .And("I have added 10")
  15:             .And("I have added 10")
  16:             .When("I add another 10")
  17:             .Then("the sum should be 30");
  18:     }
  19: }
  20:  
  21: public class CalculatorSpecificationSteps
  22: {
  23:     private Calculator _calculator;
  24:  
  25:     protected void Given_a_calculator()
  26:     {
  27:         _calculator = new Calculator();
  28:     }
  29:     
  30:     protected void And_I_have_added_10()
  31:     {
  32:         _calculator.Add(10);
  33:     }
  34:     
  35:     protected void When_I_add_another_10()
  36:     {
  37:         _calculator.Add(10);
  38:     }
  39:     
  40:     protected void Then_the_sum_should_be_30()
  41:     {
  42:         Assert.AreEqual(30, calc.Result);
  43:     }    
  44: }

The helper class can be the same class that holds your spec, or a base class, or a completely separate class altogether.  This style has better readability of the spec and also allows you to build up a “vocabulary” of reusable steps.  On the downside, it does constrain your method names.

3. With the Steps in Attributed Methods

We all know that loose coupling is a good thing, and our code may be better if we could differentiate our method names from their longer description.  We can do this by using attributes that contain regular expressions:

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     // CreateFeature omitted for clarity
   5:  
   6:     [TestMethod]
   7:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
   8:     {
   9:         Calculator calc = null;
  10:  
  11:         Feature.AddScenario()
  12:             .WithHelperObject<CalculatorSpecificationSteps>()
  13:             .Given("a calculator")
  14:             .And("I have added 10")
  15:             .And("I have added 10")
  16:             .When("I add another 10")
  17:             .Then("the sum should be 30");
  18:     }
  19: }
  20:  
  21: [ActionSteps]
  22: public class CalculatorSpecificationSteps
  23: {
  24:     private Calculator _calculator;
  25:  
  26:     [Given(@"a calculator")]
  27:     protected void SetupCalculator()
  28:     {
  29:         _calculator = new Calculator();
  30:     }
  31:     
  32:     [Given(@"I have added $number")]
  33:     [When(@"I add another $number")]
  34:     protected void Add(int number)
  35:     {
  36:         _calculator.Add(number);
  37:     }
  38:     
  39:     [Then(@"the sum should be $result")]
  40:     protected void ValidateResult(int result)
  41:     {
  42:         Assert.AreEqual(result, calc.Result);
  43:     }    
  44: }

So here I’ve used the same method for my given and my when clauses by applying two attributes.  I’ve also made use of NBehave’s ability to use regex captures to extract parameter values.  Very cool IMHO.

4. With the Scenario in an External File

In this style of spec, we put the following into a separate file (typically with a .scenario extension) and we still have the CalculatorSpecificationSteps class above:

   1: Feature: arithmetic addition
   2:  
   3: Scenario: Add numbers
   4:     Given a calculator
   5:     And I have added 10
   6:     And I have added 10
   7:     When I add another 10
   8:     Then the sum should be 30

We can then use the NBehave runner to execute this spec.  By this stage, you have probably worked out that this involves line-by-line parsing of the scenario and matching against attributed methods.

With this style of spec we have the best possible separation of intent and implementation and the readability of the specification is superb.  There is no code, funky punctuation, “dagger operators” or other cruft to try and ignore.  The downside is the need for a custom test runner (although NAnt and MSBuild tasks are provided) and a current lack of integration with TestDriven.Net and the Resharper test runner.

5. With a Table-based Scenario

The best way to think of this is as a data-driven specification.  And the best way to explain is by an example.  We take the external file above and modify as follows:

   1: Feature: arithmetic addition
   2:  
   3: Scenario: Add numbers
   4:     Given a calculator
   5:     And I have added [num1]
   6:     And I have added [num2]
   7:     When I add another [num3]
   8:     Then the sum should be [result]
   9:  
  10: Examples:
  11: |num1|num2|num3|result|
  12: |10|10|10|30|
  13: |20|20|20|60|
  14: |15|8|3|26|

This file actually represents 3 scenarios, and each scenario will get executed separately.  The values in the “Examples” section will get substituted into the scenario steps and eventually passed into the methods on the CalculatorSpecificationSteps class.

This approach allows us to very quickly and easily create a number of scenarios that differ only in the values being used, although perhaps at the expense of some readability.

Which one to use?

General advice is, of course, generally wrong and so I’m going to try hard not to give any.  However, if you develop an attributed ActionSteps class then it can be used in styles #3, #4 and #5.  Styles #4 and #5 also provide some ready-made human-readable documentation about your application and it’s possible that htye can be authored by a non-dev.  If there are reasons why you can’t (or don’t want to) use a custom test runner, then style #3 will is entirely compatible with  your test framework of choice (NUnit, MBUnit, Xunit or MSTest).

PS. only style #4 is available in NBehave v0.45.  The rest are available currently on the trunk and will be included in NBehave v0.5.

April 8 2010

My journey to BDD using NBehave

In the beginning, there was some code:

   1: public class Calculator
   2: {
   3:     private int _accumulator;
   4:  
   5:     public int Result
   6:     {
   7:         get { return _accumulator; }
   8:     }
   9:  
  10:     public void Add(int n)
  11:     {
  12:         _accumulator += n;
  13:     }
  14:  
  15:     public void Subtract(int n)
  16:     {
  17:         _accumulator -= n;
  18:     }
  19: }

[Well obviously the code I had written was a bit more complicated than this … but this should do for the sake of illustration]

And I realised that code was legacy code, and so I wrote some unit tests:

   1: [TestClass]
   2: public class CalculatorTests
   3: {
   4:     [TestMethod]
   5:     public void TestTheCalculator()
   6:     {
   7:         var calc = new Calculator();
   8:         calc.Add(10);
   9:         Assert.AreEqual(10, calc.Result);
  10:         calc.Add(5);
  11:         Assert.AreEqual(15, calc.Result);
  12:         calc.Subtract(7);
  13:         Assert.AreEqual(8, calc.Result);
  14:     }
  15: }

And after writing quite a few tests in this style, I started to realise just how horrible they are.  Firstly, the name of the test tells me nothing at all; secondly, the various operations and assertions all depend on each other; and thirdly, I need to read and understand all the code to actually get my head around the intent of the test.  And so I started using a new, simpler style:

   1: [TestClass]
   2: public class CalculatorTests
   3: {
   4:     [TestMethod]
   5:     public void Add_AddsNumberToTheAccumulator_ResultIsCorrect
   6:     {
   7:         var calc = new Calculator();
   8:         calc.Add(10);
   9:         Assert.AreEqual(10, calc.Result);
  10:     }
  11:  
  12:     [TestMethod]
  13:     public void Subtract_SubtractsNumberFromTheAccumulator_ResultIsCorrect
  14:     {
  15:         var calc = new Calculator();
  16:         calc.Subtract(7);
  17:         Assert.AreEqual(-7, calc.Result);
  18:     }
  19: }

Better!  But what if the Add method has some nasty side effect that results in it breaking on the third call?  And what if there is some quirk in the Subtract method that means it will not operate on the number 13? (IOW what happens when the tests get more complex).  In the same style as above, the tests become:

   1: public class Calculator
   2: {
   3:     public Calculator() { }
   4:  
   5:     public Calculator(int start) 
   6:     {
   7:         _accumulator = start;
   8:     }
   9:  
  10:     // Other code omitted for clarity
  11: }
  12:  
  13: [TestClass]
  14: public void CalculatorTests
  15: {
  16:     [TestMethod]
  17:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
  18:     {
  19:         var calc = new Calculator();
  20:         calc.Add(10);
  21:         calc.Add(10);
  22:         calc.Add(10);
  23:         Assert.AreEqual(30, calc.Result);
  24:     }
  25:  
  26:     [TestMethod]
  27:     public void Subtract_ApplyToThirteen_ResultIsStillCorrect()
  28:     {
  29:         var calc = new Calculator(13);
  30:         calc.Subtract(5);
  31:         Assert.AreEqual(8, calc.Result);
  32:     }
  33: }

The test relating to addition looks a little … odd.  If I hadn’t written that test then I honestly wouldn’t know what the point of it was.  And the second test?  Well, you notice that we are passing in some state and assigning it to our private field.  It’s very common to do that with Dependency Injection, but this is state, not a dependency, and so we should question it closer.  It becomes an increasingly valid concern as the state gets more and more complex.  After some careful thought, I removed the new constructor and rewrote these tests as:

   1: public void Calculator
   2: {
   3:     public void Reset()
   4:     {
   5:         _accumulator = 0;
   6:     }
   7:  
   8:     // Other code omitted for clarity
   9: }
  10:  
  11: [TestClass]
  12: public void CalculatorTests
  13: {
  14:     [TestMethod]
  15:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
  16:     {
  17:         // ARRANGE
  18:         var calc = new Calculator();
  19:         calc.Add(10);
  20:         calc.Add(10);
  21:  
  22:         // ACT
  23:         calc.Add(10);
  24:  
  25:         // ASSERT
  26:         Assert.AreEqual(30, calc.Result);
  27:     }
  28:  
  29:     [TestMethod]
  30:     public void Subtract_ApplyToThirteen_ResultIsStillCorrect()
  31:     {
  32:         // ARRANGE
  33:         var calc = new Calculator();
  34:         calc.SetupToBeThirteen();
  35:  
  36:         // ACT
  37:         calc.Subtract(5);
  38:  
  39:         // ASSERT
  40:         Assert.AreEqual(8, calc.Result);
  41:     }
  42: }
  43:  
  44: public static CalculatorExtensions
  45: {
  46:     public static void SetupToBeThirteen(this Calculator calc)
  47:     {
  48:         calc.Reset();
  49:         calc.Add(13);
  50:     }
  51: }

That certainly feels better.  I no longer have a class that is willy-nilly accepting state from unknown others, I’ve added a method which feels like it should be there anyway, my setup code is neatly wrapped up into a reusable and readable extension method, and the intent of the addition test is much clearer.

Now I’m of the opinion that once you grok and apply the AAA (Arrange, Act, Assert) syntax, then BDD is largely just a translation of terms you already know.  But if it’s so simple, then why bother at all?  Well, I found that using NBehave I can rewrite the above tests like so:

   1: [TestClass]
   2: public void CalculatorTests : ScenarioDrivenSpecBase
   3: {
   4:     protected override Feature CreateFeature()
   5:     {
   6:         return new Feature("arithmetic calculation")
   7:             .AddStory()
   8:             .AsA("user")
   9:             .IWant("my calculator to perform basic arithmetic")
  10:             .SoThat("I don't need to try and do it in my head");
  11:     }
  12:  
  13:  
  14:     [TestMethod]
  15:     public void Add_RepeatedAddition_CheckThatThreeTriesDoNotFail()
  16:     {
  17:         Calculator calc = null;
  18:  
  19:         Feature.AddScenario()
  20:             .Given("a calculator",        () => calc = new Calculator())
  21:             .And("I have added 10",       () => calc.Add(10))
  22:             .And("I have added 10 again", () => calc.Add(10))
  23:             .When("I add another 10",     () => calc.Add(10))
  24:             .Then("the sum should be 30", () => Assert.AreEqual(30, calc.Result));
  25:     }
  26:  
  27:     [TestMethod]
  28:     public void Subtract_ApplyToThirteen_ResultIsStillCorrect()
  29:     {
  30:         Calculator calc = null;
  31:  
  32:         Feature.AddScenario()
  33:             .Given("a calculator",                () => calc = new Calculator())
  34:             .And("which has a total of 13",       () => calc.SetupToBeThirteen())
  35:             .When("I perform a subtraction",      () => calc.Subtract(5))
  36:             .Then("the result should be correct", () => Assert.AreEqual(8, calc.Result));
  37:     }
  38: }
  39:  

And to me this represents a big step forward, even though it’s pretty easy to recast the previous tests in this style.  We are now linking tests directly to scenarios, and scenarios directly to user stories.  It becomes immensely clear why this test is present and what it’s hoping to achieve.  And on top of that, the tests become immensely readable.

[I would actually rename these tests before I was happy … I prefer the “ShouldDoSomething” style of test name … and there are also much cleaner ways of coding the assertions]

You may ask if this level of attention to detail is appropriate in tests around a simple calculator class.  The craftsman in me insists that if it’s worth doing, then it’s worth doing properly.  But that aside, my current project (4,000 tests and counting) has found that if tests are not scrupulously well written then they become a source of friction.  Applying this style of BDD reduced the friction and took away the drag factor.

Interestingly there are actually other ways of casting these tests with NBehave, and I will be posting about those soon, but this concludes the story of my personal journey into BDD.

PS. This NBehave syntax is available in the trunk and will be included in version 0.5.  If you can’t wait then you can download the assemblies from the build server.

April 7 2010
Newer Posts Older Posts