View on GitHub

Verde

Integration test and application health verification framework for ASP.Net MVC

Download this project as a .zip file Download this project as a tar.gz file

Overview

Verde is an integration test and application health verification framework for ASP.Net MVC. Integration tests are intended to automate verifying the different components of your application such as databases, web services, etc. are working together correctly. Unlike test runners integrated into Visual Studio or standalone applications, the Verde framework runs tests directly within your web application; leveraging the same configuration settings, security permissions, network topology, etc. This allows exposing issues that may not otherwise be encountered until an end-user actually starts using the application. Verde is intended to complement a robust suite of unit tests; the unit tests ensure all the individual units of work function correctly, while integration tests ensure everything works together.

The easiest way to add Verde to your project is via NuGet - https://nuget.org/packages/Verde/0.5.1

PM> Install-Package Verde

Getting Started

Verde provides a browser based GUI based on the the QUnit JavaScript test framework.

Screenshot

The GUI in turn makes AJAX calls to a RESTful endpoint on your server to execute tests and get back the results as a JSON data structure. These same RESTful endpoints can also be invoked directly by an automated deployment script or application monitoring service.

The framework is designed to be easily dropped into an existing MVC application with minimal configuration overhead, i.e., no changes to your web.config and routing rules are automatically injected by the framework (modeled after the MvcMiniProfiler). Basically all you need to do is add a reference to the Verde library and add a small bit to the Application_Start event in your global.asax.

// Minimal setup
Verde.Setup.Initialize(new Verde.Settings
{
    TestsAssembly = System.Reflection.Assembly.GetExecutingAssembly()
});

The minimal setup requires you to only provide a reference to the assembly where your integration tests are defined. In this case the tests are assumed to reside in the MVC Application itself, but they could just as easily be in a dedicated class library. There are several other settings you can override if you choose, but none are required. You could always wrap this statement in a conditional block so that the Verde framework is enabled based on a custom configuration setting. Additionally you can choose to authorize access to the Verde endpoints by providing a custom AuthorizationCheck delegate:

Verde.Setup.Initialize(new Verde.Settings
{
    TestsAssembly = System.Reflection.Assembly.GetExecutingAssembly(),
    AuthorizationCheck = (context) =>
    {
        return context.User.IsInRole("admin");
    }
});

The Initialize method automatically registers several MVC routes in your application's route table. By default these routes are registered under a "@integrationtests" path, but you can override this using Settings.RoutePath.

If no querystring parameters are provided to the /execute command, all tests are run. However you can also limit what tests are run to a single fixture or a single test in a fixture by appending the fixture and/or test parameters to the URL like so: /@integrationtests/execute?fixture=MvcMusicStore.IntegrationTests.ShoppingCart or /@integrationtests/execute/fixture=MvcMusicStore.IntegrationTests.ShoppingCart&test=RemoveFromCart_ValidJson

Here is a sample JSON response from the /execute command:

{
  "duration": 823,
  "failed": true,
  "tests": [
    {
      "testName": "AddToCart_ValidItem_Succeeds",
      "fixture": "MvcMusicStore.IntegrationTests.ShoppingCart",
      "duration": 219,
      "failed": false,
      "message": "Passed"
    },
    {
      "testName": "ViewCart_ExpectedHtml",
      "fixture": "MvcMusicStore.IntegrationTests.ShoppingCart",
      "duration": 430,
      "failed": false,
      "message": "Passed"
    },
    {
      "testName": "RemoveFromCart_ValidJson",
      "fixture": "MvcMusicStore.IntegrationTests.ShoppingCart",
      "duration": 175,
      "failed": true,
      "message": "  The shopping cart should have 0 items left.\r\n  Expected: 1\r\n  But was:  0\r\n\r\n"
    }
  ]
}

Now all that's left is to write some tests.

Writing Tests

Verde integration tests are simply NUnit tests decorated with the IntegrationTest attribute declared within a class decorated by the IntegrationFixture attribute. When a test executes, it in turn creates a nested handler corresponding to the application URL being tested. After the method completes, control is returned back to the current request. The bit that makes this possible is the HttpServerUtility.Execute method. The Verde.Executor namespace defines a MvcExectorScope class that is used to define the scope of this nested execution.

The source code includes a fork of Jon Galloway's MvcMusicStore sample app that has been augmented with Verde integration tests. Here are some examples:

Basic ViewResult

In this test we get an arbitrary Album and define a MvcExecutorScope for the details page of that album. It is highly recommended that a using statement be implemented to control the lifetime of the ExecutorScope. Inside the using block, the scope variable provides us access to some helpful contextual objects like the HttpContext, Controller, ViewData, Action, and ResponseText that the test can make assertions against. The ScrapySharp library, in conjunction with the HtmlAgilityPack (both easily installable via NuGet), provides a CSS selector engine that allows the test to examine the contents of the HTML ResponseText in a more elegant way than brute force string matching.

[IntegrationTest]
public void Index_Load_ExpectedHtml()
{
    // Get a product to load the details page for.
    var album = storeDB.Albums
        .Take(1)
        .First();

    using (var scope = new MvcExecutorScope("Store/Details/" + album.AlbumId))
    {
        Assert.AreEqual(200, scope.HttpContext.Response.StatusCode);
        Assert.IsTrue(scope.Controller is StoreController);
        Assert.AreEqual("Details", scope.Action);

        var model = scope.Controller.ViewData.Model as Album;
        Assert.IsNotNull(model);
        Assert.AreEqual(album.AlbumId, model.AlbumId);

        Assert.IsFalse(String.IsNullOrEmpty(scope.ResponseText));

        // Load the ResponseText into an HtmlDocument
        var html = new HtmlDocument();
        html.LoadHtml(scope.ResponseText);

        // Use ScrappySharp CSS selector to make assertions about the rendered HTML
        Assert.AreEqual(album.Title, html.DocumentNode.CssSelect("#main h2").First().InnerText);
    }
}

RedirectAction

We can also test action results that do a redirect.

[IntegrationTest]
public void AddToCart_ValidItem_Succeeds()
{
    // Get a product to load the details page for.
    var album = storeDB.Albums
        .Take(1)
        .First();

    var settings = new ExecutorSettings("ShoppingCart/AddToCart/" + album.AlbumId) { 
        User = new GenericPrincipal(new GenericIdentity("GenghisKahn"), null) 
    };

    using (var scope = new MvcExecutorScope(settings))
    {
        Assert.AreEqual(302, scope.HttpContext.Response.StatusCode);
        Assert.AreEqual("/ShoppingCart", scope.HttpContext.Response.RedirectLocation);

        // Now verify that the cart contains the item we just added.
        var cart = MvcMusicStore.Models.ShoppingCart.GetCart(scope.HttpContext);
        var cartItems = cart.GetCartItems();
        Assert.AreEqual(1, cartItems.Count);
        Assert.AreEqual(album.AlbumId, cartItems[0].AlbumId);

        // Finally clear the cart.
        cart.EmptyCart();
    }
}

POST Request and JSON Responses

We can also test a POST request and Json action results.

[IntegrationTest]
public void RemoveFromCart_ValidJson()
{
    // Add an item to the cart so we have something to remove.
    string userName = "JimmyHendrix";
    MvcMusicStore.Models.ShoppingCart cart = TestUtil.AddItemsToCart(
        userName, storeDB.Albums.Take(1));
    var recordId = cart.GetCartItems().First().RecordId;                       

    var settings = new ExecutorSettings("ShoppingCart/RemoveFromCart/" + recordId) 
    { 
        User = TestUtil.CreateUser(userName), 
        HttpMethod = "POST"
    };

    using (var scope = new MvcExecutorScope(settings))
    {
        Assert.AreEqual("application/json", scope.HttpContext.Response.ContentType);

    // Use JSON.Net to deserialize the response
        var deserializedResponse = JsonConvert.DeserializeObject<ShoppingCartRemoveViewModel>
           (scope.ResponseText);
        Assert.AreEqual(0.0d, deserializedResponse.CartTotal, 
           "The shopping cart total should be $0.00.");
        Assert.AreEqual(0, deserializedResponse.ItemCount, 
           "The shopping cart should have 0 items left.");
        Assert.AreEqual(recordId, deserializedResponse.DeleteId);
    }
}

More Details

Sequence DiagramView Larger

What's with the name?

In case it's not obvious, verde is Spanish for green, passing tests are green...clever right?