Friday, August 21, 2009

Gallio – Visual Studio Integration – Issue with output of data-driven tests

August 21, 2009 Posted by Jason Irwin , , , , , , 3 comments

I recently came across the Gallio platform, a “common object model, runtime services and tools (such as test runners) that may be leveraged by any number of test frameworks”. Essentially Gallio acts as a one stop shop for test integration and the “killer feature” for me is it’s integration into the Visual Studio test view. For those of us that do not have access to Resharper or Testdriven.NET (and have no way of running anything but MSTest tests directly through Visual Studio), Gallio’s offering is very appealing. It allows the user to run any test type supported by the Gallio platform (MbUnit, NUnit, xUnit, etc.) through Visual Studio as if they were MSTest unit tests – brilliant.

For instance, the following screenshot shows a few simple MbUnit tests in Visual Studio, as well as the test run results in the Test Results window…


An unfortunate limitation of the mismatch between MSTest’s restricted feature set and MbUnit’s far more expressive set is that some more advanced features are lost in translation.

  1: <Test()> _
  2: <MultipleAsserts()> _
  3: <Row(3, 3)> _
  4: <Row(3, 17)> _
  5: <Row(1, 2)> _
  6: Public Sub MyRowTest(ByVal val1 As Int32, ByVal val2 As Int32)
  8:     Assert.AreEqual(Of Int32)(val1, val2)
 10: End Sub

For instance, the above rowtest should run three times and fail twice. However it completes successfully with the following message:

MyRowTest(3,3) Duration: 0.016s,

Assertions: 2

Luckily for me, I posted the issue (ineloquently) on Stackoverflow, and Jeff Brown (MbUnit and Gallio lead) not only answered but logged the issue as an enhancement for an upcoming milestone. Perfecting this integration would make Gallio a must for us developers with a limited tooling budget who want to integrate non MSUnit tests into our .NET projects and do not fancy ALT-tabbing all over the place.