Tuesday 15 June 2010

Pex and Moles

Pex and Moles are Visual Studio 2010 Power Tools extension that help Unit Testing .NET applications.

  • Pex automatically generates test suites with high code coverage. Right from the Visual Studio code editor, Pex finds interesting input-output values of your methods, which you can save as a small test suite with high code coverage. Microsoft Pex is a Visual Studio add-in for testing .NET Framework applications.Moles allows to replace any .NET method with a delegate.
  • Moles supports unit testing by providing isolation by way of detours and stubs. The Moles framework is provided with Pex, or can be installed by itself as a Microsoft Visual Studio add-in.
Microsoft Moles helps with writing unit tests, but what remains is the tedious task of writing the specific unit tests that exhaustively exercise and validate the logic of the code-under-test. Microsoft Pex can help you in understanding the input/output behavior of your code, finding inputs that cause the code-under-test to crash, and exploring parameterized unit tests to check whether your code implements the desired functionality for all inputs.
When you run Microsoft Pex on your .NET code, Pex generates test cases by analyzing the code-under-test. For every statement in the code, Pex will eventually try to create a test input that will reach that statement. Pex will do a case analysis for every conditional branch in the code—for example, if statements, assertions, and all operations that can throw exceptions.
The result appears as a table that shows each test attempted by Pex. Each line in the table shows the input and the resulting output or exceptions generated by the method for the specific input.

This is great, automatic unit test generation and complete code coverage!
You can download it here

Ok been using for a while and my verdict is: Pex rocks!
I can't believe I've never known about it until now!
Pex generates MSTests or NUnit tests. MSTests are default, but if your project already has NUnit tests, it will use those.
Here is a video you should watch.
Pex first records your codes responses, and generates ASSERTS, so in future, these can be used as regression tests. It achieves 100% code coverage by analysing your code, and cleverly generating tests. Very impressive.

Pex generates edge cases, and every combination of null and empty strings to each parameter, etc, which is not what you'd like to see pollute your nice and clean nUnit tests. Plus, those would take forever to make. So by having your hand crafted nUnit tests and Pex generated MSTests separate its best of both worlds.
Resharper will also happily run both nUnit and MSTests in your solution together and gives you the green/red lights as normal.
In a couple of hours, Pex has so far generated over 1100 lines of MSTest C# code for me. And I didn't have to write a line of code (apart from me having to fix my C# code when Pex found a fault with it).
I like it very much as it allows me to concentrate on building nUnit tests and not mind-numbing combinatorial parameter checking :-)

Automated buildsUsing FinalBuilder or CruiseControl to automate your builds and run your tests? In order to have the generated tests execute on a machine that does not have Pex installed, you can check in into your depot the following assemblies:
  • Microsoft.Pex.Framework.dll
  • Microsoft.ExtendedReflection.dll


Here is a picture of Pex in action:
pex