Lack of Coverage Tools for .NET

For the longest time, I've been using NCover for coverage. It was free, gave good results and with NCoverExplorer and CI integration it was the perfect tool. Easy to use, easy to setup, and worth the typical hassle of new tools (setup, learning, configuration, etc.)

NCover has since gone commercial and the old versions won't run properly against 3.0/3.5 code. I'm ditching NCover support in TreeSurgeon because of this. TS nees to be friction free and something anyone can just grab and run and not have to deal with buying or obtaining licenses for products the generated code depends on. I looked at Clover.NET as an alternative (last time I checked it was free?) but it's $600 just for an "academic" version.

So what's with coverage and .NET these days? Are these the only options? Have all the open source/free tools gone the way of the Dodo for coverage and .NET 3.0 projects? My quick Google checks this freezing morning (it was minus 38 as I drove in this morning, that's -36 in Fahrenheit kids) don't show anything.

To be friction-free a code coverage tool needs to be:

  • Command line driven (have to be able to integrate into continuous integration process)
  • Xml report output (again, CI ready)
  • Usable with any unit testing framework
  • Support 3.0/3.5 .NET framework
  • Somewhat stable (i.e. non-alpha/beta versions)

Nice to haves:

  • Free (or relatively cheap)
  • HTML or Text output
  • Ability to set thresholds and get failures back (for example, break the build if the coverage falls below 80%)

Looking for some feedback on your experience here. Thanks!

7 Comments

  • Maybe someone needs to start another OSS project for code coverage? I'd be willing to pitch in...

  • You make a very good point sir. This has definately been a pain point for me. Especially since ncover was sucha great tool. How much are they actually charging for the commercial version?

  • @Sean: NCover was a great tool but now it's useless with 3.0 code projects (unless you buy the new version which I assume works with 3.0). Cost isn't huge (compared to Clover) and a copy will run you $150USD. However I consider that pretty pricey for the features you get back. Compare it with something like R# which is about the same price but has 10x more functionality. A code coverage tool is just that and should run in the $50 price range.

  • This is generally one of the problems I have with so many developer tools. I understand that they have a somewhat limited audience, but anything over $50 puts it outside of my pain threshhold.

    I've bought things like ultraedit, beyond compare, filesync and such, because they do something for me and the price point of $25-30 makes it a "No duh" purchase.

    But when someone is asking $150 or more, it's a struggle to justify, especially given most developer tools only last a year or so before the underlying technology changes beneath you and you have to buy a new copy and in most cases that appears to mean a very limited discount.

    It'd be one thing if my company would buy us tools, but we've got no formal process for this. I spent six months trying to fight for a copy of redgate's profiler and was never able to get it.

  • I think the problem is that a.) code coverage is so hard in .NET due to the crazy COM stupidity hoops you have to go through to do anything useful b.) now that automated unit testing is gaining adoption, PHB's eager to gauge the productivity of their developers have turned from LOC counting to Coverage counting (or both -- with tools like NDepend and Clover.NET/NCover).

    I'm not saying NDepend and Clover/NCover are bad, quite the contrary, but I think there's real money to be made by people who are trying to apply command and control-style management to unit testing.

    I'm also not saying that is bad, but it leads to a lucrative market with a steep barrier to entry (due to the COM craziness involved in making a CLR profiler/code-coverage analyzer).

  • To jump on the band wagon, I've been meaning to look at PartCover for my current contract. If you get there before me, let me know. :)

  • You can see if this list if there are code coverage tools for .NET that you don't know: http://sharptoolbox.com/categories/code-coverage

Comments have been disabled for this content.