Unit Testing with Silverlight

One of the important capabilities we shipped with the Beta1 release of Silverlight 2 was a unit test harness that enables you to perform both API-level and UI-level unit testing.  This testing harness is cross browser and cross platform, and can be used to quickly run and verify automated unit tests:

In addition to shipping this unit test harness for Silverlight, we also shipped the source to ~2,000 unit tests built with it that provide automated coverage for the Silverlight control source that we also shipped under a permissive license (you can take the control source, modify it, run the unit tests to verify the behavior, then re-ship the controls however you want).

Learning How to Unit Test Silverlight

Jeff Wilcox (who developed the Silverlight unit test framework and harness) has a great blog post that talks about how to add a Silverlight Unit Test project to a solution here. You can download the chat application that he shows testing from this expression blend blog post tutorial I did last month.  You can also watch this cool video post that Jeff created where he walks through the unit test framework and test cases we've shipped.

As Jeff shows in his post, you can now add a "Silverlight Test Project" to your Visual Studio solution which encapsulates unit tests for an application you are working on:

You can then add unit test classes to the test project that test APIs or simulate UI action within the Silverlight controls (simulate button clicks, etc).

You can then run the test project and execute the tests within it to verify and report their status.

Jeff's test framework automatically provides a browser based test harness and reporting system (which means you can run it on any browser/OS combination that Silverlight runs on):

Jeff's test framework supports quickly re-setting controls after each test (and avoids needing to re-launch a new browser instance for each test cases - which makes it really fast).

You can quickly rip through hundreds or thousands of automated tests in seconds:

Green results mean the tests passed.  Red results flag that a test case failed and log the assertion failure and/or runtime exceptions that occurred.

Summary

If you've ever struggled to try and come up with a strategy for doing automated unit testing or TDD with AJAX applications, I think you'll find Silverlight provides some much nicer test options.  Using Visual Studio you can also separate your tests into a separate project in your solution, and you do not need to embed the tests within your Silverlight application in order for them to run.

In addition to supporting the above unit test harness and framework, we are also going to support UI automation APIs with the final release of Silverlight 2.  These will enable accessibility scenarios (allowing screen readers to work with Silverlight and enable Section 508 compliance of Silverlight applications).  These UI automation APIs will also enable UI testing scenarios where you can build end to end browser UI automation that simulates real mouse and keyboard interactions and enables automated end to end experience testing.  The combination should enable you to build much more solid and maintainable RIA solutions.

Hope this helps,

Scott

P.S. For more tutorial posts and links on Silverlight 2, check out my new "Silverlight Tips, Tricks, Tutorials and Links" page.

35 Comments

  • Really valuable post, thanks.

  • Nice post :-)

    Btw, you really should stop talking to yourself on the chat client :-)

  • Having something that dumps unit testing results to a html page for WebForms/MVC would be great - Rspec does it with Rails and it looks great and I got rather jealous.

  • Thanks for the tutorial!

    Christian Pfeil

  • Nice!
    Will we see something like this for standard WPF applikations as well? UI automation APIs for WPF apps and WinForm apps would be very usefull for testing.

  • Would this mean that the unit tests are also send down to the client, since the testing harness runs locally?

  • Good post, thanks.

  • Thanks Scott. This is some great info.

  • Hey Scott
    This is great for testing View based components and controls. However ideally I would love to be able to test my Model/Controller/Utility Classes within Visual Studio / MSTest like I can with code not built on the Core CLR.
    TestDriven.NET have a way of doing this, although AFAIK it doesn't work with the latest Silverlight 2 Beta.
    It would be great to be able to host CoreCLR assemblies within a CLR process and have the two communicate (a bit like the SL's HTML Bridge). Do MS have any plans to provide this?

  • Great info, thanks.

  • How can I do this from my msbuild file? On a continuous integration server? Does it *need* a browser?

  • Good to see more example of unit testing.

  • 'daveloper',
    To your question about whether unit tests are also sent down to the client, they are only if you include the tests in the same Silverlight application project.

    I'd recommend creating a separate test project (details in the tutorial), your tests are then in one .Xap file - Test.Xap - and your app is in another, App.Xap.

    -Jeff

  • nmosafi,
    That's good feedback about being able to run Core CLR tests from within Visual Studio.

    One question I have for you: if you had a way to test "Silverlight" applications right in Visual Studio, you'd only be able to test basic APIs and class libraries: the HTML DOM bridge, the entire rich presentation framework in Silverlight, all those things would -not- be accessible to those tests. Would that be acceptable?

    In my opinion, I'm more interested in exercising those Silverlight-specific features in the browser, since I can just as easily compile the same source files in my regular desktop CLR and run them as regular Visual Studio test projects to verify that my code is working fine.

    - Jeff

  • Thanks for the tutorial!!!

  • jwilcox,

    Thanks for clearing that up!

    TDD with ASP.NET UI components isn't as intuitive compared to logic layers, this really helps out.

  • How do I get the cool background of the editor that you are showing above in an example. Off the topic..sorry :-)

  • Why are you using a Mac? Aren't Microsoft people supposed to hate them? Are you informally admitting Microsoft's inferiority? Or did you just want to demonstrate the cross-platform capabilities of Silverlight?

  • Hi Scott,

    Is there any tutorial on the UNIT TEST functionality built within VS 2008? Are you planning to take up any tutorials on the same?

  • Hi Scott,

    Great job on providing the unit test for Silverlight! I think this will really help the .net community adopt unit testing and TDD as common practices by seeing what Micrsoft themselves do. When can we expect the unit tests for ASP.NET MVC? Last I heard you needed to work on the mock framework that ASP.NET MVC was using.

    Dave Parslow

  • Helped a lot! Thanks. Back to work.
    - There should be a Scott fan-site.

  • Hi Ashutosh,

    >>>>>> Is there any tutorial on the UNIT TEST functionality built within VS 2008? Are you planning to take up any tutorials on the same?

    Stephen Walther has a great post here that should help: http://weblogs.asp.net/stephenwalther/archive/2008/03/19/tdd-test-driven-development-with-visual-studio-2008-unit-tests.aspx

    Thanks,

    Scott

  • Hi Dave,

    >>>>> Great job on providing the unit test for Silverlight! I think this will really help the .net community adopt unit testing and TDD as common practices by seeing what Micrsoft themselves do. When can we expect the unit tests for ASP.NET MVC? Last I heard you needed to work on the mock framework that ASP.NET MVC was using.

    There will be an update to ASP.NET MVC later this month that makes testing controllers much easier, and won't require mocking any objects to-do so.

    Thanks,

    Scott

  • To Everyone,

    Make sure to check out Jeff's answers above - he has jumped in and answered several of the earlier questions on unit testing.

    Thanks,

    Scott

  • I am really linking Silverlight

  • Hi Jeff

    I think a combination of both types of testing would be good. If tests can be run from MSTest then I can integrate them into my build process. Those tests which have a dependency on the browser to run would not be able to run via MSTest and would require the browser.

    Also a good point that you could compile the same source files as a normal CLR assembly and test them that way. I might try that!

    Neil

  • Hi Tom,

    >>>>>>> How do I get the cool background of the editor that you are showing above in an example. Off the topic..sorry :-)

    Here is a pointer to my VS settings file if you want the same VS color scheme: http://www.scottgu.com/blogposts/extractusercontrol/vssettings.zip

    Hope this helps,

    Scott

  • Why only be able to run and see results of unit tests from the browser? It would be useful to be able run from the command line and generate a results file so that I can include in my CI process! Please don't assume we are all vendor-locked into Team System for CI either.

  • Nice post, ready to give it the old tire kick and test drive.

  • Hi,

    Great post...
    I have one question though...is there a place where i can see sample code for testing something asynchronously...something like testing the play() for a MediaElement...

  • Raviraj,
    I'll post an asynchronous tutorial to my blog soon. In the meantime, try:

    1. Having your test class inherit from SilverlightTest base, inside Microsoft.Silverlight.Testing namespace
    2. Marking the test method for asynchronous testing with the [Asynchronous] attribute, in the same namespace
    3. Inside your test call the play() method on your MediaElement. Hook up to an appropriate event for the player, OR use a DispatcherTimer, etc., to verify that everything is OK and make any assertions. In the method with the verification at the end of testing, call TestComplete(). This is a method that is exposed by the SilverlightTest base class.

    Hope this helps.
    -Jeff

  • Do you have any estimate on when the UI automation interface will be available? We need this ASAP to implement browser based tests of Silverlight apps which we didn't build.

  • Very useful post, along with the comments. On the subject of testing Silverlight, are there any UI Test Harness tool from Microsoft that is publicly available for driving UI test Automation?

  • I have to say, coming from adobe flash ui development, this is quite a leap regarding testing.
    I am very happy with it so far.

    A

  • This post is nice!

Comments have been disabled for this content.