Testing wCF Services with SO-Aware: the Why, the What and the What the F@#$%!!@#

Service testing is one of the fundamental components of SO-Aware (Tellago Studios’ WCF Registry). In its current version, SO-Aware enables modeling tests against WCF services as well as scheduling their periodic execution. With so many Web Service testing tools in the market, some of you may be wondering why we decided to create a new one and, more importantly, how is SO-Aware different from other offers in the market.

As part of our work on various large WCF implementations, we use a variety of web service test tools and we seem to always run into the same set of issues. With SO-Aware, we have attempted to address some of those challenges using a very simple and yet highly scalable model. If you are planning on establishing a testing strategy for your WCF services, you might be considering the following:

· I want to know which services I am testing

· I want to test services that use WS-* based capabilities such as security, trust, etc

· I want to integrate my Web Service tests into other test frameworks and tools

· I want to be able to schedule and track the periodic execution of Web Service tests

· I want my architects, developers and QA staff to collaborate in the design and execution of the Web Service tests.

· I want to script highly customized tests that simulate specific business logic

The rest of this blog post will examine in detail each one of the previous challenges and explain how they can be addressed using SO-Aware.

Knowing Which Services to Test

When you are using some of the Web Services testing tools in the marketplace, it is your responsibility to figure out the details required to invoke the target service. In that sense, you need to understand where the service is located, understand its contract, figure out the message exchange patterns and the message payload for its different operations, etc. While this process is sufficient for a single service, it couldn't be more inefficient for enterprise solutions that involve hundreds of Web Services.

Following the previous argument, it is very easy to envision how a Web Service testing tool can benefit from a service repository that contains the details of the different Web Services in the enterprise as well as the details required to invoke them. That way, the tests are created, cataloged and stored together with the services in the repository and can be used to determine the correct functioning of the Web Service.

The SO-Aware™ Solution

In SO-Aware, tests are directly associated with service versions stored in the service catalog. Similarly, WCF bindings, behaviors and other artifacts required to test the service operation are kept in SO-Aware and can be reused across multiple tests. This centralized model provides the tester with a complete picture of the characteristics of the target service as well as their dependencies.

Testing Beyond basicHttpBinding

WCF is arguably the most sophisticated Web Services technology in the marketplace. Consequently, it is very difficult, if not impossible, for Web Service testing tools to interact with WCF services that implement features or WS-* protocols that go beyond simple SOAP over HTTP. For instance, it is almost impossible for the majority of Web Services test tools in the current market to test WCF services that use WS-Trust as a brokered security protocol.

The SO-Aware™ Solution

Given that SO-Aware has been entirely built on WCF, it includes the necessary infrastructure to test highly sophisticated services. Obviously, SO-Aware excels in the testing of WCF services but it is equally effective testing services developed on any Web Services technology stacks. Its WCF foundation makes SO-Aware better equipped to interoperate complex Web Services than any other testing technology in the market.

Integrating with Other Test Frameworks

Web Service testing is just a small component of the testing strategy in the enterprise. Ideally, organizations would like to integrate this type of testing into more complex test cases developed using existing frameworks such as MSTest, NUnit, JUNit, etc. However, this simple need is almost impossible to achieve with existing Web Services testing tools without undergoing a ridiculous development effort. Additionally, this integration looks different depending of the testing framework we are using.

The SO-Aware™ Solution

SO-Aware is a service registry completely built based on the principles of REST and OData. In a nutshell, every single element of SO-Aware can be accessed or modified using simple HTTP and OData. This principles also applies to the testing repository in which every test can be executed using a simple HTTP GET as illustrated in the following figure.

The RESTful nature of SO-Aware tests makes it completely seamless to integrate into other testing frameworks. For instance, the integration between SO-Aware and Team Foundation Server(TFS) can be easily accomplished by configuring TFS to execute the correct HTTP request to query or execute tests.

Record Test Results

Almost without exception, the majority of Web Services test tools in the current market can do a decent job executing isolated tests against a service but they offer next to zero capabilities in terms of tracking and analyzing the test results. This is partially due to the fact that these tools focus almost exclusively on on-demand tests in which a user is starting the test and waiting for the results. The need to schedule the execution of tests on a regular basis is a key requirement of effective testing in a SOA environment.

The SO-Aware™ Solution

In addition to the traditional "on-demand" testing, SO-Aware offers the capability of scheduling tests so that they can be executed periodically. Additionally, SO-Aware tracks the results of the executed tests and offers a series of analytics based on it as illustrated in the following figure.

Testing and Monitoring

Web Services testing and monitoring are two capabilities that can be combined to enable complete visibility into the runtime behavior or a service. Unfortunately, the current market is lacking of tools that combine both capabilities with the service metadata enabled by a service registry. If you want to enable both testing and monitoring in your SOA, you are most likely looking at a decent development undertaking to cohesively integrate the web service monitoring and testing suites you are using.

The SO-Aware™ Solution

SO-Aware combines the service registry, monitoring and testing capabilities into a single product which enables the effective monitoring of the tests scheduled against specific Web Services. Using this model, the operations team can not only monitor the execution of the tests but also determine how those tests are impacting the performance of the Web Service.

Collaborative Testing

Quite often, Web Services testing is seen as a task of the QA or the operations staff. However, this process excludes developers or architects who can provide a lot of insight into the correct functioning of the service.

The SO-Aware™ Solution

By using a centralized model, SO-Aware enables a collaborative testing approach in which developers, architects and the QA staff can work together in the design, modeling and execution of the tests. Using the SO-Aware portal, developers can design the initial version of tests which can be later modified and scheduled for execution by other members of project team. By leveraging the test tracking capabilities, the team can get insight into the execution of the tests we well as detect possible defects on the SOA infrastructure.

The Power of Scripting Tests

When comes to testing, it is very presumptuous to assume that one tool or framework can fit all your test scenarios. More often than not, big SOA implementations require very customized testing procedures that leverage internal tools or processes. Scripting languages are an ideal solution to build customize tests that orchestrate different systems or tests designed with various test tools. Unfortunately, most, if not all, Web Services test tools in the current market do not enable seamless integration with scripting languages that will facilitate the development of more specialized tests.

The SO-Aware™ Solution

SO-Aware includes two fundamental artifacts that enable the development and execution of highly complex and customized tests: A PowerShell provider and a RESTful API. In its current version, SO-Aware includes a complete provider for Windows PowerShell that enables almost the complete set of functionalities available with the portal interface from a scripting environment. Using this mechanism, testers can script highly sophisticated tests entirely from the PowerShell environment.

In addition to the PowerShell provider, SO-Aware also includes a highly interoperable RESTful API based on the OData protocol. The capabilities of the testing subsystem are completely available through this API which enables its interaction with other scripting languages such as Python or Ruby by using simple HTTP commands.

Where Are We?

We believe SO-Aware presents a unique value proposition to Web Service testing by enabling a WCF-based infrastructure that facilitates the testing of arbitrarily complex Web Services which are impossible to test using other products in the market. SO-Aware's infrastructure combines a highly scalable and interoperable RESTful API, a unique model to expose tests as RESTful resources, a sophisticated model to schedule the execution of tests and a robust tracking system to monitor the correct functioning of the tests.

For more information check our this video on our website and go ahead and download the Express version of SO-Aware . Don't forget to send us some feedback.