Archives
-
SharePoint, querying lists using DspSts, and consuming this information in Flash
UPDATE: See this blog post on the crossdomain.xml configuration file you have to put on your SharePoint server to allow Flash to access the SharePoint server using web services.
Summary
This blog entry describes how to do complex SharePoint web service calls from a Flash client application to retrieve arbitrary information from SharePoint lists. The very powerful web service DspSts.asmx is used to accomplish this task. This web service allows you to dynamically query any field and item selection from a list and apply ordering on this selection. It is also possible to restrict the number of items to return. The query applied is in the CAML query format.
Introduction
In my search for a possible new implementation of the MacawDiscussionBoard for SharePoint I’m looking into Flash as possible UI for SharePoint discussion lists. In this implementation it is key that all information sent to retrieved from SharePoint must be accomplished using the SharePoint web services interfaces. I’m using Flex 2.0 Alpha release to generate my flash application (Flash 8.5), but the techniques displayed here are for a large part also applicable for Flash 8 (the current release).
Problem I had was that I wanted to get list items not through the simple SharePoint web service interface Lists.asmx that provides a set of simple remote methos calls, but through the web service DspSts.asmx that requires a complete XML document to be passed, and has special additional SOAP headers.
Reason for using this more complex DspSts.asmx web service: the GetListItems(listName, viewName) call in Lists.asmx is too restrictive:
- It can only return fields of list items as defined in a predefined list view
- It returns all list items in the list view
The DspSts.asmx web service is way more powerful:
- Build queries dynamically in the CAML query format
- Query any field from the list
- Query a selection of items from the lis
- Apply ordering on this selection
- Restrict the number of items to return
- Get paged sets of items
Using the DspSts.asmx web service with an XML document based on a complex schema defined in the WSDL and additional SOAP headers was way more complex than I thought it would be. I couldn’t find much information on this and was really struggling until I found the following article on the IBM site: Develop Web services clients with Macromedia Flex. It still took me really a lot of time to solve the ‘puzzle’, that is why I thought I better write this down for others trying to go the same route.
I moved my code into a simple test application where you can specify the URL of a SharePoint site containing a discussion list, and the ID of this list (Use SharePoint Explorer to get the ID). The application retrieves all list items from the discussion list and displays them in a grid. Only a small set of fields is retrieved… the set of fields needed to create a hierachical structure of all discussion list items. See below for an example of this application.
The code below is in MXML (Th Flex markup language), a really powerful language for defining Flash applications. The application executes Query operation of the DspSts web service when the “Execute Query” button is clicked. Variables between ‘{’ and ‘}’ characters are data bindings. The result of the web service call is returned in XML, in the e4x format, the intrinsic XML format of ActionScript 3.
<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.macromedia.com/2005/mxml" xmlns="*">
<mx:Script>
<![CDATA[
import mx.rpc.events.*;
import mx.rpc.soap.*;
import mx.controls.Alert;
import mx.controls.gridclasses.*;
import mx.collections.*;
[Bindable] public var siteURL:String="https://MySharePointPortalServer/personal/serge";
[Bindable] public var listID :String="{020013c8-5efc-49a8-b28f-339d401c1046}";
[Bindable] public var dataGridItems:IList = new ArrayCollection();
]]>
</mx:Script>
<mx:Canvas width="638" height="414">
<mx:Label x="11" y="14" text="SharePoint Site URL:"/>
<mx:Label x="12" y="44" text="List GUID (use \{..\}):"/>
<mx:TextInput x="138" y="12" width="375" id="siteURL_textinput" text="{siteURL}"/>
<mx:TextInput x="138" y="42" width="375" id="siteID_textinput" text="{listID}"/>
<mx:Button x="521" y="43" label="Execute Query" click="wssDspStsService.Query.send()"/>
<mx:DataGrid id="listRowsOutput" height="325" dataProvider="{dataGridItems}">
<mx:layoutConstraints>
<mx:EdgeAnchor left="13" right="13" bottom="13"/>
</mx:layoutConstraints>
<mx:columns>
<mx:DataGridColumn headerText="Ordering" columnName="Ordering" />
<mx:DataGridColumn headerText="ThreadID" columnName="ThreadID"/>
<mx:DataGridColumn headerText="ID" columnName="ID"/>
<mx:DataGridColumn headerText="Title" columnName="Title"/>
<mx:DataGridColumn headerText="Author" columnName="Author"/>
<mx:DataGridColumn headerText="Created" columnName="Created"/>
</mx:columns>
</mx:DataGrid>
</mx:Canvas>
<!-- For more information on format: See the WSS SDK, and search for "DspSts". Select the topic "Query Method".
Online documentation: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/spptsdk/html/soapmqueryRequest_SV01071735.asp
-->
<mx:WebService
id="wssDspStsService"
wsdl="{siteURL}/_vti_bin/DspSts.asmx?wsdl"
service="StsAdapter"
port="StsAdapterSoap"
useProxy="false"
showBusyCursor="true"
fault="Alert.show('Failed to load the DWSL. Error: ' + event.fault.faultstring)" load="wssDspStsService_AddHeaders()">
<mx:operation name="Query" concurrency="single" resultFormat="e4x" fault="wssDspStsService_fault(event)" result="wssDspStsService_result(event)">
<mx:request xmlns="http://schemas.microsoft.com/sharepoint/dsp">
<queryRequest>
<dsQuery select="/list[@id='{listID}']" resultContent="dataOnly" resultRoot="Rows" resultRow="Row" columnMapping="attribute">
<Query QueryType="DSPQ">
<Fields>
<Field Name="Ordering"/>
<Field Name="ThreadID"/>
<Field Name="ID"/>
<Field Name="Title"/>
<Field Name="Author"/>
<Field Name="Created"/>
</Fields>
<OrderBy>
<OrderField Name="Ordering" Type="xsd:string" Direction="DESC"/>
</OrderBy>
</Query>
</dsQuery>
</queryRequest>
</mx:request>
</mx:operation>
</mx:WebService>
<mx:Script>
<![CDATA[
// We need to generate the following SOAP headers:
//
// <soap:Header xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
// <dsp:versions xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp">
// <dsp:version>1.0</dsp:version>
// </dsp:versions>
// <dsp:request xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" service="DspSts" document="content" method="query">
// </dsp:request>
// </soap:Header>
//
// Only issue is that I can't service, document and method as attributes of request, but only as child elements. But it seems to work!
// On the other hand: it already works if only the request header is there, ot does not matter about its attributes.
private function wssDspStsService_AddHeaders()
{
trace("Add HEADERS");
wssDspStsService.Query.addSimpleHeader("versions", "http://schemas.microsoft.com/sharepoint/dsp", "version", "1.0");
var qName: QName = new QName("http://schemas.microsoft.com/sharepoint/dsp", "request");
var requestHeader: SOAPHeader = new SOAPHeader(qName, {service:"DspSts",document:"content",method:"query"});
wssDspStsService.Query.addHeader(requestHeader);
}
private function wssDspStsService_fault(event: FaultEvent)
{
Alert.show("Failed to execute the query. Error: " + event.fault.faultstring);
}
// Returned XML is in followin format
// <queryResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/sharepoint/dsp" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
// <dsQueryResponse status="success">
// <Rows>
// <Row Created="2005-10-23T14:03:20" ThreadID="{20051023-1203-12E1-8E71-1F4426DE71D5}" ID="1" Title="Disc1" Ordering="20051023140320" Author="Serge van den Oever"/>
// <Row Created="2005-10-23T14:03:32" ThreadID="{20051023-1203-2649-92C4-FA7641C9F5F7}" ID="2" Title="Disc2" Ordering="20051023140332" Author="Serge van den Oever"/>
// <Row Created="2005-10-23T14:03:45" ThreadID="{20051023-1203-2649-92C4-FA7641C9F5F7}" ID="3" Title="Disc2" Ordering="2005102314033220051023140345" Author="Serge van den Oever"/>
// </Rows>
// </dsQueryResponse>
// </queryResponse>;
private function wssDspStsService_result(event:ResultEvent)
{
var queryResultXML:XML = wssDspStsService.operations.Query.result[0];
var dsp:Namespace = queryResultXML.namespace();
queryResultXML.setNamespace(dsp);
var queryResultRows:XMLList = queryResultXML..Row;
trace("#rows=" + queryResultRows.length());
dataGridItems.removeAll();
for each(var item:XML in queryResultRows)
{
dataGridItems.addItem({
Created: item.@Created,
ThreadID: item.@ThreadID,
ID: item.@ID,
Ordering: item.@Ordering,
Title: item.@Title,
Author: item.@Author
});
}
}
]]>
</mx:Script>
</mx:Application>
The format of data returned from the DspSts web service
One of the nice and easy things of Flash is that you can have the results of a web service call return as an Object. This object can have fields of any type, hierarchical structures with fields of type object, and arrays of any type (also of type object). Problem is that the converter from the SOAP result to the object does its own interpretation of types, it does not check the schema data that can be returned in a SOAP result, you see an small part of this schema below:
<x:element name="Ordering" minOccurs="0" d:displayName="Ordering" type="x:string" />
<x:element name="ThreadID" minOccurs="0" d:filterSupport="IsNull;IsNotNull;Eq;Neq;" d:displayName="Thread ID" type="x:string" />
<x:element name="ID" minOccurs="0" d:filterSupport="IsNull;IsNotNull;Eq;Neq;Lt;Gt;Leq;Geq;" d:displayName="ID" type="x:int" />
<x:element name="Title" d:filterSupport="IsNull;IsNotNull;Eq;Neq;Lt;Gt;Leq;Geq;Contains;BeginsWith;" d:displayName="Subject" type="x:string" />
<x:element name="Author" minOccurs="0" d:filterSupport="IsNull;IsNotNull;Eq;Neq;Lt;Gt;Leq;Geq;" d:displayName="Posted By">I got into trouble with the Ordering field in a discussion list item. This field contains an ordering value in the format YYYYMMDDHHMMSS, for example 20051020140312. This field is interpreted as a field on type number in the conversion to an object. This is NOT what we want, because a reply on this list item gets an Ordering field with the original Ordering field value, with the timestamp of the reply appended. This string can become really long, does not fit in a number, and as a number is useless for ordering.
I found that it is also possible to return the result as XML, or e4x, the intrinsic XML format of ActionScript 3. The downside is that I have to interpret all returned XML data manually (it is all seen as strings), but the upside is that we don’t need to return the schema data in the result, which can become really large, because all users are returned as a restriction enumertation for the Author field. This means that all users that ever visited the site containing the discussion list (and are therefore registered in the UserInfo table for this site) are returned in a list. No problem for a site only used by a few users, but a huge problem if you have a large user base.
Things I couldn’t accomplish in the WebService implementation of Flash/Flex 2
During my adventures with WebServices in Flash/Flex 2 I couldn’t accomplish the following things:
Complex SOAP headers
I needed to create SOAPheader with the following format:
<dsp:request xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" service="DspSts" document="content" method="query">
What ever I tried, the closest I could get was:
<dsp:request xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp">
<service>DspSts</service>
<document>content</document>
<method>query</method>
</dsp:request>I will never know if this has the same effect, because the service call already worked when I added a header with the name ‘request”. See the code for the creation of the SOAP header.
Access the SOAP result in case of a fault
The SOAP envelope returned from a web service call can contain really detailed information of the exact problem that occured. Flash only gives access to a very general error message through event.fault.faultstring. I could’t find a way to access the actual SOAP envelope returned by the web service call. If anyone has more information on how to accomplish this task, please let me know.
Information on e4x
The information on XML as native datatype in ActionScript 3 is quite sparse. Luckely enough the e4x standard is a well documented standard. For more information have a look at:
- http://en.wikipedia.org/wiki/E4X, good introduction and valuable pointers to more information
- http://developer.mozilla.org/presentations/xtech2005/e4x/, a great overview of e4x
Final words
If I look back at the code above it looks really simple. It is actually really simple, if you know what to do. And that is where the problem lies. There are so many examples on using web services from Flex/Flash, so much information in the documentation, but not about the difficult nitty-gritty parts of using more complex web services. The Flex/Flash implementation proofed to be very powerful and flexible. I’m really impressed. I hope that this blog post will help others in building Rich Internet Application on top of SharePoint using the Flash platform. If you have any questions, let me know!
-
Macromedia Flex 1.5/2: development and pricing model...
As Pau stated in a reaction in this blog entry, current development of applications using Flex is expensive. I have no idea how expensive, I never used the current Flex system, but if I may believe the many stories on the internet it is very expensive (everything is relative in this case however;-)). I heard a price tag of $10.000, but hey, thats quite normal in server software licenses. By the way, there is also a free license for non-commercial/nin-institutional applications and for bloggers who want to showcase on their blog. Have a look at: http://www.macromedia.com/software/flex/productinfo/faq/#section-8
How about the new Flex 2 system? If you read this story by David Wadhwani, VP of Product Development for Macromedia Flex, he writes:
If you're familiar with Flex 1.0, it's very important to recognize that Flex 2 is far more than just a new release. It represents a major milestone in the evolution of the Flex technology and a continued evolution in Macromedia development processes. From a technical standpoint, Flex 2 introduces capabilities that enable developers to build an entirely new class of Rich Internet Applications, ushering in a new generation of RIAs. At the same time, we're opening up Flex development to a much broader group of developers by re-introducing Flex Builder, which has been built from the ground up on the Eclipse open-source IDE framework and now includes the Flex Framework and the compiler. That means that Flex applications can be deployed as a stand-alone option by placing a compiled SWF file on any web server or in conjunction with Flex Enterprise Services 2.
This means that no server component is needed is you need webservice connections only.
There will be multiple product in the Flex product line. One of them is Macromedia Flex Framework 2. This is a framework that build on top of the foundation of Flash 8.5. It is a client side framework. If you only utilize this, the price must be low. How low? Probably in the same range as VS.NET is my guess.
Also with the current Flex system it is possible to build stand-alone Flash applications if I read this blog post correctly.
-
Macromedia, I'm impressed! Flex you way into RIA
Macromedia (or should I say Adobe?) did release an alpha version of their new Flex 2.0 platform and the Flash player 8.5. As you can read in some of my previous blog posts I was searching for the best toolset to build RIA applications. I think I found it! Although I did not have much hands-on time yet I was really impressed. Not the fancy authoring environment that is just suboptimal for designers like me, but a full fledged IDE, based on Eclipse 3.1. Complete with forms designer to design your constraints based interface.
Flash 8.5 features ActionScript 3.0. The next advance in the Javascript based language used in Flash. Some of it’s most powerful features: completely type dafe, also in the runtime. Powerful exception handling, delegates, regular expressions, XML as native type. Finally a powerful, grownup language like I’m already so used to in the .Net development space (c#).
I will dive into this new technology, and keep you posted!
Check it out @ http://labs.macromedia.com
-
WSS SP2 and SharePoint Portal Server using .Net 2.0 and SQL Server 2005?
Bil Simser describes in this post that it is possible to use WSS SP2 with SharePoint Portal Server, although not all new features are supported in Portal Server.
Patrick Tisseghem says in a comment on this post that there will be no SP2 for SPS.
I’m wondering how you can get SharePoint Portal Server freshly installed on ASP.NET 2.0 and SQL Server 2005.
I think it will be something like:
- Install Windows Server 2003
- Install SQL Server 2000 + SP4
- Install SharePoint Portal Server
- Install WSS SP1
- Install SPS SP1
- Install the .Net Framework 2.0
- Install WSS SP2
- Upgrade SQL Server 2000 to SQL server 2005
- Uninstall SQL Server 2000
Or is there a faster route?
-
Discussion board and RIA technologies
In a previous blog post I was writing about the possibilities for the implementation of a new version of the Macaw Discussion Board. I mentioned a few technologies that I was looking at. The reason I am looking into those technologies is not only for the implementation of the discussion board, but also to have a look at what the best way would be to build the next generation Rich Internet Applications. Besides good old DHTML with Ajax technologies, Flash as a runtime platform looks like one of the possible ways to go for the near future.
A great overview of possibilities for developing for the Flash platform is given by Aral in this post. He also discusses NeoSwiff, a tool that allows you to write Flash Applications in C# .NET using Visual Studio 2003, and Xamlon Pro Flash Edition, which lets you use XAML to create your UI declaratively. As a .Net developer those last two alternative are especially interesting, because for me it will be .Net/C# that is used at the server side.
Darron also has a great post where he also discusses the problem with developer tools for creating RIA running on Flash. He also mentions NeoSwiff.
I dived in both Xamlon and NeoSwiff. The adventure with Xamlon was over soon. It integrates with both VS.NET 2005 and VS.NET 2003. I tried installation on multiple machines and I keep getting a warning that my license key expired. Even the forums of Xamlon couldn’t provide me with a solution. I was also discouraged with this news post stating “… Xamlon is backing away from XAML, saying developers aren't really ready.”
I moved over to NeoSwiff, and I must say I’m impressed. One of my most important requirements is the possibility to easily create an application that resizes it’s controls when the window is resized. All I got during my Flash adventures was a zooming application. NeoSwiff is really great in this respect. See this post by Darran for a great example!
However, in the first two minutes of starting with NeoSwiff I encountered the three most important problems for me:
- No direct support for web services
- No debugger
- No UI designer
For the first and third point Jesus describes a solution. See his post. Completely happy with this results I continued with NeoSwiff. Everything went great. I designed a small UI in a normal C# .Net forms project, and copied over the code. Next part was the web service. I don’t want to do an HTTP GET call (function and parameters on the url) like Jesus does in his post. I wanted to call using either a soap message (using a HTTP POST) or through proxy code generated based on the WSDL of the service. NeoSwiff does not provide a tool to generate the proxy code, and the code produced by Microsoft’s wsdl.exe can’t be used. I had to go the manual way. But to no avail. NeoSwiff has two ways to do a request: XmlRequest (as used by Jesus) and WebRequest. With the XmlRequest it is not possible to specify the body of the request. You can only specify an url. The WebRequest provides a way to specify the body of the request, but only through a StringDictionary, resulting in name-value pairs as used in a post back from a HTML form. I also couldn’t change the content type, it remained application/x-www-form-urlencoded, the content type used on form post backs. I have been trying for hours and hours but I didn’t get any further. Maybe I missed something, maybe it is something they are still working on. Ig someone knows: please tell me!!
Of course I could work around the limitations by writing a set of web services I can always call through url syntax, but I want to reuse all available SharePoint web services. Thats my whole point: I want to build RIA’s againt web services.
I think the NeoSwiff platform can become a great platform if they succeed in getting access to web services working. It is so nice to work in VS.NET using code completion and all the power available in this IDE!
-
SharePoint discussion board upgrade: which way to go?
The Macaw Discussion Board, a reworked version of the SharePoint discussion board, works quite well, but it is not really the way I would like to have a SharePoint discussion board to be implemented. I don’t like the technical implementation. It is built on top of the existing discussion board functionality. I converted all list views to data views and did extensive modifications to the XSLT code in the data views. Due to the restrictions of the XSLT in a data view web part there are absolutely no possibilities for code reuse or centralized localization. See the conclusion of my MSDN for more information on those restrictions. Although the usability of the discussion board is good, it is not the way it could look in the new era of Rich Internet Applications. I want it to be slick, smooth, single-page UI, maintainable, configurable, localizable…a rewrite is dawning.
And now my biggest issue! Technology! Which technology should I use. Should it be Atlas, Ajax.Net, Backbase, Flash 8, OpenLaszlo, or one of the other RIA technologies that are available?
One of the most important things is the availability of good controls (or widgets as some others call them). I don’t want to be stuck with the standard HTML controls. Those are so ‘90s. Modern controls are needed like at least a hood tree control and resize controls.
Flash and OpenLaszlo
I’m not really a Flash developer. I really love their approach however: a complete, small and accepted platform running in any browser that provides a really rich user experience. I did some adventures with Flash MX2004 pro in the past. I wasn’t impressed by the quality and stability of the controls in their component framework 2.0. I don’t seem to be the only one as you can read at the ActionStep home page, and as you can see at http://www.osflash.org where everyone seems to restart on a new control library to replace the Flash component framework. I have no idea yet if Flash 8 made any progression in this matter, I will dive into this soon. Another problem I have with the Flash development environment is that I’m a developer, and still can’t really get used to the Flash way of development. Maybe that is the reason I like the approach taken by OpenLaszlo. OpenLaszlo applications are written in XML and Javascript. The OpenLaszlo server then compiles the XML and Javascript into .swf files, which run in the Flash player. I’m currently downloading the system and will soon have a look at it. It looks really promising!
Backbase
Another interesting technology is Backbase. It is a javascript solution for Rich Internet Applications that runs on most browsers. They have a powerful set of controls available and it all looks really powerful and smooth. There is full support for single page web based user interfaces. Extra data can be retrieved on the fly, it uses ajax technologies. The biggest advantage is that it does not depend on any third party plug-ins like Flash. one of my biggest problems with is is that although there is a community version available for non-commercial applications, an implementation of the new version of the discussion board running within an organization will require a commercial license. This will make it more difficult to get people using the new version of the discussion board.
Ajax.Net
In the .Net world everyone is talking about Ajax.Net and Atlas. Ajax.Net is a powerful ajax library written by Michael Schwarz that integrates ASP.NET development with client side ajax techniques. It is available today, it works with the .Net 1.1 framework. I must say that I don’t have any experience with the library yet, but as far as I can see right now is that the library focuses on the client-server communication, not on providing a full-fledged widget set. Development in that direction is on it’s way in the community.
Atlas
Atlas is the code name for the ajax library Microsoft is working on. We are a Microsoft house, so although other systems might be better or may be further, we will finally end up doing Atlas development. For now however, Atlas does not seem to be the best approach for SharePoint development. With Service pack 2 for WSS there is the possibility to do ASP.NET 2.0 development within SharePoint context. Jan Thielens is working on hosting ASP.NET 2.0 web parts into the current version of SharePoint through “Son of SmartPart”. Problem will be that not many companies will move over to the .NET 2.0 framework on their SharePoint servers yet. The demands for the extra functionality will be too high. Another issue is that no information came out yet on the support of the .Net 2.0 framework for SharePoint Portal Server. I didn’t see a service pack for SPS yet.
Conclusion
Do I have a conclusion? Not yet actually! There are many possibilities to investigate and I hope to get feedback from you on what you think what would be the best direction to take. There are probably many things I overlooked. My guess on best approach in order of best bet are currently as follows:
1. OpenLaszlo (Flash as runtime)
2. Flash 8 (Flash as runtime)
3. Backbase (Standard browser support)
4. Ajax.Net (Standard browser support, use of ASP.NET 1.1 at server side)
5. Atlas (Standard browser support, use of ASP.NET 2.0 at server side) -
New MSDN article on SharePoint: SharePoint Data View Web Part Extension Functions in the ddwrt Namespace
One of my blog entries turned into an MSDN article after a lot of reviews and editing!
Have a look at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/odc_SP2003_ta/html/OfficeSharePointDDWRT.asp?frame=true!
Thanks Susan Harney for all your great work and guidance on getting this out to MSDN!