Writing custom LINQ provider

In this post, I will show  how to create a sample LINQ provider.Creating LINQ provider can be far more complex, but here I will demonstrate the very basic one to start.

This sample is done by using VS2008 Beta 2. I finished of this is post by creating a fictious Person query provider that in turn the gives a glimpse of creating custom LINQ provider for pulling data from source, in a more descriptive and convenient way, rather calling out boring GetBy methods.

So,my fist task , is to declare an entity class.

 

For simplicity, It contains only three properties ID, Name and Age. The properties are created using Automatic property feature of .net 3.0, That means , i don't need to declare anymore private variable and put them under get / set block. It's now more like declaring property in an interface, the only different is that framework does the rest of the work for me.

Now, To make , custom LINQ provider , a class must implement two interface. IQueryable and IQueryProvider.

IQueryable is the interface that make objects queryable in a LINQ expression. Without it , it is not possible to use objects in queries like this

 

IQueryProvider is the interface that is pointed to by IQueryable , which is also responsible for creating and executing expressions. IQueryable is also a member of IEnumerable, which enables the query result to be used inside a loop.

Now, in the sample Provider, i have created a PersonConext class that implements the IQueryable <T> (in this case IQueryable <Person>) and IQueryProvider . IQueryable has some IEnumarable implements but the most important implementation is the providver creation and type initialization.

 

As, i have implemented the provider in the same class that implements IQueryable interface, therefore , PersonContext's provider points to itself(this). But we can move the provider to a different class, in that case the framework need to keep instances of two class, one for IQueryable class and another for IQueryProvider class.So, i guess the solution, is to have them in same class.Next, ElmentType is the item , we are dealing with (in this case Person) and Expression.Constant(this), tells the framework that the expression which is used are with constant values.

IQueryProvider, only have two sets of method , Create<S> that returns IQueryable pointer  (this is the entry point for expression intialization) and Execute<TResult> that basically uses the Expression instance  to query and extract result out of LINQ query.

IQueryProvider.Execute method is called during the iteration of IEnumerables (query), therefore, provider calls an external source or db to get its result during iterateration process only.

Basically, when we do a LINQ query , internally it does methods calls, so inside Execute method, the expression type will be of MethodCallExpression. and for each arguments  of MethodcallExpression(which is also a type of Expression), it needs to be evaluated , untill the leaf is reached.

Now, Each MethodCallExpression, generally starts with an UnaryExpression and ends with an BinaryExpression (If the query has a where clause ) or ParameterExpression (If the query is simple select <something>). In between, there is LamdaExpression ( ex . { p = > p.age > 20 } ), to get the next expression we need to call LamdaExpression.Body in this case, where LamdaExpression.Body in most of the case is BinaryExpression.

BinaryExpression can be of various type ,like, ExpressionType.LessThan , ExpressionType.Equal, etc.

Inside ProcessExpression of the sample provider.This shows an idea about how to parse expressions at different states.

Finally, the right part of the  BinaryExpression is of ConstantExpression Type (in this case), but it can be of Unary or Parameter Expression, in more complex situations. In this case, we just need to extract the constant value (age/name/id)  to filter Person.

Though, in the end the query graph could much more complex , which is beyond the scope of this post,  still the basic structure remains the same. Finally, I have added the sample provider code to download. Feel free to download, hope this will give some pointers to custom providers.

To build custom providers with pleasure use LinqExetender (www.codeplex.com/linqextender). The only toolkit that enables you to do it in style.

kick it on DotNetKicks.com

6 Comments

  • You have to implement IQueryable for custom providers, as IQueryable has Provider Property , it points to the provider that will process your query. In other words, when IQueryable&nbsp;is implemented, framwork will automatically go to Provider and call its CreateQuery with the represented Expression&nbsp;from&nbsp;user query, duing intialization,&nbsp;where you can&nbsp;process&nbsp;data&nbsp;from expression tree.&nbsp;Similarly, framwork will&nbsp;call its Execute routine , when we will try to run the&nbsp; query in a loop or call properites like Single, First , last , Etc...
    But if you have only a array or simple data structure&nbsp;or you want to&nbsp;return all the data then process it in memory regardless of expression state, then&nbsp;you can go by Implementing IEnumerable&lt;T&gt; , where the query will be&nbsp;processed interenally, also in this case&nbsp;the call will directly go to GetEnumerator block as there is no IQueryProvider to point to, defined by IQueryable.
    Hope this helps,

  • Hi Mehfuz,

    Is there any way to generate expression for LINQ on objects??, my app is full of objects which i would process it thru reflection, the LINQ to SQL works like a charm for me, but i would also like to add this feature in my app with LINQ on objects. Have a custom provider and use that in my complex objects.

    Let me know your thoughts.

    -Fahad

  • Bascically, a collection of objects that has IEnumerable implemeted is already LINQ-a-ble by .net 3.5. Here to add that LINQToSql is iteself a provider, but LINQ also gives you the power to make providers on your own (ex. Linq To codeplex) and that is the point where "creating custom provider" comes into play.

  • Yup, i know it is LINQ-a-ble, my question is i would like to generate dynamic expressions.
    Say i have some rules applied for a filter condition and i don't know which object i will be applying it. My custom provider will generate an expression according to some custom rule i have on this object, and this would return me the resultant data after i LINQ-xecute it :).
    I would luv to hear from you on this.
    -Fahad

  • Not sure, why you need it, i am curious to know what is your objective that can not be accomplished with existing solution.

    Also, to let you know, when you type a LINQ query in your IDE , it generates a Expression tree, which you can trap in your custom provider and make decision accoridingly.May be that can help you achive your objective. Please check out this and other posts of mine to find out more.

  • Great example. Saved me many hours of research. Thanks.

Comments have been disabled for this content.