A Linq2Kdb+ Query provider
Over the winter break I wanted to learn more about a few technologies:
- T4 - the built into Visual Studio code generation tool
- The Kdb+ columnar database
- Expression trees
The result ended up in a Linq to Kdb+ query provider. The following snippet translates to
using (IConnection connection = new Connection("localhost", 5001)) { var tkc = new TestKdbContext(connection,Console.Out); connection.Run(@"\l sp.q"); IEnumerable<string> names = from row in tkc.sRecord where row.status == 20 && row.city == GetCity() select row.name; }
which ends up being transformed to something like this Q function
{?[s;enlist (&;(=;`status;20);(=;`city;enlist x));();(enlist `name)!enlist `name]}
My thoughts -
- T4, combined with the T4 toolbox proved to be a fine alternative to CodeSmith. With it I was able to generate some strongly typed classes which mapped to a Kdb tables, and also create a strongly typed KB query context.
- Kdb+/Q is certainly an impressive technology. It combines a columnar database server and complete functional and query language in a 156kb executable. That said, getting proficient in reading Q code seems like it'll take five times as long as it will to learn to write it.
- Expression trees are definitely fun to play with, there's tons of opportunity in compiled lambdas for custom filtering and fine-grained entitlement checks. Creating the query provider was made possible (in the time I had) with the IQToolkit and this guide.
Update: Source code is now available