During the last few months, I’ve been very fortunate to have the opportunity of collaborating closely with the StreamInsight product team. One of the things that impressed the most about that team (in addition to their talent) is their mindset and commitment to produce continuous releases of the technology. You don’t have to be an expert in Microsoft culture to realize that that level of periodical releases is not common among new Microsoft technologies.

This week, just after four months of the first release, Microsoft announced the general availability of StreamInsight 1.1. This version can run side by side with StreamInsight 1.0 and provide several enhancements that I am really excited about:

Observable and Enumerable event sources

In this new release, StreamInsight supports any IObservable or IEnumerable object as event sources or event sinks. Why is this important? Let me give you a few reasons.

Mainstream sources

IEnumerable and IObservable are two of the most predominant mechanisms for representing collections in the .NET framework. This means that now we can use StreamInsight queries against a broader range of event source and, what is more important, against event sources that can be created by non-StreamInsight developers.

Query Testing

Using IEnumerable and/or IObservable event sources drastically improves the ability of testing StreamInsight queries without the need of using specific adapters. This is incredibly important when implementing complex queries which are fairly common in real world CEP scenarios.

Simulation

Simulation is a very common scenario in CEP applications. An important aspect of a correct event simulation is to structure the event sources following the specific patterns we are trying to emulate. The use of IEnumerable or IObservable event sources drastically facilitates this capability on StreamInsight applications.

Let’s take a look at an example.

Suppose that we have a database that stores series of records representing the temperature values produced by different sensors. The following ADO.NET entity framework model abstracts the interaction with our sample database.

The following code uses StreamInsight 1.1 to convert the events recorded in a data into a point event stream.

   1: Server server= Server.Create("My StreamInsight Server");

   2: var cepApplication= server.CreateApplication("SampleApplication");

   3: DemosEntities source= new DemosEntities();

   4: var enumQuery = from s in source.SensorValues

   5:                 select s;

After that we can execute StreamInsight queries that use traditional techniques such as windowing. The following sample query aggregates our events using a tumbling window pattern.

   1: var tempStream= enumQuery.ToPointStream(cepApplication, 

   2:    e => PointEvent.CreateInsert(e.CurrentTime, 

   3:    new {e.Temperature, e.SensorID} ), 

   4:    AdvanceTimeSettings.StrictlyIncreasingStartTime);

   5:  

   6: var streamQuery = from s in tempStream

   7:                   group s by s.SensorID into g

   8:                   from window in g.TumblingWindow(TimeSpan.FromSeconds(2), 

   9:                                  HoppingWindowOutputPolicy.ClipToWindowEnd)

  10:                   select new {Sensor= g.Key,  

  11:                      EventCount = window.Count(), 

  12:                      TempAvg = window.Avg(e => e.Temperature) } into agg

  13:                   select agg;

Finally, we can convert our query to an enumerable or observable sequence.

   1: foreach(var s in streamQuery.ToEnumerable())

   2:          {

   3:              Console.WriteLine("Sensor= {0}, Number of Events= {1}, 

   4:                  Average Temperature= {2}", 

   5:                  s.Sensor, s.EventCount, s.TempAvg);

   6:          }

Other improvements

In addition to the aforementioned capabilities, StreamInsight 1.1 also includes some important performance improvements in areas such as the latency of hoping windows or the throughput of join operations on edge streams.

Kudos to the StreamInsight team for this new release! Keep it up guys!