Speeding up pages with lots of media content items

For a few versions now, Orchard has been treating media (images, videos, etc.) as content items. There is a special kind of field, MediaLibraryPickerField, that enables you to add images, or collections of images, to any content type. For example, on the Nwazet web site, I’ve added a “Image Gallery” field to the product content type:

The field is configured to allow for multiple images:

Managing the images from the product editor is extremely easy, including the order of the images, which can be changed by dragging and dropping them:

The first image has a special role in this site, in that it’s the default image that is used on lists of products, such as the one on the home page:

This list, as well as many others, is built using a projection, and I have placement and a few template overrides to display things properly. The problem is that this creates a massive select N+1 problem: when displaying the results of the projection, the template override for the media field accesses the MediaParts property in order to get the image’s data (its path, dimensions, etc.), and this triggers a database query to get the media content item corresponding to the product. This gets repeated for each product, so you get as many database queries as you are displaying products.

Projections are not extremely good at dealing with relationships without writing quite a bit of code, so if we’re after a simple solution, we’ll have to get creative.

The solution I found is to figure out ahead of time, from the list’s scope, what the list of ids for all the media content items is, and then make one big query for all those guys. I can then stick that collection of media items into a cache, and look that up later when I display each product. That should get us down to 2 queries instead of the number of products plus one.

I could (and probably should) have done that from a shape table provider, but in order to keep all this in the theme, I took some liberties with The Rules. Instead, I overrode the Content-ProjectionPage.cshtml template that renders all projections. You could use a more specific alternate, but remaining generic like this actually can apply the benefits of the media cache I’m about to build to any projection in the site.

// Pre-fetch images
var projectionItems =  ((IEnumerable<dynamic>)
    ((IEnumerable<dynamic>)Model.Content.Items)
    .First(i => i.Metadata.Type == "List").Items)
    .Select(s => (ContentItem)s.ContentItem);
var mediaLibraryFields = projectionItems
    .SelectMany(i => i.Parts.SelectMany(
        p => p.Fields.Where(f => f is MediaLibraryPickerField)))
    .Cast<MediaLibraryPickerField>();
var firstMediaIds = mediaLibraryFields
    .Select(f => f.Ids.FirstOrDefault())
    .Where(id => id != default(int))
    .Distinct()
    .ToArray();
var firstMedia = WorkContext.Resolve<IContentManager>()
    .GetMany<MediaPart>(firstMediaIds, VersionOptions.Published, QueryHints.Empty);
var mediaCache = Layout.MediaCache == null 
    ? Layout.MediaCache = new Dictionary<int, MediaPart>() 
    : (Dictionary<int, MediaPart>) Layout.MediaCache;
foreach (var media in firstMedia) {
    mediaCache.Add(media.Id, media);
}

The code first drills into the the current shape to find the Content zone (this may vary in your case), on which it enumerates child shapes for the List shape, which is the one we’re after. Note that if you’re using a different projection layout, that shape may not be List, but Raw or something else. Then it gets the Items under that list, which are a list of shapes again, but those have a reference to the content item being displayed. So at the end of that first statement, we have a list of all the content items being displayed by this page of the projection results.

The second statement extracts the list of media library picker fields that these content items may have.

Then we transform that into a list of distinct first media content item ids. Note that no database query has been necessary for any of this so far, because fields are stored as XML on content parts that we already have in memory after the projection has executed.

Once we have the list of all first media content item ids, we can just do a GetMany on ContentManager in order to fetch the actual content items. This is when the actual querying happens.

The results are then added to a dictionary from id to part that is added as a property of Layout, which caches it for the duration of the request, and makes it easily available to any shape on the page.

Case in point, from my MediaLibraryPicker.Stamp.cshtml template override for the field when using the “Stamp” display type, I can retrieve the media from that cache:

@{
    var field = (MediaLibraryPickerField)Model.ContentField;
    var imageIds = field.Ids;
    if (imageIds.Any()) {
        var cm = Model.ContentPart.ContentItem.ContentManager as IContentManager;
        var title = cm == null || Model.ContentPart == null
            ? "" : cm.GetItemMetadata(Model.ContentPart).DisplayText;

        var mediaCache = Layout.MediaCache as Dictionary<int, MediaPart>;
        var firstImage = mediaCache != null
            ? mediaCache[imageIds.First()]
            : cm.Get(imageIds.First()).As<MediaPart>();
        <div class="gallery">
            <a href="@Url.ItemDisplayUrl((IContent) Model.ContentPart)">
                <img src="@Display.ResizeMediaUrl(
                    Path: firstImage.MediaUrl, 
                    Width: 130, Height: 87,
                    Mode: "crop")" class="main" alt="@title"/>
            </a>
        </div>
    }
}

To be on the safe side, the code can fallback on querying the content manager if the cache isn’t found, but this should never happen.

And there we are, the projection can now display all those images with only 1 query for all the media, which speeds things up quite a bit.