Kenny Kerr

Earlier this month I heard about Microsoft’s Softwear line of t-shirts. It was supposedly launching on the 15th of December. Launching where?

I have enjoyed the ease of use and surprising power of the built-in Windows Movie Maker product but it really hasn’t progressed with the times and the newest “Windows Live” version of Movie Maker seems to be going backwards.

What is the initial value displayed for a particular topic? Before you come up with an answer take another look at the ConnectData method used by Excel to subscribe to a particular topic.

UpdateNotify is the primary method provided by the IRTDUpdateEvent callback interface that an RTD server uses to let Excel know that updates are available. I have however noticed that this method tends to be used incorrectly by many developers (specifically those developers without a strong foundation in COM). I’ve also received a few emails from readers reinforcing my belief that its proper usage is not obvious to many developers.

In the description of a minimal C# implementation I suggested you simply reference the “Microsoft Excel Object Library” to get the definition of IRtdServer. Of course this introduces an unnecessary reference and can be problematic. This assembly ends up being tied to a particular version of Excel even though the COM interfaces for the RTD server remains the same. As we’ve seen in the C++ implementation, there’s no reason to have a hard reference to any Excel binaries and we can define all of the interfaces ourselves.

The last few entries in this series got a bit long but I didn’t feel like spinning off a mini-series just to focus on COM fundamentals. I skimmed over a lot but hopefully it was enough to give you an idea of what’s involved. Hopefully from now on there should be enough groundwork in place to keep things short.

Having got a minimal single-topic RTD server working in C# and C++ it makes sense to take a look at what it takes to support multiple topics. Fortunately it’s quite straightforward. For starters we’ll update the minimal C# implementation to support multiple topics. But first let’s recap how topics are fed into the RTD server.

The Excel RTD function is prototyped as follows:

=RTD(progId, server, topic1, [topic2], ...)

Unfortunately this is a bit misleading. The topics here really should be called parameters. So an RTD function starts with a ProgId, an optional server name, and one or more parameters. Each unique list of parameters forms an individual topic for which Excel will call the RTD server’s ConnectData method.

Imagine you’ve populated four cells in Excel with the following functions:

A1 =RTD("ProgId", ,  "one")
A2 =RTD("ProgId", ,  "one")
A3 =RTD("ProgId", ,  "one", "two")
A4 =RTD("ProgId", ,  "two", "one")

Excel will call the RTD server’s ConnectData method once for A1 and A2 as they have the same list of parameters. It will then call ConnectData for both A3 and A4 as they have a different list of parameters. As far as the RTD server is concerned Excel is interested in three unique topics with each topic identified by a unique number provided by Excel. That number is how the RTD server communicates values back for individual topics when Excel calls the RTD server’s RefreshData method.

With that let’s update the C# RTD server to support multiple topics. The initial version just printed out the time as follows:

DateTime.Now.ToString("hh:mm:ss:ff")

Instead of hard coding the format string lets take the first parameter of the topic and use that as the format string so that the user can display the time in whatever format he or she desires.

Start by replacing the m_topicId member variable with a dictionary of topic Ids and format strings:

private Dictionary<int, string> m_topics;

You should create the dictionary in the ServerStart method and then add topics to it in the ConnectData method:

string format = strings.GetValue(0).ToString();
m_topics[topicId] = format;

The DisconnectData method also needs a bit of work. Previously we simply stopped the timer. Now that wouldn’t make sense as there may still be other topics that are in use. Instead we need to simply remove the specific topic from the dictionary.

Finally, the RefreshData method now has to enumerate the topics in the dictionary and build the multi-dimensional array. Remember that the first dimension is for the list of topic Ids and the second is for the values corresponding to those topics.

You can test it in Excel using a variety of RTD functions with different format strings. Here’s an example:

=RTD("Kerr.Sample.RtdServer", , "hh:mm:ss")

Below is the complete source code for the updated RTD server:

[
    Guid("B6AF4673-200B-413c-8536-1F778AC14DE1"),
    ProgId("Kerr.Sample.RtdServer"),
    ComVisible(true)
]
public class RtdServer : IRtdServer
{
    private IRTDUpdateEvent m_callback;
    private Timer m_timer;
    private Dictionary<int, string> m_topics;

    public int ServerStart(IRTDUpdateEvent callback)
    {
        m_callback = callback;

        m_timer = new Timer();
        m_timer.Tick += new EventHandler(TimerEventHandler);
        m_timer.Interval = 2000;

        m_topics = new Dictionary<int, string>();

        return 1;
    }

    public void ServerTerminate()
    {
        if (null != m_timer)
        {
            m_timer.Dispose();
            m_timer = null;
        }
    }

    public object ConnectData(int topicId,
                              ref Array strings,
                              ref bool newValues)
    {
        if (1 != strings.Length)
        {
            return "Exactly one parameter is required (e.g. 'hh:mm:ss').";
        }

        string format = strings.GetValue(0).ToString();

        m_topics[topicId] = format;
        m_timer.Start();
        return GetTime(format);
    }

    public void DisconnectData(int topicId)
    {
        m_topics.Remove(topicId);
    }

    public Array RefreshData(ref int topicCount)
    {
        object[,] data = new object[2, m_topics.Count];

        int index = 0;

        foreach (int topicId in m_topics.Keys)
        {
            data[0, index] = topicId;
            data[1, index] = GetTime(m_topics[topicId]);

            ++index;
        }

        topicCount = m_topics.Count;

        m_timer.Start();
        return data;
    }

    public int Heartbeat()
    {
        return 1;
    }

    private void TimerEventHandler(object sender,
                                   EventArgs args)
    {
        m_timer.Stop();
        m_callback.UpdateNotify();
    }

    private static string GetTime(string format)
    {
        return DateTime.Now.ToString(format, CultureInfo.CurrentCulture);
    }
}

Continuing on from part 1, here I’m concluding the walkthrough of a minimal RTD server written in C++.

It’s a COM Class

What we haven’t done yet is actually define the COM class to implement the RTD interface. Start by defining the class as follows:

class DECLSPEC_NOVTABLE DECLSPEC_UUID("B9DCFAAD-4F86-44d4-B404-9E530397D30A") RtdServer :
    public CComObjectRootEx<CComSingleThreadModel>,
    public CComCoClass<RtdServer, &__uuidof(RtdServer)>,
    public IDispatchImpl<IRtdServer>
{

It’s not actually as daunting as it looks.

DECLSPEC_NOVTABLE is an optional hint to the compiler indicating that we don’t need the vtable, which can result in a reduction in code size.

DECLSPEC_UUID associates a GUID with the class so that you can later reference it using the __uuidof keyword. This avoids having to define the GUID elsewhere in a CPP file for example.

CComObjectRootEx provides the code for implementing the reference counting portion of the IUnknown interface that IDispatch derives from.

CComCoClass provides the code for creating instances of the COM class.

IDispatchImpl provides the implementation of IDispatch. I’ll talk more about this implementation in the upcoming section on Automation.

Next up is the implementation of IUnknown’s QueryInterface method. For this ATL provides a family of macros:

BEGIN_COM_MAP(RtdServer)
    COM_INTERFACE_ENTRY(IRtdServer)
    COM_INTERFACE_ENTRY(IDispatch)
END_COM_MAP()

This actually just builds a data structure and a set of supporting functions. The actual IUnknown methods are provided by another class such as CComObject used to create instances of a particular COM class.

We also need to identify a resource that includes the registration script for the class:

DECLARE_REGISTRY_RESOURCEID(IDR_RtdServer)

For this to work you need to add a resource script to your project and add a text file as a "REGISTRY" resource type. Here’s what a minimal registration script looks like for an in process RTD server:
    
HKCR
{
    Kerr.Sample.RtdServer
    {
        CLSID = s '{B9DCFAAD-4F86-44d4-B404-9E530397D30A}'
    }
    NoRemove CLSID
    {
        ForceRemove {B9DCFAAD-4F86-44d4-B404-9E530397D30A} = s 'Kerr.Sample.RtdServer'
        {
            InprocServer32 = s '%MODULE%'
            {
                val ThreadingModel = s 'Apartment'
            }
        }
    }
}

This script just adds a ProgId and CLSID to the registry so that instances can be created given one or the other. It also defines the threading model for the COM class indicating in what type of apartment the COM class expects to be created. In this case, I’ve specified “Apartment” to indicate that the COM class needs to be created on a thread that is initialized as a single thread apartment and includes the necessary plumbing to support that apartment model namely a message pump. ATL takes care of parsing the script when registering and unregistering the COM server and updating the registry accordingly.

Now we can finally declare the interface methods that we must implement ourselves:

HRESULT __stdcall ServerStart(/*[in]*/ IRTDUpdateEvent* callback,
                              /*[out]*/ long* result) override;

HRESULT __stdcall ConnectData(/*[in]*/ long topicId,
                              /*[in]*/ SAFEARRAY** strings,
                              /*[in,out]*/ VARIANT_BOOL* newValues,
                              /*[out]*/ VARIANT* values) override;

HRESULT __stdcall RefreshData(/*[in,out]*/ long* topicCount,
                              /*[out]*/ SAFEARRAY** data) override;

HRESULT __stdcall DisconnectData(/*[in]*/ long topicId) override;

HRESULT __stdcall Heartbeat(/*[out]*/ long* result) override;

HRESULT __stdcall ServerTerminate() override;

And that’s it for the class definition. The only thing remaining is to add the class to ATL’s map of COM classes so that it can automatically register, unregister, and create instances of it.

OBJECT_ENTRY_AUTO(__uuidof(RtdServer), RtdServer)

It’s an Automation Server

Component Automation, more commonly known as OLE Automation, is a set of additional specifications built on top of the COM specification geared towards improving interoperability with scripting languages, tools and applications. There are pros and cons to Automation. For example the Automation Marshaller can be very handy in lieu of custom proxies even for C++-only systems. On the other hand Automation types and interfaces can be quite unwieldy to use from C++. Fortunately ATL does a great job of making Automation programming in C++ a whole lot less painful.

Although there are many parts to Automation, we only need to cover a few of them to support the RTD server. Firstly we need to implement the IDispatch-based RTD interface. ATL’s implementation of IDispatch relies on a type library so we’ll need one of those too. Finally, the RTD interface relies on Automation safe arrays and variants so we’ll need to know how to handle those.

An interface, like IRtdServer, that derives from IDispatch is known as a dual interface because it allows interface methods to be access either directly via the interface vtable or indirectly via the methods provided by IDispatch for resolving a method by name and then invoking it. Excel does the latter. ATL provides a generic implementation of IDispatch that relies on a type library to resolve method names and invoke methods. If you’re coming from a .NET background, a type library is the precursor to CLR metadata.

A type library is generated by the MIDL compiler given an IDL file as input. IDL is used for much more than just generating type libraries but that’s all we need it for right now. In a future post I’ll share some other tricks that you can perform with IDL and RTD servers.

Start by adding a “Midl File” called TypeLibrary.idl to your project. Here’s what it should contain. I won’t go into detail into what this all means but it should be pretty self-explanatory and it should not surprise you to learn that IDL stands for Interface Definition Language.  :)

import "ocidl.idl";

[
  uuid(A43788C1-D91B-11D3-8F39-00C04F3651B8),
  dual,
  oleautomation
]
interface IRTDUpdateEvent : IDispatch
{
    [id(0x0000000a)]
    HRESULT UpdateNotify();

    [id(0x0000000b), propget]
    HRESULT HeartbeatInterval([out, retval] long* value);

    [id(0x0000000b), propput]
    HRESULT HeartbeatInterval([in] long value);

    [id(0x0000000c)]
    HRESULT Disconnect();
};

[
  uuid(EC0E6191-DB51-11D3-8F3E-00C04F3651B8),
  dual,
  oleautomation
]
interface IRtdServer : IDispatch
{
    [id(0x0000000a)]
    HRESULT ServerStart([in] IRTDUpdateEvent* callback,
                        [out, retval] long* result);

    [id(0x0000000b)]
    HRESULT ConnectData([in] long topicId,
                        [in] SAFEARRAY(VARIANT)* strings,
                        [in, out] VARIANT_BOOL* newValues,
                        [out, retval] VARIANT* values);

    [id(0x0000000c)]
    HRESULT RefreshData([in, out] long* topicCount,
                        [out, retval] SAFEARRAY(VARIANT)* data);

    [id(0x0000000d)]
    HRESULT DisconnectData([in] long topicId);

    [id(0x0000000e)]
    HRESULT Heartbeat([out, retval] long* result);

    [id(0x0000000f)]
    HRESULT ServerTerminate();
};

[
    uuid(358F1355-AA45-4f59-8838-9A21E7F4628C),
    version(1.0)
]
library TypeLibrary
{
    interface IRtdServer;
};

The MIDL compiler parses this file and produces a few things. The main thing we need is a type library but it also produces the C and C++ equivalent of the IDL interface definitions so that we don’t have to define those ourselves. The type library produced by the MIDL compiler needs to be included in the DLL as a resource. You can achieve this by adding the following to your resource script:

1 TYPELIB "TypeLibrary.tlb"

Finally, you can update your ATL module class to tell it where to find the type library:

class Module : public CAtlDllModuleT<Module>
{
public:

    DECLARE_LIBID(LIBID_TypeLibrary)
};

LIBID_TypeLibrary will be declared in the header file and defined in the C source file produced by the MIDL compiler.

It’s an RTD Server

With all that out of the way we can finally add the actual minimal RTD server implementation. For this example I’m just going to port the minimal C# implementation that simply supports a single topic displaying the time.

We need just a few variables to make it work:

TimerWindow m_timer;
long m_topicId;

TimerWindow is simple C++ implementation of the Windows Forms Timer class used by the C# implementation. It just creates a hidden window to handle WM_TIMER messages and then calls the RTD callback interface on this timer:

class TimerWindow : public CWindowImpl<TimerWindow, CWindow, CWinTraits<>>
{
private:

    CComPtr<IRTDUpdateEvent> m_callback;

    void OnTimer(UINT_PTR /*timer*/)
    {
        Stop();

        if (0 != m_callback)
        {
            m_callback->UpdateNotify();
        }
    }

public:

    BEGIN_MSG_MAP(TimerWindow)
        MSG_WM_TIMER(OnTimer)
    END_MSG_MAP()

    TimerWindow()
    {
        Create(0);
        ASSERT(0 != m_hWnd);
    }

    ~TimerWindow()
    {
        VERIFY(DestroyWindow());
    }

    void SetCallback(IRTDUpdateEvent* callback)
    {
        m_callback = callback;
    }

    void Start()
    {
        SetTimer(0, 2000);
    }

    void Stop()
    {
        VERIFY(KillTimer(0));
    }
};

As with the C# implementation, since the timer relies on window messages it requires a message pump to function. Fortunately this particular RTD server lives in a single threaded apartment that provides one.

Now we can implement the RTD server methods quite simply. I’m not going to explain here why they’re called as I’ve already talked about the semantics in the walkthrough of the minimal C# implementation.

HRESULT ServerStart(/*[in]*/ IRTDUpdateEvent* callback,
                              /*[out]*/ long* result)
{
    if (0 == callback || 0 == result)
    {
        return E_POINTER;
    }

    m_timer.SetCallback(callback);
    *result = 1;
    return S_OK;
}

ServerStart passes the callback interface to the timer which holds a reference to it. It returns 1 to indicate that all is well.

HRESULT ServerTerminate()
{
    m_timer.SetCallback(0);
    return S_OK;
}

ServerTerminate clears the callback held by the timer to ensure that it isn’t accidentally called subsequent to termination.

HRESULT ConnectData(/*[in]*/ long topicId,
                              /*[in]*/ SAFEARRAY** strings,
                              /*[in,out]*/ VARIANT_BOOL* newValues,
                              /*[out]*/ VARIANT* values)
{
    if (0 == strings || 0 == newValues || 0 == values)
    {
        return E_POINTER;
    }

    m_topicId = topicId;
    m_timer.Start();
    return GetTime(values);
}

ConnectData saves the topic identifier, starts the timer, and returns the initial time.

HRESULT GetTime(VARIANT* value)
{
    ASSERT(0 != value);

    SYSTEMTIME time;
    ::GetSystemTime(&time);

    CComBSTR string(8);

    swprintf(string,
             string.Length() + 1,
             L"%02d:%02d:%02d",
             time.wHour,
             time.wMinute,
             time.wSecond);

    value->vt = VT_BSTR;
    value->bstrVal = string.Detach();

    return S_OK;
}

GetTime is a simple helper function that produces a string with the current time and returns it as a variant.

HRESULT DisconnectData(/*[in]*/ long /*topicId*/)
{
    m_timer.Stop();
    return S_OK;
}

DisconnectData simply stops the timer.

HRESULT Heartbeat(/*[out]*/ long* result)
{
    if (0 == result)
    {
        return E_POINTER;
    }

    *result = 1;
    return S_OK;
}

Heartbeat simply returns 1 to indicate that all is well.

HRESULT RefreshData(/*[in,out]*/ long* topicCount,
                               /*[out]*/ SAFEARRAY** result)
{
    if (0 == topicCount || 0 == result)
    {
        return E_POINTER;
    }

    CComSafeArrayBound bounds[2] =
    {
        CComSafeArrayBound(2),
        CComSafeArrayBound(1)
    };

    CComSafeArray<VARIANT> data(bounds, _countof(bounds));
    LONG indices[2] = { 0 };

    HR(data.MultiDimSetAt(indices, CComVariant(m_topicId)));

    CComVariant time;
    HR(GetTime(&time));

    indices[0] = 1;
    HR(data.MultiDimSetAt(indices, time));

    *result = data.Detach();

    *topicCount = 1;
    m_timer.Start();
    return S_OK;
}

The RefreshData method is where you need to finally deal with Automation safe arrays and you can start to appreciate just how much work the CLR’s marshaller does for you. Fortunately ATL provides a fairly clean wrapper for the various structures and API functions needed to create and interact with safe arrays. RefreshData creates a safe array by first defining its dimensions and bounds using ATL’s CComSafeArrayBound class. Here the safe array will have two dimensions. The first dimension has two elements and the second has one. The first dimension of the array will always have two elements. The first element is for the topic Ids and the second is for the values. The second dimension will have as many elements as there are topics with data to return. In our case there is only one.

The rest of the implementation just goes about populating the safe array with the topic Id and value and then restarts the timer.

That’s all for today. I skimmed over many details as this entry was getting way too long, but I hope this walkthrough has given you some idea of what’s involved in writing a minimal RTD server in C++.

With that out of the way, I can start talking about some of the more interesting challenges and techniques you can employ in your implementations and various other topics related to RTD server development.

Continuing the discussion of Excel RTD servers, this time I’m going to show you how to write a minimal RTD server in native C++. There’s quite a bit more to discuss so I broke this entry into two parts.

Last time I showed you a minimal RTD server written in C# and it sure is easy to bang it together. There isn’t a whole lot of code to write thanks to the mountains of abstractions provided by the CLR and the .NET Framework. Those abstractions can however get in the way. If nothing else they get in the way of our understanding of what is actually going on under the covers and that in turn can lead to unexplained bugs. If you’ve learnt anything from reading my blog I hope it is the value of understanding how things work, not just how to get them to work. Did you catch the difference?

Although C++ affords you many more options for implementing and deploying an RTD server, here I’m focusing on the simplest implementation of an in-process apartment model COM server. Subsequent entries will explore additional scenarios that go beyond this starting point.

It’s a Visual C++ Project

Every good project starts with an empty Visual C++ project.  :)

Create a new project in Visual Studio based on the Visual C++ “Win32 Project” template. Select the Application Settings tab in the Win32 Application Wizard. Select the “DLL” application type and also check the “Empty project” option.

Add a precompiled header to the project and include the following:

#pragma once

#include <atlbase.h>
#include <atlcom.h>

#define ASSERT ATLASSERT
#define VERIFY ATLVERIFY
#define TRACE ATLTRACE
#define HR(expr) { HRESULT expr_hr = expr; if (FAILED(expr_hr)) return expr_hr; }

atlbase.h is the main ATL header that includes the main Windows SDK headers and the most common ATL functions and classes used by virtually all native C++ applications (that use ATL).

atlcom.h provides most of the code for generating all the boilerplate code required to implement a COM server.

The macros are optional but help to simplify the code and should be self-explanatory.

It’s a DLL

Since we’re limiting this implementation to an in-process COM server it will be packaged as a dynamic-link library. DLLs built with the C run-time library use an entry point function called DllMain. This should be used as an indicator of when the operating system is loading and unloading your DLL and not much else.

STDAPI_(BOOL) DllMain(HINSTANCE /*module*/,
                      DWORD /*reason*/,
                      void* /*reserved*/)
{
    return TRUE;
}

DLLs typically communicate with the outside world by providing a set of exported functions. One way to define these exports is via a module definition file. Add a file to the project called Exports.def and include the following text:

LIBRARY "RtdServer"

EXPORTS

The name in quotes must match the name of the DLL produced by the linker. Any functions you wish to export are listed under the EXPORTS header. You need to also tell the linker about your module definition file. You can set this in the linker’s “Input” properties from within the project’s property pages.

Although DllMain isn’t strictly an exported function, I tend to stick it along with any other exported functions in a C++ file called Exports.cpp just so that it’s obvious where all the code paths within the DLL originate from.

It’s a COM Server

A COM server must provide a way for a client to get the class object for a particular CLSID. COM servers hosted in a DLL achieve this by exporting a function called DllGetClassObject. The DllRegisterServer and DllUnregisterServer functions are also typically exported and called by installers or simply by the built-in regsvr32.exe tool to register and unregister the server. Finally, the DllCanUnloadNow function should also be exported to allow the COM runtime to unload your DLL promptly.

These functions are well defined in the COM specification and I won’t go into their implementation details too much. The best thing you can do is rely on ATL to provide the implementation. Start by defining an ATL module class for your DLL as follows.

class Module : public CAtlDllModuleT<Module>
{
};

Amazingly you can now implement the DLL’s entry point and all of the necessary exported functions simply by delegating to the ATL implementation as follows:

Module s_module;

STDAPI_(BOOL) DllMain(HINSTANCE, DWORD reason, LPVOID reserved)
{
    return s_module.DllMain(reason, reserved);
}

STDAPI DllRegisterServer()
{
    return s_module.DllRegisterServer(FALSE);
}

STDAPI DllUnregisterServer()
{
    return s_module.DllUnregisterServer(FALSE);
}

STDAPI DllGetClassObject(REFCLSID clsid, REFIID iid, void** object)
{
    return s_module.DllGetClassObject(clsid, iid, object);
}

STDAPI DllCanUnloadNow()
{
    return s_module.DllCanUnloadNow();
}

Now that the exported functions are implemented you can add them to the list of exports in the module definition file:

EXPORTS
DllRegisterServer PRIVATE
DllUnregisterServer PRIVATE
DllGetClassObject PRIVATE
DllCanUnloadNow PRIVATE

The PRIVATE keyword prevents other binaries from directly linking to these functions. You can of course still call them via LoadLibrary/GetProcAddress which is exactly what COM and tools like regsvr32.exe would do.

That’s it for today. Next time we’ll conclude the minimal C++ implementation with: