QQQ is sending out an SOS

Thomas Marquardt, one of our developers on ASP.NET, recently published an article on MSDN about ASP.NET performance monitoring.  Along with the article are a number of great utilities (all complete with C# source code). 

We developed them internally to help automate diagnosing customer issues with production apps, and they are also used as part of our ASP.NET stress testing program (I'll provide more details about our stress lab in a future blog entry).  These source utilities include:

Snap.exe -- a command-line tool for logging performance data for processes.

HttpClient.exe -- a simple client that records time to last byte (TTLB) for HTTP requests.

ErrorHandler.dll. This is an IHttpModule that you can add to the HTTP pipeline to log unhandled exceptions to the event log. It is better to log errors to a SQL Server database (that is what we do in our stress lab -- since it makes reporting across hundreds of servers easier), but the example uses the event log for simplicity.

QQQ.exe -- a command-line tool for stress testing an ASP.NET application. When used in combination with a stress client, such as Homer or ACT, this tool will attach debuggers to the worker process and monitor certain performance counters.  It can be tuned to automatically break into the debuggers when performance degrades.

QQQ is probably my favorite utility -- especially given that its name is complete nonsense.  "QQQ" stemmed from the fact that Dmitry (ASP.NET dev manager) likes to name all of his test ASP.NET pages with those initials, usually under various temporary directories scattered accross his harddrive.

QQQ is usually used in combination with the windbg and cdb debuggers (which you can download from: http://www.microsoft.com/ddk/debugging/installx86.asp).  These debuggers are more appropriate for system level dubugging (versus the VS debugger) in that they have less effect on the process they are attached to (unlike the VS debugger which tends to impact timings within the target process -- making stress investigations harder).

QQQ is also typically used in conjunction with a .NET Framework debugger extension we call "sos.dll".  Strike.dll was the parent debugger extension we used in V1 of the .NET Framework -- "sos.dll" affectionately stands for "son of strike".  You can download son of strike at: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/DBGch03.asp.

If you are doing hard-core production ASP.NET development, and want to build up an infrastructure for testing and monitoring your web applications, then I'd definitely recommend spending an hour browsing through the above code.  There are a lot of cool tips/tricks you can pick up and re-use.

Quick Update: A few people have pinged me as to how they can get the source to the above utilities.  Note that a link to them is in the top article above.  The direct url is: http://download.microsoft.com/download/3/c/8/3c820668-3095-40e2-b462-0bc8a0b6690f/monitoring_tools.exe