Roland Weigelt

In recent months, I wrote a few articles that hinted that I was developing a webcam-related utility (here, here, here and here). About two weeks ago, I released it to my colleagues who liked it and found it useful. Now, after some improvements, I am ready for a release to a larger audience.

What is it?

CamProps is a free utility for quickly adjusting and managing the properties of webcams and other video capture devices.

Where can I get it?

You can download CamProps at https://www.roland-weigelt.de/camprops/

Why should I use it?

The picture quality of many webcams can be improved by manually tweaking properties like focus, exposure or gain. When doing so, you may run into a few issues, though:

• If you are in a room with changing lighting conditions, you have to adjust the settings each time you want to use the camera.
• Some third-party software may decide to reset the camera back to default.
• And some devices simply forget all settings after a reboot.

CamProps lets you

• quickly adjust camera properties,
• manage different sets of configurations, and
• restore the settings with a single click.

Which devices can I use?

CamProps works with all webcams and video capture devices that support opening the webcam driver property dialog (either the default provided by Windows or a specific dialog by the manufacturer's driver).

For instance, when you press the “” button for a Logitech C920 without the Logitech software installed, this will open the following dialog:

(This dialog may look different for other devices)

How do I start?

• Head to https://www.roland-weigelt.de/camprops/
• Download and install the program.
• Read the “How to…” section on setting up your first configuration, activating a configuration and backing up the automatically stored settings.

In a previous blog post, I wrote about how to access webcam properties from C# using DirectShow.Net. The last release of the library is version 2.1 from February 2010, with project files for Visual Studio 2008.

Fortunately, porting the project to compile for .NET 5 went pretty quick.

The project file

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0-windows</TargetFramework>
    <RootNamespace>DirectShowLib</RootNamespace>
    <SignAssembly>true</SignAssembly>
    <AssemblyOriginatorKeyFile>DShowNET.snk</AssemblyOriginatorKeyFile>
    <DelaySign>false</DelaySign>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
    <PackageId>DirectShowLib.Net</PackageId>
    <Authors>http://sourceforge.net/projects/directshownet/</Authors>
  </PropertyGroup>

</Project>

Things to note:

• The target framework is net5.0-windows because the code (obviously) uses Windows-specific features.
• I added <GenerateAssemblyInfo>false<GenerateAssemblyInfo> to use the AssemblyInfo.cs file that came with the original project.

AssemblyInfo.cs

The following modifications were necessary:

• Change AssemblyVersion from 2.1.0.* to 2.1.0.0 because the compiler complained about the wildcard.
• Remove the lines related to the .snk file (now handled in the project file).
• Add SupportedOSPlatform("windows").

using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Security.Permissions;


[assembly : AssemblyTitle("DirectShow Net Library")]
[assembly : AssemblyDescription(".NET Interfaces for calling DirectShow.  See http://directshownet.sourceforge.net/")]
[assembly : AssemblyConfiguration("")]
[assembly : AssemblyCompany("")]
[assembly : Guid("6D0386CE-37E6-4f77-B678-07C584105DC6")]
[assembly : AssemblyVersion("2.1.0.0")]
#if DEBUG
[assembly : AssemblyProduct("Debug Version")]
#else
[assembly : AssemblyProduct("Release Version")]
#endif
[assembly : AssemblyCopyright("GNU Lesser General Public License v2.1")]
[assembly : AssemblyTrademark("")]
[assembly : AssemblyCulture("")]
[assembly : AssemblyDelaySign(false)]
// Path is relative to the resulting executable (\Bin\Debug)
#if USING_NET11
[assembly : AssemblyKeyFile("..\\..\\DShowNET.snk")]
#endif
[assembly : AssemblyKeyName("")]
[assembly : ComVisible(false)]
[assembly : CLSCompliant(true)]
[assembly : SecurityPermission(SecurityAction.RequestMinimum, UnmanagedCode=true)]
[assembly: System.Runtime.Versioning.SupportedOSPlatform("windows")]

LGPL

The DirectShow.Net library is licensed under the Lesser General Public License (LGPL) Version 2.1.

To be able distribute my software at some point in the future, the LGPL license requires – in addition to proper attribution – to

• ship the library as an individual DLL (instead of ripping out just the parts that I need), and to
• make the source code available publicly.

In order to comply with the LGPL, I published my fork of the library on https://github.com/RWeigelt/DirectShow.Net.

Please note that I did not perform any tests of features that go beyond accessing webcam properties. The library compiles, I can use it in my project, and so far it did not break my code – that’s good enough for me at this moment…

I am currently working on a small webcam utility written in C#/WPF that I intend to release as freeware later this year.

On the main window, I use a ComboBox for selecting a webcam in a dropdown list, with the toggle button of the control styled to look like a heading:

When you move the mouse pointer over the toggle button, the text and the glyph turn blue – nothing special.

The application does not have a main menu, only a settings menu that you open by clicking a gear icon:

The icon also turns blue when the pointer hovers over the general area:

But wait a second… is this the same blue? A quick check:

• The XAML says the text color is #0987c3.
• The color picker in Photoshop tells me that the color of most icon pixels is… also #0987c3!

What is going on?

On a screenshot zoomed to 300%, the gear icon looks like this:

And this is the text at 300% size:

The colored pixels to the left and the right of the characters are caused by ClearType. You will also notice that many pixels “inside” the character shapes also have a different color. This is because less pixels than one might expect are fully covered in blue – which means that more pixels have their color modified by ClearType.

What did I do wrong?

It is not what I did do, but what I did not do. I did not tweak WPF’s text rendering to suit my needs. Which sounds curious, because writing a desktop application that looks good at typical application font sizes is not exactly an exotic usage scenario.

When WPF was developed, it was designed to be device-independent and ready for a future of high pixel density displays (“HiDPI”). So WPF rendered text exactly as specified by the font metrics, which on normal density displays led to blurry characters. It took until .NET 4 for WPF to offer a way to render small text with an emphasis on readability instead of correctness (read this old article by Scott Hanselman for a look way back in time).

What to do

As a WPF developer, you can influence the text rendering by using the attached properties TextOptions.TextFormattingMode and TextOptions.TextRenderingMode. You apply them directly on an element, e.g., a TextBlock or set them on a window.

TextOptions.TextFormattingMode

The property TextOptions.TextFormattingMode can be set to Display or Ideal (which is the default).

• With Ideal, text is rendered according to the font's metrics. Which is fine for large text, but not so much for small text on a non-HiDPI display.
• With Display, the text rendering favors of aligning letters to pixels. This is especially important for small text on normal density screens.

TextOptions.TextRenderingMode

The property TextOptions.TextRenderingMode controls the way rendered text will be anti-aliased. Possible values are Aliased (which we will see can be confusing), Auto (the default), ClearType, and Grayscale.

To show off the possible combinations, I placed the gear icon directly in front of the text (where it does not make any sense, just for comparing colors):

Display	Aliased
Display	Auto
Display	ClearType
Display	Grayscale
Ideal	Aliased
Ideal	Auto
Ideal	ClearType
Ideal	Grayscale

So many options…

In a first round, I eliminated:

• Display&Aliased (no anti-aliasing),
• Ideal&ClearType (the combination that caused me to write this article in the first place),
• Ideal&Auto (which in my simple scenario looks the same as Ideal&ClearType), and
• Display&ClearType (even though it looks better than Ideal&ClearType)

Which left me with the following options:

Display	Grayscale
Ideal	Aliased
Ideal	Grayscale

In the end, I decided to use Ideal&Aliased because I wanted the bolder text without a larger font size. And Segoe UI in “real” bold does not look good at this font size. Interesting that with Ideal, the setting Aliased is actually anti-aliased…

I set TextOptions.TextFormattingMode="Display" on my application window but did not specify TextOptions.TextRenderingMode there.

For the dropdown list, I set TextOptions.TextFormattingMode="Ideal" and TextOptions.TextRenderingMode="Aliased" on the ContentPresenter of the toggle button.

A note on “Display” vs. “Ideal” used with “Grayscale”

While writing this blog post, I had to juggle a larger number of image files than usual. When I saw the letter “P” in the enlarged versions of Display&Grayscale and Ideal&Grayscale, I first suspected I had mixed up the screenshots. But that was not the case; it is just a coincidence that the “H” and the “P” look sharper.

This becomes clear when you look at a different text:

Display	Grayscale
Ideal	Grayscale

Note the way the “m” is rendered and how more vertical lines are aligned to full pixels in Display mode.

A good example for small sample sizes leading to wrong conclusions…

My Logitech C920 webcam requires some tweaking of settings like exposure, gain, focus, etc. to get good image quality. I uninstalled the “feature rich” Logitech software and now change the settings using the bare-bones Windows webcam properties dialog. This works well for me; unfortunately, the settings are not persisted reliably. After a cold-boot, or sometimes after simply starting an application that uses the webcam, I have to make the adjustments again.

That is why I became interested in what would it take to read and write the webcam properties and to open the properties dialog from C#. The result of a web search was a bit intimidating as I came across multimedia APIs that go way beyond what I intended to do. After all, I only wanted to access the properties, not write a full-blown video capture suite.

In the end I settled on DirectShow.Net, a C# wrapper around DirectShow under LPGL license. Even though DirectShow is an old API and the DirectShow.Net project seems to be no longer active, I found one very important reason to use it: A working sample that opened the webcam properties dialog.

This blog post starts with a guided tour through the sample, with the intended side-effect of making the sample easier to discover on the web. Additionally, I will describe how to access the properties from your code.

Step 1: Download DirectShow.Net

• Visit  http://directshownet.sourceforge.net/
• Go to “Downloads”
• Download the latest version of the library (DirectShowLibV2-1.zip at the time of this writing)
• Download the samples (DirectShowSamples-2010-February.zip)
• Unpack the ZIP files so that the folder Samples is in the same directory as Docs, lib and src.
• The lib folder contains the file DirectShowLib-2005.dll which the samples reference.

Step 2: Run the “DxPropPages” demog

• Open Samples\Capture\DxPropPages\DxPropPages-2008.sln in Visual Studio and let the “One-way upgrade” finish.
• Run the project.
• In the window that appears,
  • select your webcam and
  • press the “Show property pages” button.
  
   
• On my computer, a properties dialog with two tabs appears. Depending on your drivers/webcam software, the dialog may have been replaced. But the generic, low-level dialog looks like this:
  
  
  
  (Remember “Video Proc Amp” and “Camera Control”, we will come across these names later)

Step 3: Look at the code

• Stop the program.
• Open the code view of the file Form1.cs.
• Set a breakpoint at the start of
• the constructor e Form1(),
• the comboBox1_SelectedIndexChanged() event handler, and
• the DisplayPropertyPage() method.
• Run the program in the debugger.

How to get the available webcam(s)

When the first breakpoint is hit, you will see the following lines:

foreach (DsDevice ds in DsDevice.GetDevicesOfCat(FilterCategory.VideoInputDevice))
{
    comboBox1.Items.Add(ds.Name);
}

The code gets all available “video input devices” (which include webcams) and fills the dropdown that you used in step 2 to choose your webcam.

A DsDevice instance has two important properties for identifying a device:

• Name returns a human-readable name (i.e., what you saw in the dropdown list)
• DevicePath returns a unique identifier.

At this point, the sample does not store the instances, only the names, even though we need the DsDevice instance for the selected webcam later. I am not sure whether there is a reason for this other than keeping the sample code short and to be able to re-use the CreateFilter() method (which we will look at soon).

How to open the properties dialog

Now continue to run the program. The comboBox1_SelectedIndexChanged event handler gets called automatically during startup. If your webcam is not the first device, let the program continue and select the webcam in the dropdown.

After the breakpoint has been hit, look at the code.

• The purpose of the event handler is to set the field theDevice (of type IBaseFilter) which we need later.
• The call of Marshal.ReleaseComObject(theDevice) when switching between devices is a reminder that we are dealing with COM and its reference counting (instead of relying on garbage collection).
• Note that the variable name devicepath is misleading; the dropdown contains the display names of the devices. This becomes clear when we look at the CreateFilter() method: The second parameter is called friendlyname which is more appropriate.

Inside the CreateFilter() method, some “COM stuff” happens. The important bit for us is that the returned IBaseFilter is assigned to the field theDevice, which is used in the button1_Click handler when calling DisplayPropertyPage().

The method DisplayPropertyPage() contains even more COM stuff that we can ignore for now, because the method does exactly what its name says. We will see later that we need some basic understanding what is happening inside, though.

How to make the controls in the dialog appear “Windows 10”-like

The steps described my blog post “Windows 10 Theme for a Properties Dialog” for a WPF application are also valid for WinForms. In the case of the sample application the change also affects the controls of the main window.

Step 4: Start to experiment

The code inside DisplayPropertyPage() uses the ISpecifyPropertyPages interface. Two other interesting interfaces are IAMVideoProcAmp and IAMCameraControl. The names correspond to the pages of the properties dialog. Using the two interfaces, you can access the properties you see in the dialog.

How to read or write the webcam properties from your code

The interfaces IAMVideoProcAmp and IAMCameraControl both offer GetRange(), Get() and Set() methods.

For IAMCameraControl, these methods are defined like this:

int GetRange(
	[In] CameraControlProperty Property,
	[Out] out int pMin,
	[Out] out int pMax,
	[Out] out int pSteppingDelta,
	[Out] out int pDefault,
	[Out] out CameraControlFlags pCapsFlags
	);

int Set(
	[In] CameraControlProperty Property,
	[In] int lValue,
	[In] CameraControlFlags Flags
	);

int Get(
	[In] CameraControlProperty Property,
	[Out] out int lValue,
	[Out] out CameraControlFlags Flags
	);

When using the methods:

• You specify the property you want to access via an enum value of type CameraControlProperty. Your device may not support all properties, though – if you look at the screenshots above, you will notice that some sliders are disabled. Therefore it is important to check the return value to be 0 (zero) for a successful call.
• The CameraControlFlags value contains information whether the property is (or should be) set automatically and / or manually.

Let us say you want to access the “exposure” property of your webcam (this may or may not work on your webcam; if not, you can try another property).

For a quick-and-dirty test, resize the “Show property pages” button so can add another button next to it, double click the new button and insert the following code into the “Click” event handler:

var cameraControl = theDevice as IAMCameraControl;
if (cameraControl == null) return;

cameraControl.GetRange(CameraControlProperty.Exposure,
	out int min, out int max, out int steppingDelta,
	out int defaultValue, out var flags);

Debug.WriteLine($"min: {min}, max: {max}, steppingDelta: {steppingDelta}");
Debug.WriteLine($"defaultValue: {defaultValue}, flags: {flags}");

When I run the program, select my Logitech C920 webcam and press the button I added above, the following appears in the debug output window in Visual Studio:

min: -11, max: -2, steppingDelta: 1
defaultValue: -5, flags: Auto, Manual

This means that the exposure can be adjusted from -11 to -2, with -5 being the default. The property supports both automatic and manual mode.

Not all properties have a stepping delta of 1. For the Logitech C920, for instance, the focus property (CameraControlProperty.Focus) has a range from 0 to 250 with a stepping delta of 5. This is why setting the property value to e.g. 47 has the same effect on the hardware as setting the value to 45.

Calling the Get() and Set() methods is simple. For instance, setting the focus to a fixed value of 45 looks like this:

cameraControl.Set(CameraControlProperty.Focus, 45, CameraControlFlags.Manual);

The CameraControlFlags.Manual tells the webcam to switch off aufo-focus.

Where to go from here

Note the COM pitfall in the sample

If you are as inexperienced working with COM interop as I am and look at the original sample code inside DisplayPropertyPage(), you may notice that the line

ISpecifyPropertyPages pProp = dev as ISpecifyPropertyPages;

seems to have a corresponding

Marshal.ReleaseComObject(pProp);

Does this mean that we need a similar call in our experimental code we added above?

No, because if you add the (only supposedly) “missing”  Marshal.ReleaseComObject(cameraControl) to your code and click the button repeatedly, you will run into this exception:

System.Runtime.InteropServices.InvalidComObjectException
  HResult=0x80131527
  Message=COM object that has been separated from its underlying RCW cannot be used.
  Source=DxPropPages
…

What is happening here? The answer is that simply “casting” to a COM interface in C# is not something that has to be “cleaned up”. The code may imply that, but you could change the line

Marshal.ReleaseComObject(pProp);

to

Marshal.ReleaseComObject(dev); // oDevice would work, too

and it still would run without leaking references.

How do I know? Because Marshal.ReleaseComObject() returns the new reference count and changing the line to

Debug.WriteLine(Marshal.ReleaseComObject(oDevice));

will output 1 each time we opened and close the properties dialog. The value of 1 is correct, because want to continue to be able to access the device object.

Placing a copy of that line in front of the call of the external function OleCreatePropertyFrame() obviously does not make sense and will lead to an exception. But if you do it anyway, just for testing, the debug output will show 0 instead of 1. This shows us that passing the object as a parameter in COM interop – at least in this case – caused the reference count to be increased. This is why Marshal.ReleaseComObject() is called after OleCreatePropertyFrame(), not because of the cast to ISpecifyPropertyPages.

Practice defensive coding

As already mentioned, not all webcams support all properties. And if a webcam supports a property, the allowed values may differ from other devices. That is why you should use GetRange() to determine

• whether a property is supported (return value 0),
• the range of the allowed values, and
• whether the property can be set to “auto”.

Last, but not least: When you access a USB webcam – like any other detachable device – be prepared for it not being available. Not only at startup, but also while your program is running, because the device could have been unplugged unexpectedly.

In a past blog post I wrote about a surprising change I encountered during the migration of a .NET Core 2.2 application to .NET Core 3.0. I have just migrated that same application to .NET 5 and guess what, I stumbled across another “tiny difference with big consequences”.

The situation: I have a WPF desktop client that communicates with an ASP.NET Core server that in turn communicates with a small TypeScript browser application. I use Web API for accessing files and meta data, and SignalR to call remote functions.

After the migration I ran the application and things were working fine – until a certain point, where I received the following exception in a call of SignalR’s HubConnection.InvokeCoreAsync() (actual function name replaced with “doSomething”):

Microsoft.AspNetCore.SignalR.HubException
   HResult=0x80131500
   Message=Failed to invoke 'doSomething' due to an error on the server.
   Source=System.Private.CoreLib

On the server side, my own code was not even called.

It took me a while to notice that one of the parameters of the function had the type of a class with a specific constellation of constructors:

• A public constructor with parameters
• A private default constructor.

It turned out while SignalR in .NET Core 3.1 has no problem (de)serializing an instance of a class with a private default constructor, SignalR in .NET 5 does. So simply removing that constructor fixed the problem. In terms forcing the usage of the other constructor when explicitly creating objects of that type, nothing changed; I did not need a private default constructor for that.

P.S. Do not ask me why I had a private default constructor in the first place. But the project is a couple of years old now, and things like this do happen.

Today I played around with some code in an WPF application that opens a (COM-style) properties dialog.

When opened outside my application, the dialog looked like this:

But when I opened the dialog from my code, it looked like this:

The solution is simple but appears blog-worthy to me because I had a hard time searching the Web using rather unspecific search terms like “properties dialog”, theme, style, etc.

In the end I came across a theme-related Microsoft article for C++ mentioning a manifest file and connected the dots to a StackOverflow question/answer about changing the Windows User Account Control level for a C#/WPF application using a custom manifest.

What to do

Create the custom manifest file

• In Visual Studio (2019), open the context menu of the project for the WPF application.
• Add > New item.
• Ctrl+E (to jump to the search box), enter “manifest”.
• Press the “Add” button to add the “app.manifest” file to the project.
• Visual Studio selects the file automatically (see project properties > Application > Resources > Icon and manifest > Manifest)

Edit the manifest

• Open the “app.manifest” file.
• Scroll to the bottom.
• Uncomment the section behind the “Enable themes for Windows common controls and dialogs (Windows XP and later)” comment.
• Rebuild.

That’s it!

In case you are wondering about my (webcam) experiment: This will be the topic of an upcoming blog post.

In a previous blog post I wrote about how mental models help you understand how something works. These models in your mind explain what did happen in the past and provide a reasonable (but not always reliable) prediction for will happen in the future. Interestingly, mental models can do that even though they are simplified, incomplete, and often enough simply wrong.

Empathy also has been a topic on my blog; I wrote about what it is and why it is important and why it is good for you, too.

In this article, I combine the two topics and present you my personal mental model of empathy. Note that it is a mental model, not the mental model for empathy. As such, it is, well… simplified, incomplete, and probably wrong. Still, it is “good enough” for many purposes and hey, it comes with pictures!

To start things off, imagine you have developed a non-trivial application as a single developer.

You, the developer

Here you are, living in your frame of reference, depicted as a nondescript coordinate system:

(I learned at university that axes without units are the worst. Have I told you how mental models are likely to be incomplete?)

Your knowledge about the application

This is what you, the developer, could theoretically know about the application:

You, the user

Now you try to imagine “the user”. Not being the developer of the application, the user is in a different situation:

A first step is trying to put yourself in the other person’s shoes:

The user’s knowledge about the application

You assume that not being the developer means that you cannot have knowledge of the inner workings. As a user, you only have the UI, the manual and other public sources at your disposal for understanding the application. This means that the user inevitably must know less about the software:

Next you are trying to employ some empathy. You think that it is unrealistic that user has read and understood all publicly available information. So most likely the user will know less than theoretically possible:

(Let’s ignore the relative sizes, you get the idea)

But: You are not the user

Always be aware that it is not you, but somebody else in the situation you are looking at:

This person could be highly motivated to use your application. Never underestimate the amount of work people will put into learning something if they feel it enables them to create results that are worth it. Just think how deep some people dig into Adobe After Effects or Microsoft Excel.

But in general, users most likely acquire just enough knowledge to get the job done. This means that they will know next to nothing about some features. At the same time, they can be very proficient in other features if a task at hand demands it.

So the graph depicting the user’s knowledge could look like this, with the sectors standing for parts of the application or large features:

Another user

A different person…

…may use the application for other purposes.

He or she may have more or less experience, in different areas:

A good start, but not quite there, yet

At this point, we have established that you are not the user and that different users have different needs and thus varying levels of knowledge.

But we are still stuck in our own point of view. For instance, we assess the user’s knowledge in terms of the application’s structure as we see it from a developer’s perspective. Even if we consider scenarios during development, we still tend to think in application parts as well as larger or smaller features.

Next: Stop thinking in your frame of reference

The next step is to acknowledge that users have a different frame of reference. As developers, we work on our software for long periods of time, which gives it a high importance in our daily life.

For users, our software is just another application. For them, is is not exactly the center of the universe.

Or, in other words, they live in their own coordinate system:

As an experiment, try to watch yourself how you use and experience the software you did not develop yourself. How different applications, tools and utilities have different importance. And how some software simply does not excite you enough to really learn more than the absolute basics about it.

Now capture that feeling. Be assured, some user out there will feel the same towards your application. A user like this may never become a big fan of your software, but if you design for them, you automatically help everybody else.

If you…

• lead the eye to the right points (by visual design, not a way-too-long onboarding experience)
• write clear UI texts (that even people can understand who do not want to read) and
• design features that are not simply thin UI layers over existing APIs (because users do not think in APIs),

everybody wins.

Using the software vs. getting the job done

Some application features map directly to a user scenario, some scenarios may span multiple features.

But be aware that “real world” scenarios do not necessarily start and finish inside your software. Users may start with ideas, thoughts, purpose, constraints, raw data, etc. outside your application. The work done inside the application may be a first step towards something very different (which means that e.g. a simple CSV export feature can be more important than the fancy result display that you developed). And in between, the scenario may take the users to use other software.

A person in the situation of achieving a specific goal will likely have a different view of the world. To visualize this, think of a coordinate system that looks very different from yours:

Things to remember

• The way you view your application is influenced by your knowledge of application parts and individual features.
  Other people may view it in a different way, based on what they see along their path through the user interface.
• Assume that people do not actually want to use your software, they want to achieve a specific goal.
  If that aligns with your software, good for everybody.
• Be aware that even though you and your users have the software as a connection point, your world view may be very different
  As a visual reminder, imagine yourself in a cartesian coordinate system and the user in a spherical coordinate system with a different origin. Neither of you lives in the “correct” or “better” coordinate system, they are just different. But the other coordinate system surely does not have your software as the center of the world, that is for sure…

P.S.

If you are interested in a (non-technical) example of completely different coordinate systems, take a look at my post “Exercise in Thinking: Do Racing Stripes Make a Car Go Faster?”.

In my spare time, I work on a software for the courtside advertising system of a local sports arena. To create the playlists for the advertising loops on the LED modules, users can drag and drop preview thumbnails of video and image files. Which brings us to the topic of this blog post, i.e. how to create these thumbnails.

Ways to create thumbnails

Win API

When I started the project in C#/WPF many years ago, I used some Win API code from a StackOverflow answer by Daniel Peñalba.

StorageFile.GetThumbnailAsync

After a failed attempt at a UWP rewrite, my software now runs on .NET Core 3.1, with a WPF management client, a ASP.NET/SignalR server and a HTML5/Typescript playback client (using a Chrome browser in kiosk mode, although I may change that when Microsoft Edge WebView2 comes out of preview).

Working with UWP introduced me to StorageFile.GetThumbnailAsync, which I kept using because the StorageFile API also gives me easy access to metadata (e.g. width, height and duration of a video).

The StorageFile API is not exactly known for high performance, but in my case this is not an issue, because the number of media files added in one go is typically in single digits.

FFmpeg

As mentioned above, StorageFile.GetThumbnailAsync works for me. So, why am I considering another method for the next version of my software?

For an answer I have to digress a bit: My software initially used individual MediaElement controls for the eleven LED modules. While I did manage to sync up the video playback, making it robust and reliable for many hours of looped playback required many workarounds in the code. To make matters worse, the reliability depended on the video format.

The videos to be played come from a variety of sources; some are produced by advertising agencies or video production companies, others are the work of semi-professionals. Some match the requested specs, many do not. In the end, I made it a habit to transcode each video myself.

Fortunately, this did not cause a noticeable loss of quality. Which encouraged me to try a different approach when I rewrote the playback to be performed in a browser. Now my software uses FFmpeg to combine the individual videos into one MP4 file. FFmpeg is “A complete, cross-platform solution to record, convert and stream audio and video” and this quote from the website is not an understatement.

In addition to offering a wide variety of features to manipulate inputs and outputs, FFmpeg also has been able to handle virtually any media file it has come across so far. Until now I could not fully benefit from this robustness because StorageFile.GetThumbnailAsync does not work with the same range of file formats. For example, on my computer, MOV files cause an exception (note the emphasis, YMMV).

FFmpeg to the rescue: Creating a thumbnail from a video or image file is something that FFmpeg can do easily – with two caveats:

1. It is slower than GetThumbnailAsync (at least when calling the executable as a separate process, like I do)
2. It does not work for general purpose files, only for videos or images.

My plan is to continue to use StorageFile.GetThumbnailAsync as the default method and to use FFmpeg as a fallback in case of an exception.

Demo code

I have posted a Visual Studio demo project on https://github.com/RWeigelt/ThumbnailCreationDemo. The code shows the use of StorageFile.GetThumbnailAsync and FFmpeg (for Win API, see the StackOverflow answer). Please note that the code is not intended to be a ready-to-use library; instead, you should run the demo, maybe set a few breakpoints, follow the code and use the parts that are of interest to you.

A few prerequisites:

• Your Visual Studio 2019 (I used 16.7.1) should be able to compile .NET Core 3.1 command line applications.
• You must have FFmpeg on your computer. You can get it from https://ffmpeg.org.

• Before you run the project, change the following line in “Program.cs” so the path matches the location of the FFmpeg executable on your computer:

  const string _FfmpegExeFilePath = @"C:\Program Files\FFmpeg\bin\ffmpeg.exe";

The project comes with two example files, “Image.png” and “Video.wmv”, with 768 x 80 pixels each (the size of a LED advertising module). During a build, the files will be copied to the output directory.

Running the project inside Visual Studio creates two thumbnail files for each media file (“…thumb1.png” and “…thumb2.png”). At the end you will be offered to open an Explorer window for the output directory, so you do not need dig around in the file system.

While the code for using StorageFile.GetThumbnailAsync is straightforward, the usage of FFmpeg needs some explanation.

The call of ffmpeg.exe is hidden behind a Task using a TaskCompletionSource. For the purpose of the demo code, the Task returns the duration in milliseconds (instead of e.g. the exit code of the FFmpeg process). This is something you may want to change for production code.

The command line parameters:

• For video files,
• -ss 5 means “seek to the position five seconds into the media file”.
• -an prevents audio streams from being handled in any way
  
• -i specifies the input file
• -vframes 1 gets one frame from the input file
• -s specifies the output size
• -y to confirm overwriting an existing file
  
• The last parameter is the output file; the extension of the thumbnail file tells FFmpeg that we want the file in PNG format.
  

Note that the order of the parameters makes a difference: Parameters in front of the input file (-i) are treated as options that affect the input, parameter behind affect the output.

For instance, the -ss 5 in front of the -i simply seeks into the file (as one would expect). If the -ss 5 is placed behind the -i, then FFmpeg will actually decode five seconds of the input file frame by frame, which takes more time. The difference is noticeable even for this simple demo, try it out. For more information see https://trac.ffmpeg.org/wiki/Seeking.

What about metadata?

As I wrote, on my computer, calling StorageFile.GetThumbnailAsync causes an exception for a MOV file (yes, this could surely be fixed with the right codec pack, but for the moment I keep things the way they are to test error handling).

But I can retrieve metadata using StorageFile.Properties – that is because the metadata is not determined at the moment I ask for it, it simply is read from the file without actually looking at its content. This is fortunate because for my purposes, reading the metadata via the StorageFile API is much easier than using e.g. ffprobe’s output.