The Debugger Visualizer Item Template in VS2008 is Broken
Someone at Microsoft forgot to update the Debugger Visualizer item template for Visual Studio 2008. Suppose that you add a visualizer to your project by selecting the "Debugger Visualizer" item template, as seen below:

The template automatically adds a bunch of boilerplate code so that you can easily get started with writing your own visualizer. That's all well and good, but when you actually give your shiny new visualizer a test drive, you will be told that the CLR was "Unable to cast object of type 'Your Visualizer Type' to type 'Microsoft.VisualStudio.DebuggerVisualizers.DialogDebuggerVisualizer'."

You can avoid this problem in several ways. Naturally, you can take the high road and avoid using the item template in the first place. If you are like me, that is a bad option because, in general, automation is a good thing. Work smart, not hard!
You can do the "right" thing and actually fix the Debugger Visualizer item template itself. That would probably be an interesting adventure into the world of Visual Studio template files, and would forever solve this problem on your machine. But, if you are like me, this option sounds like too much work considering that you might never use the Debugger Visualizer item template again. It isn't broken if you don't use it, right? ;)
It turns out that we can spend just a few seconds to fix the problem at hand. Here's the problem: the Microsoft.VisualStudio.DebuggerVisualizers.dll assembly reference that is added to your project by the template is the wrong version. Observe:

Notice how the Version of the assembly is 8.0.0.0. That's the problem. That's the version of the assembly which shipped with VS2005. The Debugger Visualizer item template should have added Version 9.0.0.0 instead. All that you have to do is delete that assembly reference and add a new one, which references the correct version of the Microsoft.VisualStudio.DebuggerVisualizers.dll assembly. The screenshot below shows the proper assembly to reference:

After you add that assembly reference, rebuild the project, and redeploy your visualizer, the exception seen previously does not get thrown. Note, however, that if you intend on using your visualizer in both VS2005 and VS2008, you must have two separate builds since each version of Visual Studio requires a different version of that DebuggerVisualizers assembly to be referenced.