Ab3d.DXEngine Users Guide

• Articles
  • Introduction
  • QuickStart
  • Overview
  • Initialization
  • Using Ab3d.PowerToys
  • Tips and Tricks
  • Ab3d.DXEngine Deep Dive
  • Diagnostics
  • Debugging frame and shaders
  • System requirements
  • Deployment
  • Troubleshooting

 

Debugging DXEngine in Visual Studio Graphics debugger or NSight

To get basic insight of rendering performance use the Diagnostics Window or DXEngineSnoop tool as described in Diagnostics article.

To get a more detailed insight into rendering process, you can use Visual Studio Graphics debugger or NVIDIA NSight.

Shaders and effects source code

Before writing your own shaders and effects you may check the following samples that come with full shader and effect source code:
In Ab3d.DXEngine.Wpf.Samples project those samples are:
DXEngineAdvanced/CustomFogShaderEffectSample
DXEngineAdvanced/ShadedPointCloudSample.

There is also a Ab3d.DirectX.ShaderFactory project where you can dynamically edit the shader's HLSL code and immediately see the changed shader in action.

But to get an even better understanding of how to build your own shader and effects, it is recommended to purchase the "Effects source code" (see Purchase web page). This will give you full source code of all the shaders (HLSL) and effects in the DXEngine. This will provide you will many building blocks that you can reuse for your own shaders.

Creating DirectX 11 debug device

To enable a graphics debugging application to attach to your application, the DirectX 11 device that is used by DXEngine need to be created with a debug flag. This can be done by setting the CreateDebugDirectXDevice static field to true (before the DXEngine is initialized):

Ab3d.DirectX.DXDiagnostics.CreateDebugDirectXDevice = true;

Capturing a frame

After the DirectX device is created with a debug flag, then it is recommended to use DirectXOverlay as PresentationType on the DXViewportView. This will use a SwapChain in the DXEngine and NSight or Visual Studio Graphics debugger will be immediately able to "attach" to the SwapChain and you will be able to capture individual frames.

If you want to use DirectXImage, then the capture cannot be done from NSight or Visual Studio Graphics debugger because they will not be able to see the SwapChain. But you can still capture the image from those two applications. First start any of those two debugging application with your application attached. Then you will need to capture the frame from your application. This can be done by calling the following method:

Ab3d.DirectX.DXDiagnostics.CaptureNextFrame(_dxView.DXScene);

This method can be called from Visual Studio Immediate Window.

An easier way to call this method is to use DXEngineSnoop tool (more about using DXEngineSnoop tool), attach to your application and then from the menu select "Capture next frame in VS".

After capturing a frame, you will be able to analyze it and see many details about how the frame is rendered. But to debug the shaders, they will need to be compiled with debug information.

Debugging HLSL shaders

To debug the shaders in HLSL you will need to compile the shaders with debug information.

When the shaders are compiled by fxc.exe tool, then add the following additional parameters: "/Zi /Od /Gfp":
/Zi - enable debugging information
/Od - disable optimizations
/Gfp - prefer flow control constructs

You will not be able to debug the shaders that come with DXEngine because they are not compiled with debug information. But with "Effects source code" you can compile the DXEngine shaders with debug information and then debug them.

In this case you will need to make sure that DXEngine will load the shaders with debug information instead of shaders that are included in the DXEngine.dll. To do that, you can pack the compiled shaders in another dll as EmbeddedResources and then call the RegisterShaderResourceStatic static method on EffectsManager to load the shaders:

var resourceAssembly = this.GetType().Assembly;
var assemblyShaderBytecodeProvider = new AssemblyShaderBytecodeProvider(resourceAssembly, resourceAssembly.GetName().Name + ".Resources.Shaders.");

EffectsManager.RegisterShaderResourceStatic(assemblyShaderBytecodeProvider);

See the following help file to see other options on how you can provide your own shaders to the DXEngine: ShaderBytecodeProvider help page.