Cg Programming/Unity/Debugging Shaders with Visual Studio

This tutorial discusses how to debug Unity shaders with Microsoft Visual Studio.

Preparations

• Update your software:
  • Download and install the DirectX SDK (June 2010).
  • Install the latest Windows Update.
  • Install Visual Studio 2015 (or higher) with the latest update. This is important because the updates often fix graphics debugging related bugs.
  • Install Visual C++ for VS2015. Note that in Visual Studio 2015, Visual C++ is not installed by default. When installing, be sure to choose Custom installation and then choose the C++ components you require. Or, if Visual Studio is already installed, choose File | New | Project | C++ and you will be prompted to install the necessary components, or run the installer again. More information:

http://blogs.msdn.com/b/vcblog/arch...isual-studio-2015-affecting-c-developers.aspx

Example

• Create a new project in Unity.
• From the main menu select Create > 3D Object > Cube.
• In the Project window, right click and select Create->Shader->Standard Surface Shader. Leave the name as NewSurfaceShader.
• Open the NewSurfaceShader.shader file and add this line after "CGPROGRAM":

#pragma enable_d3d11_debug_symbols

• In the Project View, select Create > Material.
• Select the new material and go to Inspector->Shader->Select Custom/NewSurfaceShader.
• Drag the new material onto the cube.
• Go to File->Build Settings->Select PC Standalone, then click Switch Platform (if not already).
• From the main menu, select Edit > Project Settings > Quality. In the Inspector View disable anti aliasing for all quality levels.
• Save the project.
• From the main menu, select File > Build Settings... Tick the box "Development Build" and "Script Debugging".
• From the main menu, select Edit > Project Settings > Player. In the Inspector View select Resolution and Presentation > Standalone Player Options > Display Resolution Dialog > Disabled.
• Select Player Settings->Inspector->Other Settings. Untick "Auto Graphics API for Windows" and check that Direct3D11 is at the top of the list. Also check if the Unity Editor title bar states "<DX11>".
• From the main menu, select File > Build & Run, name the file test.exe, and click Save. Note the path to the executable.
• Create a new project in Visual Studio 2015.

File->New->Project->Installed->Templates->Other Languages->Visual C++->Empty Project. Note: if Visual C++ is not available, follow the instructions at the "Update Software" section above.

• Select a name and location for the project and click Ok.
• Go to View > Solution Explorer. Select the project (the project, not the solution).
• Go to Project > Properties > Configuration Properties > Debugging.

Note: if Debugging is not available in the Configuration Properties, you selected the solution instead of the project in the Solution Explorer, or you created a C# instead of a C++ project.

• In the Command field, replace "$(TargetPath)" with the path to the Windows executable you built earlier.
• In the Command Arguments field, add "-force-d3d11".
• Click OK.
• Select Build > Build Solution.
• Go to Debug > Graphics > Start Diagnostics. Install the Graphics Tools if a popup shows up.
• The standalone build should fire up and this message should appear in the top left: "Frames captured: 0. Use Print Screen key to capture a frame".
• Press the Print Screen key.
• Alt-Tab out of the standalone build and go to Visual Studio. Note: to close the Unity test app, select ALT-F4.
• You should see a frame capture in Visual Studio. Double your captured frame and a new Visual Studio window should open.
• On the new Visual Studio window, Place the crosshair on the pixel of interest and left click. You can zoom in and out of the frame using the mouse scroll wheel.
• Go to the Graphics Event List window, select the first (top) occurrence of "obj:x DrawIndexed".

Note: If you get a "No source available" warning later on, try one of the entries in the Camera.Render tree instead.

• Go to Graphics Pixel History window > obj:x DrawIndexed > Triangle > Vertex Shader > click the green arrow next to it.
• Now you can single step through your shader.

Troubleshooting

• If it displays the message "Source not available", then either:
  • You didn't add this line to the shader: #pragma enable_d3d11_debug_symbols
  • You didn't install the latest Visual Studio update.
  • You selected the wrong item in the Graphics Event List window. If the DrawIndexed item does not work, try one of the items in the Camera.Render tree.
  • Many variables display a NaN value due to a bug in Visual Studio. Variables only show the correct value if the line of code has just been executed. If you want to view the value of variables elsewhere in the shader, you have to step over them and note them down manually.
  • If the game is not displayed correctly after pressing Debug > Graphics > Start Diagnostics, then force use the integrated graphics instead of the high performance video card.
  • If Visual C++ is not available, follow the instructions at the "Update Software" section above

Further reading

• https://msdn.microsoft.com/en-us/library/hh972447.aspx

< Cg Programming/Unity

Unless stated otherwise, all example source code on this page is granted to the public domain.