A journey from Windows-only to cross-platform: Porting my Vulkan renderer.

Previously, I successfully got my custom renderer running on Linux, pushing my cross-platform experience to the next level.

All major features are working as expected, including mesh shaders and ray tracing.

Why this matters to me:
My renderer originally started as a Windows-focused project (Win32 + DirectXMath), only the Vulkan API part can be re-used. So porting it required significant refactoring.

During the process, I have to design proper platform abstraction, decouple engine and platform/input layers, deal with other platform-specific issues…etc.

This practice is absolutely meaningful and worthwhile. And I'm happy to share the problems I dealt with and the lessons learned.

Stage1 - WSL2 to fix the code compilation

As like any other engineers who never did programming on Linux, I also started my journey with the Windows Subsystem for Linux.

It is not perfect, but good enough for some quick compiling tests as a long-term Windows user.

In this stage, I was mostly doing code fixes for Linux that including:

• Decoupled clients and platform layers. Both Windows/Linux have different ways to render a window and handle inputs. (Win32 vs GLFW)
  They just can't use the same logic.
  
• Removed Windows-only types and customized those types for cross-platform use. Such as RECT, HWND…etc.
  My game timer was also high performance counter based before. I replaced that by the chrono timer instead.
  
• Replaced DirectXMath library by OpenGL math library (GLM). This is especially painful for me lol as I previously just spam DirectXMath types everywhere.
  Thankfully GLM can be run on Windows as well. I made sure my transformation functions or other math functions are still working as intended.
  
• Replaced memcpy_s by memcpy. I thought it would be a good practice to have the _s function, but then I realized it would not always be supported.
  I followed the GCC compile hint and re-wrote those to memcpy(). To avoid incorrect copies? Just insert some assertions to catch.
  
• Standardize all std::math usages. For example use std::round instead of using std::roundf. As I spotted a few warnings from GCC.
  
• Avoided wide char hard-code for Linux.

At this stage, I made the code be compiled and generate an executable by CMake successfully.

But another boss just entered the room as well - Runtime issues.

Stage 2 - Dual boot Ubuntu to fix runtime issues

After all compiling issues were solved, I was really excited to see how it would go in runtime.

But I expected some issues that prevented me from carrying on and here we go!

In this stage, I was focusing on runtime issues fixups that including:

• Graphics card driver (NV) doesn't work properly for WSL2. Certain posts on the NVIDIA Developer Forums suggested that it is possible to run and make Vulkan detect my card.
  But those posts might be outdated and I've really tried those methods. The WSL2 would still only show the Mesa (software simulated GPU).
  So I eventually set up a dual-boot environment and carried on my research. At this point it also makes sense to test on a real Linux environment.
  
• Implemented Linux-specific swapchain initialization. Obviously I couldn't re-use the Win32 surface eh?
  
• Normalized file path separator usage. Always use "/" instead of "\\" for cross platform purposes.
  At the beginning this affected my scene loading.

At this stage, I finally saw my scene was rendered! That was a big wow moment.

But then the final boss just entered the chat - Black objects and crashes when I resize my window.

Stage 3 - Important fixes for runtime rendering

Despite the renderer starts working, all objects in the scene are black! What happened exactly?

I firstly noticed only material-based objects were affected, because skybox was still working as well as other engine passes.

Then I checked PSO, shader loading, and rendering passes - still all good! Finally, I had a look into my material data and spotted incorrect data:

// ...... other data reading, skipped for keeping context short ..... //

FileIn.read(reinterpret_cast<char*>(&CullMode), sizeof(CullMode));
FileIn.read(reinterpret_cast<char*>(&BlendMode), sizeof(BlendMode));
FileIn.read(reinterpret_cast<char*>(&CutoffValue), sizeof(CutoffValue));
FileIn.read(reinterpret_cast<char*>(&MaxReflectionBounce), sizeof(MaxReflectionBounce));

// material graph data
UHUtilities::ReadStringVectorData(FileIn, RegisteredTextureNames);
ImportGraphData(FileIn);

// fix up RegisteredTextureNames path
for (std::string& TexName : RegisteredTextureNames)
{
	TexName = UHUtilities::StringReplace(TexName, "\\", GPathSeparator);
}

// material constant data
if (Version >= UH_ENUM_VALUE(UHMaterialVersion::GoingBindless))
{
	FileIn.read(reinterpret_cast<char*>(&MaterialBufferSize), sizeof(MaterialBufferSize));
}

FileIn.close();

Turns out my MaterialBufferSize remained 0 and affected the runtime data copying! No property will be copied due to materials did not create any valid data buffer at all 😅.

And I learned this was a file serialization issue, I used some types that are only safe for Windows, but not for Linux!

So I did another refactoring for my data, that including:

• Specify an exact size for every enum class. For example:

  enum class UHTextureType : uint32_t

  enum class size could be interpreted differently on different platforms.

• Avoid using size_t, bool, long, and raw structures directly when doing classic reinterpret_cast<> I/O. Prefer fixed-size types such as uint8_t, uint64_t for cross-platform safety.
  When outputting a structure, do not reinterpret the whole structure also. As they might have their own padding rules.
  Instead, output individual members from the structures.
  
• Resaved all my assets after the data types changed.
  

After these fixes, I have finally made it render as like my very top screenshot shows!

The whole process took me 18 days to finish, not too crazy and I learned a lot from this practice.

Other misc fixes:

• Crash when resizing GLFW. Due to API design, it does not have similar functionalities as ‘WM_SIZE’. I simply refactored my workflow to adapt that and fixed the issue.
  
• Stupid thread issue fix. I spotted a Vulkan validation layer error only on Linux. Turns out my termination order was wrong. I did something like: WaitGPU() call before Thread→WaitTask() call.
  Moving GPU waiting after the thread waiting solved the issue. Not sure why I didn't notice this before haha.

Summary

If you are also a passionate graphics/rendering engineer who enjoy building your own renderer.

I highly recommend giving cross-platform practice a try.

You might learn something you've never thought before! As different platforms could have different constraints for rendering.

Such kind of practice will also encourage you to do more high-level thinking, not just as simple as “I implemented feature X and Y”.