Looking for Feedback on a Game Engine Design.

I don't think such a thing is even technically possible unless you limit yourself to very basic games (e.g. text-only in a terminal console). There are too many fundamental differences between platforms once you consider graphics, audio, input, etc. The limitations of Cosmopolitan C are there for a good reason. Furthermore it's not likely to be a useful feature anyway. Re-compiling for a different platform is trivial, assuming the code to interface with that platform exists. A real-world multi-platform engine usually has to rebuild the entire game including assets for a new platform anyway, since there can be differences (e.g. in compressed texture or audio format).

@Aressera Currently I've manually constructed VERY basic games with the format and prototypes of the engine, but I do believe that it would be possible. Something like SDL or SFML compiled each time for every platform and stored in the executable (Though this would be quite cumbersome and bloat size of the executable, so maybe stored in a dynamically linked library to the side?). I understand the limitations of Cosmopolitan C are there because creating very high level cross-platform apps is unimaginably difficult, but I'm willing to work at it over years if it actually has a practical use in the community (The very reason I posted this, there's a good chance that re-compiling and distributing multiple executables only peeves me).

CakeQuester said:
Something like SDL or SFML compiled each time for every platform and stored in the executable

Does it really make sense to distribute to the end user all the binaries for all possible platforms? No. At the very least, such a thing should be part of the distribution system (e.g. Steam) or installer, but not part of the final executable. Having the binary for platform A on platform B is just a waste of space and adds needless complexity for no benefit.

CakeQuester said:
so maybe stored in a dynamically linked library to the side

Different platforms load DLLs in different ways. You won't be able to use the OS-specific loading functions.

Why not just take the route Unity takes? They have precompiled executables for all build platforms available and just pick the right one for the current platform and package it with the assets. Then you just distribute N different archives for N different platforms. Easy. There is no advantage to what you are proposing compared to Unity's approach, and many disadvantages.

CakeQuester said:
would a game engine like this be useful to developers here - one that compiles once and runs on most platforms out of the box, without requiring users to download extra runtimes such as .NET or Java?

Not really, that's not normally a pain point.

Developing all the game mechanics into code, converting all the ideas into art and animation assets, and creating levels, user interfaces, and similar, those take time and a lot of complexity. But managing builds? That's easy.

On all the cross-platform games over the years, some spanning 7 systems at once, getting all the builds put together was barely a blip on the radar. The team could have 50 or 100 developers building the game, and yet just one build engineer who is only occasionally contributing to make sure the automated builds are flowing smoothly.

Existing game engines like Unreal also make it trivially easy. In general you can use exactly the same scripts to build the game, just swap the platform out with -platform= with whatever you need. Windows, Linux, MacOS, XSX, XSS, XBOne, PS5, PS4, Switch, Switch2, Android, iOS, SteamVR, SteamDeck, It's also just a simple dropdown in the interface if you want to build it in the editor. This isn't done by building a universal compiler language, but by actually compiling it with libraries for each platform. Unity similarly supports all the major platforms. A few build servers running continuous intgration do it all. Hobby developers don't usually have the resources to finish a game on a single platform, let alone cross platform unless they are using an engine like that which does the work for you.

It doesn't really matter if it's a team of 3 or a team of 300, the build machines automatically build every platform after each version control submission, and churn out packages for every platform so QA can hammer on whatever latest build there is.

A unified binary doesn't really work. Instead of allowing the systems to be customized on every operating system with system-specific libraries and an abstract unified interface, which takes advantage of every platform's differences, a unified binary forces all the systems into the same unified box which tends to limit them to the common functionality. Where different libraries allows expansion and growth, unified executables constrain and limit.

Java tried it nearly 30 years ago with a promise of “write once, run anywhere”. It was pushed hard by mobile phones and the running joke was “write once, debug everywhere”. J2ME is still around but is mostly used when forced to use it, not as a willing choice. The dotnet runtime has tried it, but in practice it usually works on Windows but is hit-and-miss on every other platform, an example, System.Drawing is mostly the Windows API wrappers and seemingly random if something is supported or not on Linux or Mac. GTK# and similar third-party libraries are better than the official libraries, but tend to be more complex and also have plenty of bugs when crossing platforms. Groups build all kinds of compatibility layers and shims, but even then in short order you're no longer trying to build for a single unified interface but instead building to not just the unified language but also to the assorted shims, abstractions, and compatibility libraries.

The typical path to engine, back before the big engines showed up 20 years or so ago, was to build a series of games. The first game was you cobbling together your own library and gameplay loop. The second game you stripped out all the stuff specific to your game and made your second game. The third game you stripped it out again, abstracted what you needed, and again refined it into common parts. About the time the studio reached a fourth game the collection of tools, libraries, and interfaces was mature enough it could be considered an engine. These days engines are treated as middleware, and you can download Unreal that has about a million work-years of engineering inside it from 35 years of development and leveraged contributions from thousands of studios and from many purchased businesses and technologies.

You an certainly build something like that if that's what you want to do. There is learning that can be done. Just keep in mind that well-funded teams from Microsoft and Oracle staffed with armies of PhD's and industry veterans still haven't managed to do it well despite decades of trying with Java and dotnet. Before them it was the best minds at IBM and Bell Labs and other companies, which in part is where we get the C language family from in the 1970s, as well as modern operating systems with a POSIX (Portable Operating System Interface) that Unix variants generally follow, all were attempts to make a unified, “run everywhere” system. Commercial success isn't likely, but there is lots of learning you can do.

Sounds like you're reinventing the “fat binary” idea that Apple used back in the day with PPC/Intel universal apps. It worked fine then, but the tradeoff was larger executables and some weird edge cases.

My only concern would be how you’d handle stuff like graphics APIs - DirectX vs Vulkan vs Metal. Do you see your engine abstracting all of that under one umbrella too, or mainly focusing on the executable/runtime part?

Somewhat unrelated, but I'll share my thoughts on all that, for my own project: my own programming language incorporates all multimedia functionality I need right into the language runtime. It doesn't support hardware-accelerated graphics, though. I plan to make it run everything as full recompiles from source (no incremental building or .exe shipping). All I need then is to ship the language runtime ONCE. As long as the language doesn't update, it remains static. This only works if compile times are at 100k LoC/s or more, ideally more like 500k or 1M. I'm actually working on a linux live distro that only ships my compiler and nothing else. You just boot from a stick, and then all software you want to run, you run from source code.

That way, I don't need to care about any weird stuff. The language runtime ships with OpenAL, and maybe SDL or something for window creation. And since you can painlessly boot into it from a stick, I actually don't even have to ship the language to other OSes anymore, although I can still do that easily. Different CPU ISAs will still be troublesome because they will require another backend for the compiler.

@epiccc Eventually I would love to abstract all of the graphics APIs under one umbrella, but I'm not sure I could do that by myself, even in the many years I have to work on this project. The executable/runtime itself would be hellish to implement and probably take years, but a unified graphics API on top of that? If I'm being honest, the engine is mostly just a though experiment, but I posted here to get feedback and see if others thought it would be feasible, even if it took a million years to implement. From this post, I am now coming to terms with the fact that it seems I'm part of a very small subset of developers who hate having to distribute different executables for different platforms. Furthermore, even if I ever managed to finish the engine and format, it would probably be very difficult to maintain unless I tweaked it very significantly (Which I am open to doing, the current idea for the format is very rough, even though I have been working at it for several months. Maybe something like a pre-packaged JIT while the user executable is in a bytecode? That could keep code sizes down. Though, I'm just throwing things at the wall at this point).

@frob Thank you for the feedback - you're absolutely right. My solution to the minor inconvenience of recompiling and re-distributing for each platform is probably clumsy and suffers from scope creep (As my projects often do). Furthermore, you raise a valid point: countless smarter and better-funded people have attempted similar solutions and failed, so it's fair to ask why I'd succeed where they didn’t. Even so, I believe this project has potential - though almost certainly in a more refined form. That said, commercial success isn’t really my goal. I plan to release most of the engine as open-source and free; what matters to me is that it doesn’t end up as a complete failure. I’d simply like it to be known and (possibly) used by more than just myself someday.

CakeQuester said:
That said, commercial success isn’t really my goal. I plan to release most of the engine as open-source and free; what matters to me is that it doesn’t end up as a complete failure. I’d simply like it to be known and (possibly) used by more than just myself someday.

I don't think that's a healthy mindset. Nobody will use it, realistically. Anyone looking to get anything done will use Raylib, Godot, Unity, Unreal, or roll his own. Realistically, you will always be the only one to ever use your own stuff. Publishing your works is ok, but don't do it with the goal of actually being seen or having anyone appreciate it. It will cause huge disappointment later on, and lots of stress during times when things aren't going great. I don't recommend doing any hobby project with any other motivation than enjoying the project for what it is, doing it for the sake of doing it. Go on Github and look for hobbyist game engine repositories.

I think we all probably started out programming because we wanted to make games eventually, and we sticked with it because it's so fun. Never did we care about being seen or popular or even acknowledged. Just you and the computer, programming. As soon as secondary concerns enter into the equation, the innocent joy gets lost. It can take years to regain that once you've lost it.