Multithreaded Game Engine Architecture With Data Oriented Design

https://www.unrealengine.com/what-is-unreal-engine-4

Yeah, unfortunately UE4 pretty much fails at points 1 and 2 - horrible objects are horrible for cache.
(To be fair, the engine is pushing 20 years old so this isn't really a surprise, even Unity which is younger suffers from the same problem.)

How would I do it?
Nothing like how UE4 does...

Since the topic is so complex, I'll just put some very high-level notes while not even trying to formulate a complete answer to the actual question.

1) For cache-friendliness, try to ensure each operation's required memory is in a continuous memory region, and preferably accessed linearly from start to end. For example updating a particle system or an animated skeleton. Avoid "jumping" from heap object to another through pointers. A well-structured entity-component system is possibly a good match here, at least if the systems don't need accessing each other's data a lot.

2) A typical approach nowadays is to structure the execution of a frame into a task graph, and the tasks are executed on any available cores by worker threads. Data from each task or operation flows to the next, for example physics simulation & animation together produce the final positions of game objects, which go to culling, from which a visible object list goes to render command generation, which is finally submitted to the graphics API. When the culling / render stage for the current frame is running, you can already be calculating the logic for the next frame.

For both points 1 & 2 it's probably best to keep scripting for low-volume, high-level operations (e.g. ask pathfinding to start moving this object toward position x) or configuration (stats of game objects, list of render passes and postprocess effects for the actual rendering code). For such infrequent use even a singlethreaded script VM could be fine.

Here's Naughty Dog talking about their engine's multithreading & memory allocation: http://www.gdcvault.com/play/1022186/Parallelizing-the-Naughty-Dog-Engine

Also old Bitsquid development blog entries may be interesting reading: http://bitsquid.blogspot.com/

This thread discuss's some of the cache issues you'll face. http://www.gamedev.net/topic/680410-basic-multithreaded-question/

Yeah, unfortunately UE4 pretty much fails at points 1 and 2 - horrible objects are horrible for cache.
(To be fair, the engine is pushing 20 years old so this isn't really a surprise, even Unity which is younger suffers from the same problem.)

How would I do it?
Nothing like how UE4 does...

I was considering that cache sensitive sections could be written in C++, but I take your point. OTOH it does seem to utilize all my cores pretty evenly.

This isn't a language thing; you can still write C++ code which fucks with the cache (and most likely will, see Mike Acton's Typical C++ Bullshit talk/rant/thing) and when code is 10+ years old then cache fuckery is basically a given.

On cores; utilizing cores and ultilizing cores effectively are two different things.

10+ year old legacy infused systems are just not good examples of doing things well today.
(In other news, sky is blue, water is wet, turns out the grass on the other side really isn't greener...)

My understanding is that UE4 may have ideas from 20 year old code but it is not simply the next release of UE3 - it was being developed in parallel for some time before the switch. So criticism should be more of the concepts than of it being legacy code as such.

Regardless, an engine is only as good as the games you make out of it. A lot of the ideas I see thrown around for cunning multithreading involving messages and double-buffered objects and heavily decoupled component based systems sound like they'd be a nightmare to actually make a game with - which is why I'm glad nowhere I've worked has gone down that path, higher performance or not. :)

Bit of UE4 changed, sure, and are still in the process of changing... but the vast majority of it was the same, with horrible memory layouts and huge classes which make efficient work/cache usage a nightmare.

(Not directed at yourself, but I always laugh at the oft repeated meme that UE4 was a 'rewrite' - some clever PR spin on that one... the biggest 'rewrite' was probably the blueprints stuff... much else was the same, maybe cleaned up a bit.)

Other good references:

https://blog.molecular-matters.com/2014/11/06/stateless-layered-multi-threaded-rendering-part-1/

https://www.cg.tuwien.ac.at/research/publications/2007/bauchinger-2007-mre/bauchinger-2007-mre-Thesis.pdf

http://jendrikillner.bitbucket.org/blog/blog/stateless_rendering/

-Graham

Some more references links:

http://www.gamedevpensieve.com/hardware/cpu/cache
http://www.gamedevpensieve.com/programming/paradigm/data-oriented-design

http://www.gamedevpensieve.com/programming/threading

Thank you all!

I decided to switch to classic ECS.

Data Oriented Design is very cool :)

You cannot achieve an optimum in those regards with a game engine.

You would have to create custom code and completely tailor the content to a single target platform.

(This is what we do for our automotive customers for our ARM/PowerVR cores.)

If you want to start talking about compromises to support a wide breadth of platforms then I'd suggest reading the latest revision of David Eberly's 3D Game Engine Design to get a basis for the tasks necessary. His engine is designed to be easy to use and explain not necessarily wring out the CPU & GPU for max performance.

Next would be understanding a modern commercial engine; I'd pick the fibre optimizations in the Naughty Dog engine an example to get started today.

Next would be understanding what the GPU hardware bottlenecks are and where the tech is going for the next generation.

At the moment, for most markets, this would mean understanding the design of Vulkan (unless you're targeting the XBox then DirectX 12).

I think it's safe to assume that DirectX 12 will also achieve max performance on Windows - Microsoft will ensure it - but I don't think Vulkan will be far off from it.

That would be an example of compromise you would make if you wanted to target multiple platforms (use Vulkan instead of DirectX 12).

If you are looking for the best-of-the-best then it's Vulkan running on Linux with a nVidia GPU; that's the platform that puts the most raw power in your hands with the least overhead without creating something custom.

Currently, no title pushes the limits of what can be done.

We're starting to run into a sort of Heisenberg problem where the development cycle is so long for such a title that by the time it's released new hardware is available.