Rate my engine!

Quote:

Original post by RuneLancer In short, my engine is as a singleton.

That's going to cause some discussion. Long story short, I think that's a serious mistake.

Quote:

Among its members, it contains a vector used to store scene data. Typically when rendering a frame, my game clears this vector. Each interface (for example, the window/font manager) then shoots data into it. Typically they cumulate all of their data into an array or vector and shoot it into the engine in one operation. Once the vector has been filled with the render data, I call my engine's render method which renders the data all at once.

Why is this data stored in the engine class? Wouldn't it make a lot more sense for each subsystem to pass its data in?

Quote:

Ideally I want to abstract everything graphic from the rest of my game. I don't want my game to work with vertices at all, other than generating the data. That way I'd be able to write a different graphic engine and support, say, OpenGL at a later date if I wanted to, or reuse the engine elsewhere with little to no tweaking. The game is just responsible for generating the data - the graphic engine is responsible for doing stuff with it.

That's all well and good, but there's a lot of design subtleties involved to be careful of in doing that. YOu should do it; just don't expect it to be easy.

Quote:

Also, I'm a little uneasy with refilling the vector every frame. It seems like I could benefit from having some sort of dirty region to identify new data so I don't have to regenerate a large amount of vertices that remain the same from frame to frame (ie, map data.)

Maybe. This vector itself seems sketchy to me. What, precisely, is in it?

Quote:

In short, my engine is as a singleton.

Unnecessary and crippling.

Quote:

Among its members, it contains a vector used to store scene data. Typically when rendering a frame, my game clears this vector. Each interface (for example, the window/font manager) then shoots data into it. Typically they cumulate all of their data into an array or vector and shoot it into the engine in one operation. Once the vector has been filled with the render data, I call my engine's render method which renders the data all at once.

This sounds like you have a single vector containing a whole bunch of geometry that is acquired from various sources (the game, for example), and then rendered in one go. This is very, very, limiting and will not scale well at all.

I'm reluctant to continue with more critique/advice/discussion until I can see more of this "engine." Can you describe the architecture in more depth, and/or provide some sample code?

I'm curious how making it a singleton is unecessary and crippling. The engine sits next to the game - I'm never going to create any extra copies of it. Alternatively I could just instantiate it once and pass a pointer around, but I don't see what this adds, in this particular situation. I'm not criticizing, I'm asking - I'm curious why GD.net seems so anti-singleton. ;)

@Promit:
I see what you mean about passing the data in. Correct me if I'm wrong, but you're suggesting I don't cumulate the data into my engine, and essentially render the geometry as I need it? (ex, once the window subsystem has generated the geometry for the windows and text, they'd all be sent into the engine to be rendered immediately.)

I'm not opposed to keeping the geometry in the subsystems (technically I'd have to anyhow :P) What I want mainly is to have all of the management be specific to the graphic engine. Occlusion tests, for instance, sorting, all that. Basically I don't want to end up rewriting the same code for each subsystem before passing the data off to the engine.

The vector contains the vertices to render (position, color, etc, etc..) It's basically passed off to DirectX directly during the render phase in one go, and assumed to have all necessary optimisations/tests performed on it beforehand. Ie, it's the final list of vertices to render.


@jpetrie:
Like I said, this is a draft. I don't have any code to show - this is the planning phase. I don't want to start writing code if the very idea behind it is flawed. Hence why I'm asking for input over this. I don't expect to get everything right the first time, but I'd rather not waste a few weeks writing the engine if it turns out the entire idea is flawed and will have to be scrapped. :P

I'm going to move this to 'Game Programming' (although it could possibly go in 'Software Engineering' or 'General Programming' [oh]) as it doesn't seem like you're focussing on the DirectX-specific side of architecture/design.

Quote:

I'm curious why GD.net seems so anti-singleton. ;)

There is a bit of history behind it. It seems that despite the many discussions about the design pattern people still keep on using it [smile]

April 2005 was a good month for Washu. It's well worth reading the few entries he's put out on the subject if you want to know why you should be worried about your 'singletonitis'...

hth
Jack

Quote:

Like I said, this is a draft. I don't have any code to show - this is the planning phase. I don't want to start writing code if the very idea behind it is flawed. Hence why I'm asking for input over this. I don't expect to get everything right the first time, but I'd rather not waste a few weeks writing the engine if it turns out the entire idea is flawed and will have to be scrapped. :P

Good idea. Here's another thread where I commented on some dangers of singletons. The same points apply, basically: you say you only plan on having one, and that's fine, create only one. Don't cripple yourself by placing an arbitrary restriction on your design so early. Singletons also are easy to use as crutches and band-aids for bad design -- you talk about needing to "passing a pointer around," which is what you should do, but only when you really need it. Since singletons allow global access, they trick you into believing you need that global access. You don't. In practice, the number of instances where you must pass things around are relatively small (if your design is good).

Isn't the OGRE engine littered with singletons? If singletons are so deadly (which I completely disagree on) why does a large popular open source engine use them so much??

I use singletons in engine design when there should be only one. Where having two would screw things up, or you simply shouldn't need more than one. Most of the time its better to have global accessibility as well. Having to go Engine::GetInst ().GetFileManager () is worse than FileManager::GetInst ()....

I'm currently designing and creating an engine that'll be used for DirectX10, and have done a lot of the base classes such as template helpers, math classes, file classes etc and I've got a few singletons.

* FileSystem (keeps a list of archive codecs, loaded archives etc)
* ThreadManager (handles creation of named Win32 threads)
* InputManager (handles DirectInput enumeration etc)
* PluginSystem (loads and unloads plugin dlls)

- Renderer and Engine will be singletons too.

(my Log class is just static member functions, no need for a singleton there)

Quote:

Isn't the OGRE engine littered with singletons? If singletons are so deadly (which I completely disagree on) why does a large popular open source engine use them so much??

Yes, and that's one of it's huge weak points. Being popular and open-source says nothing about whether or not it is well designed.

Quote:

Most of the time its better to have global accessibility as well.

That's amusing. Wrong, but amusing. Having global accessibility is usually a crutch; if its not there because you already have an unstructured and tightly-bound design, it will allow you (without much thought; accidentally) to create an unstructured and tightly-bound mess.

Quote:

I use singletons in engine design when there should be only one. Where having two would screw things up, or you simply shouldn't need more than one.

Then you're misusing them, because the second point ("or you simply shouldn't need more than one") means you simply should not create more than one, and the first point is rarely if ever true in game development (if you think its true, that's a sign that your design and/or implementation is broken, as I touch upon in the thread I linked).

Quote:

* FileSystem (keeps a list of archive codecs, loaded archives etc) * ThreadManager (handles creation of named Win32 threads) * InputManager (handles DirectInput enumeration etc) * PluginSystem (loads and unloads plugin dlls) - Renderer and Engine will be singletons too.

None of those need to be singletons, consequently, none of them should be.

Hrm, so what's the point of singletons then? If they're such a crutch, why even use them in the first place? Personally I've never run into problems or even found them to be a restriction at all, but then again programming styles vary from person to person. I just don't see how it's anything more than a matter of style (assuming you're not actually crippling your project through outright bad design, but that goes without saying...)

Anyway, ignoring the singleton issue, how would the data in a graphic engine be manipulated? Should I shoot the data for each subsystem into my engine once it's ready, handle the engine-specific manipulations I want my vertices to go through, then render them? (Which would mean going through these steps a handfull of time each frame - nothing major peformance-wise I'd guess, just curious design-wise how not doing this in one step sounds.) From the comments I've gotten (and after a bit of consideration) cumulating everything into a vector doesn't sound like a great idea.

I think that'd be all I'd need - some means of managing and rendering the vertices my other subsystems produce. I can add other things as I need them in the engine afterwards (lighting, fog, shaders) and not have to worry (much; maybe a bit when I'll have to add normals for lighting ;) ) about my vertices anymore once that part's been handled...

Thanks for the input so far - I'm not expecting to write a perfect system the first try (let's face it, nobody should ;) ) but having the more important parts of the engine built around something solid and functional will be a great start.

Singletons are only truly justified when (a) more than one instance of a class causes incorrect behavior and (b) when absolute global access to the object is required.

On the rare occasion when (a) is true in game development, (b) must still be satisfied. The correct use of singletons in game development is excedingly rare for that reason. The only system I've found to be justifiably argued (though not strictly required) as a singleton is as the basis for a logging class. The use of a singleton here is justified in that the file access should be centrally managed so that another instance of the logger doesn't attempt to access the same file (more than one instance is an error) and so that every class has access to the logger to log its state and error codes (global access is required). Truth be told, the satisfaction of condition (b) is mearly a convenience issue, as a reference to the logging class could simply be passed to each class, function or method that needed logging facilities. However, IMHO, the inconvenience of doing so (especially since you'd likely want all or some of the logging conditionally compiled out for the release,) simply makes the singleton approach the more sound choice.

[Edited by - ravyne2001 on May 23, 2007 5:40:45 PM]

My viewpoint on this is:

Striving for a good design is fantastic, but sometimes brute force nasty methods are simple necessary to get the damn thing out of the door.

The last company I worked at used global variables which were intended to have only a single instance, littered all over the code, not even singletons. Very successful company, jobs were completed on time, games were shipped multi-platform, you've probably played some of them.

Practice & theory, often go hand in hand, sometimes do not. Finding the right balance is the real problem.

But wouldn't multiple instances of a class accessing a renderer cause a problem? Admittedly I've moved to DirectX not very long ago from OpenGL, and I'm a bit sketchy on the details. It seems to me like having multiple, different objects each generating a device wouldn't be such a good idea (like I said, I want to wrap up everything in this engine so I could swap it out for, say, an OpenGL version if I wanted - I can't set up Direct3D and keep all of its interfaces in my game as global variables.) Or maybe make these things static members of my engine could also work...

Either way, I'm more interested in the actual DirectX-related material and not the means by which it is designed - I can always alter the engine to be a bona-fide class instead of a singleton if it causes issues before I put the finishing touches on it and move on to integrating it in my game.

I'm still hoping to hear more about that aspect - the actual engine, not singletons vs. classes (though there's some good stuff being tossed around) - though. :)

Quote:

My viewpoint on this is: Striving for a good design is fantastic, but sometimes brute force nasty methods are simple necessary to get the damn thing out of the door. The last company I worked at used global variables which were intended to have only a single instance, littered all over the code, not even singletons. Very successful company, jobs were completed on time, games were shipped multi-platform, you've probably played some of them. Practice & theory, often go hand in hand, sometimes do not. Finding the right balance is the real problem.

The thing about singletons, though, is that they tend to come into play early on -- in the design phase or early implementation, before time crunch is a problem. They need to be killed off there, so that they don't propagate their design rot into the code for the lifetime of the project. Once that happens, refactoring them out can be unbearably painful and in almost all cases there will not be time or budget to do it properly, as you point out.

However the "balance" issue is not a valid argument against singletons, because at the beginning of a project you should be much more concerned with writing robust, maintainable and extensible code -- singletons are none of these and have no business being there. If you prevent their existence early on, you won't have as many problems with them later.

Quote:

But wouldn't multiple instances of a class accessing a renderer cause a problem?

Why would it? It's easy to build an implementation that does not have such issues, and its easy to see situations where it makes sense. Assume a renderer represents a fixed rendering destination; you might want multiple for multiple viewports. Assume a renderer represents an interface to a specific graphics card; some machines have multiple. Those are particularly naive examples, but they serve to illustrate the point.

Quote:

It seems to me like having multiple, different objects each generating a device wouldn't be such a good idea

Don't build a one-to-one correspondence between renderer and "device" then. It's easy enough to maintain a single device without restricting yourself to a single device, thus allowing you to create multiple devices (multimonitor support?) in the future.

Quote:

I'm still hoping to hear more about that aspect - the actual engine, not singletons vs. classes (though there's some good stuff being tossed around) - though. :)

This warrants its own post. Here's a quick off-the-top-of-my-head run-down of how the rendering system I'm currently building works, so far.

There's a collection of so-called "back end" objects, that implement compile-time-polymorphic interfaces to an underlying rendering API (this is so I can build versions of the rendering assembly -- I use C# for this -- for both D3D9 and D3D10; theoretically I could add OpenGL support, but there's little reason to). These objects, and what they map to in the D3D9 interface, are as follows:

GraphicsDevice  --> D3D9 deviceTextureResource --> D3D9 textureBufferResource  --> D3D9 vertex buffer, index bufferShaderProgram   --> D3D9 vertex shader, pixel shader

These are fairly thin wrappers. Also grouped into this logical collection of functionality are some enumerations for API-agnostic ways to specify primitive topology, shader program targets, buffer contents, and so on, and some routines to marshal between my API-agnostic values and the compile-time-selected underlying render API values. There's also some stuff for doing buffer and texture resource in-memory IO (for locking resources).

Abstracting the underlying rendering API began as a way to clean up the rest of my code and ensure I was always making use of underlying API objects in the same fashion (such as always passing the same creation flags to certain classes of vertex buffer, et cetera). It wasn't originally done with API portability in mind -- I don't really recommend that be something you consider as a major feature. So far, this seems to have worked out well as far as portability goes, but that's just a fluke. I didn't really design for it. When I get further along, I'll probably write up an article about how it worked out, since so many people seem infatuated with the albatross of API portability.

Anyway.

The rest of the renderer is built on top of that thin abstraction. The client interacts primarily with a Renderer object, which hold a GraphicsDevice internally. Multiple Renderer objects share the underlying device unless explicitly told to create their own (actually it's a bit more involved than that, but whatever). No requirement, restriction, or assumption of the existence of only one GraphicsDevice or Renderer at all.

Renderer aggregates a couple of other classes that the user tends to deal with. ResourcePool, through which the client requests the creation of new types of resources (geometry objects, 2D canvas objects, et cetera). RenderQueue, which is ultimately responsible for the bulk of the actual rendering work, and MethodMapper, which is used to construct an association between concrete renderable instances (RenderInstance) and objects that dictate the rendering process (RenderMethod) via a common interface (RenderDescription).

A given Renderer contains only one each of ResourcePool, RenderQueue and MethodMapper (and they are not creatable by client code), but they aren't (and can't be) singletons because each Renderer needs to have a unique instance of each.

Actual rendering works as follows. The RenderInstance object represents a specific instance of something that will be rendered. It contains a reference to the source object (for example, the chunk of geometry data, so you don't need to duplicate it), and some assorted per-instance data, like the world transformation matrix, and a RenderDescription. RenderDescription is a uniform means of specifying how the client would like an instance to be rendered (which effects to apply, do you want to render this instance with debug indicators, et cetera -- I'm still working on interesting ways to use this class as a means of controlling shader metageneration).

RenderInstance objects are inserted into the RenderQueue, which sorts them into appropriate buckets (solid, alpha, requires-shadow-pass, et cetera) based on the RenderDescription (which has a method of forcing specific and/or multiple buckets, if you like). To perform actual rendering, instances are associated with a particular RenderMethod (which implements both CPU-side processing -- setting shader parameters and maybe deforming geometry -- and GPU-side processing -- binding shaders and handing the geometry from the source object to the graphics card) via the MethodMapper, based on the contents of the RenderDescription.
When the queue is flushed, each method pushes its source object's geometry to the card to perform actual rendering.

There are two primary points of extensibility, so far. The actual sorting in the render queue into various render buckets can be customized by the client if they provide a class that subclasses the RenderBucket class. New means of rendering things can be implemented by provided subclasses of the RenderMethod class; for example a client might want to implement software skinning by implementing a class derived from RenderMethod that overloading the appropriate method that is called when the render method does its CPU-side geometry processing. The method has full access to the state of the geometry at that point, as well as at least read access to most of the other relevant portion of the render pipeline, so it can deform geometry according to whatever wave model it likes as a preprocess step prior to submitting the geometry to the graphics card.

So that's a brief (heh) overview. Some caveats:
- This is still a prototype, and might change quite a bit as I refactor it.
- This was designed primarily with extensibility of rendering methods in mind (in other words, its for prototyping graphics algorithms).

Quote:

Original post by jpetrie Quote: Isn't the OGRE engine littered with singletons? If singletons are so deadly (which I completely disagree on) why does a large popular open source engine use them so much?? Yes, and that's one of it's huge weak points. Being popular and open-source says nothing about whether or not it is well designed. Quote: Most of the time its better to have global accessibility as well. That's amusing. Wrong, but amusing. Having global accessibility is usually a crutch; if its not there because you already have an unstructured and tightly-bound design, it will allow you (without much thought; accidentally) to create an unstructured and tightly-bound mess. Quote: I use singletons in engine design when there should be only one. Where having two would screw things up, or you simply shouldn't need more than one. Then you're misusing them, because the second point ("or you simply shouldn't need more than one") means you simply should not create more than one, and the first point is rarely if ever true in game development (if you think its true, that's a sign that your design and/or implementation is broken, as I touch upon in the thread I linked). Quote: * FileSystem (keeps a list of archive codecs, loaded archives etc) * ThreadManager (handles creation of named Win32 threads) * InputManager (handles DirectInput enumeration etc) * PluginSystem (loads and unloads plugin dlls) - Renderer and Engine will be singletons too. None of those need to be singletons, consequently, none of them should be.

Lol its funny how theres all these tutorials on the net showing how u do singletons and why they're so great, and then theres these people who think that having a globally accessible, one-instance needed class instance made through object orientation is the devil. Isn't that entirely defeating the purpose of OO design? Yeh sure, its a 'global' but its a far cry from global variables like "extern Device * g_Device;"

I use the singleton design pattern because it allows a program with several DLL's linked in to access the SAME instance very easily and very straight forward.

Sure the Engine class will probably handle the actual creation and destruction of the sub-systems, but why does that mean they have to be annoying and indirectly accessible when they are used so much.

Like my above example: I'd rather code with "IO::GetInst ()" than "Engine::GetInst ().GetIOSystem ()".

Cool, so you didn't really understand anything I just said, then, did you?

Just because you build something with "object orientation" does not magically make it okay. You don't seem to understand much about what "OO design" actually is. It's not just about using "design patterns" and the "class" keyword. It's about building reusable, modular, extensible software components via a number of methodologies and design principals, all of which are pretty much completely violated by the use of singletons in just about every context.

Quote:

"IO::GetInst ()" than "Engine::GetInst ().GetIOSystem ()".

This, for example, is little more than syntactic sugar and illustrates rather well that you don't seem to have much of a clue what's really going on here.

Singletons came out of C, in my oppinion. Sometimes it is nice to not instantiate classes and use global functions. Sometimes it makes sense to use a single instance; for instance a single instance of a File System object, or a single instance of a Current User object. Sometimes, developers missuse them as static classes --which are good for global code containers.

Quote:

Original post by thre3dee Like my above example: I'd rather code with "IO::GetInst ()" than "Engine::GetInst ().GetIOSystem ()".

You are using a singleton in both of those, its just a matter of what you implemented as a singleton. So, basically, this example misses the point entirely.

[Edited by - Driv3MeFar on May 23, 2007 10:15:08 PM]

Seems the thread's turned into a singleton debate, but I think I got my answer so it's all good. :) Actually, it's fairly interesting.

One thing caught my interest though.

Quote:

Then you're misusing them, because the second point ("or you simply shouldn't need more than one") means you simply should not create more than one

Originally I was going to say, isn't that the whole point of a singleton? But as I was writing that I realized how unlikely that situation would be in a well-designed system. If your code is going to access something willy-nilly and you have no control or no idea over what it does to the point where you can't ensure only one instance of your class exists, your code is broken. And if your code is logical and well-designed, you shouldn't have to rely on a singleton for something like this after all.

Seems to me, though, it could still have some uses in enforcing the accesibility of data. Maybe something involving threads, but I can't really think of any concrete examples at the moment where critical sections or mutexes wouldn't do the trick. It's 1 am anyway. Brain no function well coffee without. ;)



@jpetrie:

Very appreciated! That's exactly the kind of answer I was looking for, and way more. You've given me a lot of material to ponder over. :)

In terms of API portability, I have no real plans for using OpenGL any time soon. The project originally began in OpenGL (screenie) but for various reasons I decided to restart it. One of the main reasons being the lack of structure in my game engine - it was getting really messy and hard to manage, and despite the screenshots, it wasn't all that far along yet anyway.

I like having the option open to me just in case, but I'm aiming a bit more for something independant from my game so I can just pluck it out and reuse it for future projects. Having it integrated too tightly with the game will make that difficult. :)

If I understood your architecture right, your engine receives references to objects it needs to render, places them in a render queue, and various internal classes manage the how and when of the actual rendering based on what kind of object it is/treatment it requires. You also (mostly) abstract the API from the client and let your interfaces deal with that aspect. Assuming my (grossly oversimplified) understanding is correct, that sounds a lot like what I was hoping to achieve, but considerably more structured. I think you've put me on the right track - I'm going to work this out on paper and post about what I can come up with later on.

Again, thanks - that really helped. :)

Well actually I've been thinking and realised I hadn't been focusing on creating the engine classes a singular modular unit that is governed by an 'engine' super class. Rather I had been simply adding sub-system classes as their own thing that would eventually be managed by an 'engine' class, and not designing them as "sub" systems of the engine so to speak.

I guess you could say that I had been coding it more along the lines of a big library as opposed to an engine.

I've now come up with a new structure design for the engine:



So only the Engine class is a singleton and thus can be referenced anywhere, and the sub-systems can be gotten from that.

That being said, a class that uses a sub-system might have the Engine * passed in to its ctor so it can get any sub-system references it needs..

BTW, this is a very basic version of my proposed structure, and its pretty early on in terms of development so this change will be very easy since I haven't actually started coding the Engine super class yet.

Anything you can recommend?

By the way, the engine is currently split up into about 8 modules, each being its own DLL (with the exception of the Math (vector3, quaternion etc) module, with memory tracking and logging etc. This is to help build times and code changes/updates as well as keep the engine structured and easy to find code. It's being coded in VisualStudio 2005 so its all built in the one project.

I've currently got it as:
(SGE is the initials of the engine name...)

* SgeCore.dll (helper classes, thread management, memory allocation/tracking, logger, time and window classes)

* SgeData.dll (file system, plugin system, archives, resource/manager base classes etc)

* SgeInput.dll (controller/ms/kb device management and polling)

* SgeMath.lib (math functions, vector/quaternion/matrix etc classes)

* SgeNet.dll (networking and internet)

* SgeScript.dll (scripting [Squirrel proposed], xml, config-scripts)

* SgeSound.dll (sound management, playing etc [library to be decided, FMODEx or OpenAL])

* SgeVideo.dll (DirectX10 renderer, texture, vertex buffers, shaders etc etc)