🔒 Locked

My graphics engine (complete source code)

Started by okonomiyaki Jun 1, 2006 at 7:58 PM 21 replies 3.5k views

Original Post

Jun 1, 2006 at 7:58 PM

Hello everyone! I have been working on a graphics engine from scratch (that I call AliquiX. I'm bad at names...). See details below. AliquiX Engine (Visual Studio 2003) Sample AliquiX App (Visual Studio 2003) The point of this post:

I am asking experienced developers to look at my source code and help me code better. I am looking for constructive criticism. "Never do this...", "That's awful!", etc. I have been programming for several years but only recently have I been serious about C++ and graphical development. There is quite a bit of code here and you can look through it as deep as you like.
I would also like comments on general design. How I manage entities, the scene, system components, vertex data and uploading buffers, lights, etc.
I hope that people new to this area of development can use my source code to learn how to solve certain problems and see how something might work. Feel free to PM me or message here to ask any questions about it.

I've never developed anything big or serious, so keep in mind that this is my first attempt at something like this. I've thought hard about many aspects of its design, and I've been influenced by DirectX, Ogre, stuff you guys have said, and other articles on Gamedev. This is something I'm going to use to create some things, and I'm really not striving for full portability or making it easy for everyone to use. Now to describe AliquiX. It is an engine written in C++ that is far from being finished. What it can do:

Manage core components like the renderer, input, clock, scene, etc.
A scene manager handles scene nodes that define hierarchal relationships as well as hierarchal transformations for objects.
Entities define that objects that have some action to be taken each frame. Entities are added to the scene by attaching them to a scene node. Example entities: Mesh, Light, Camera.
Shaders are written separately and meshes can be assigned any shader at any time. I will extend this to more an "effect" system where effects may run through multiple shaders for one object.
Meshes are divided into chunks, and these chunks automatically manage their vertex data and give functions to deal with the data easily. When a shader is run, it requests that the chunk upload the streams it needs to run (if not already uploaded). The meshes can also interleave the data if it's not interleaved, or generate a new chunk to render its normals, etc.
A resource manager deals with resources and re-uploading when needed. Vertex and index buffers are dealt with abstractly by pre-allocating space on the video card and managing the space with the resource manager.
The whole engine is written abstractly. As of now there is only a DirectX renderer, but an OpenGL renderer is possible (though would be kind of difficult because the engine is similar to DX). I wanted to write it abstractly to make things clear and easy, but I'm not really going for full cross-platform/cross-API support.

There are still many things I need to implement, like model loading and better support for large static geometry. I also need to improve the vertex buffer management scheme. And then on to collision detection, spatial structures, etc. But before I move on I'd like input on what's there. I've been asking questions on these forums for years, so I also want to say thanks for everyone who has helped me! I love doing this stuff. I hope others can benefit from my work. I've included a sample app at the top that would use this engine. Here is an example of how I set up a scene:


AXScene *Scene = AXBaseSystem::GetSceneMgr();
Scene->AddSceneListener(new InputListener);

AXSceneNode *Root = Scene->GetRoot();

Root->CreateChildWithEntity(Scene->CreateEntity("Mesh", "Axes", "Axes"));

AXEntity *Box = Scene->CreateEntity("Mesh", "RootBox", "Box");
AXSceneNode *BoxNode = Root->CreateChildWithEntity(Box);

AXLightEntity *Light1 = Scene->CreateLight("Light1");
Light1->Type = AX_LIGHT_POINT;
Light1->Dynamic = 1;

AXSceneNode *LightNode = BoxNode->CreateChildWithEntity(Light1);
LightNode->Translate(0.0f, 5.0f, 0.0f);

AXSceneNode *LastBox = BoxNode;
LastBox->Translate(0.0f, 0.0f, 5.0f);
for(unsigned int i=0; i<50; i++) {		
	std::string Name = "Box" + AXStringConvertFromInt(i);
	AXEntity *Box = Scene->CreateEntity("Mesh", Name, "Box");

	AXSceneNode *RotatingBox;
	RotatingBox = LastBox->CreateChildWithEntity(Box);
	RotatingBox->Rotate(AX_PI/15.0f, AXVector3(1.0f, 0.0f, 0.0f));
	RotatingBox->Translate(5.0f, 0.0f, 0.0f);

	LastBox = RotatingBox;
}

AXEntity *Camera = Scene->CreateCamera("Player", 50.0f);
AXSceneNode *CameraNode = Root->CreateChildWithEntity(Camera);
CameraNode->Translate(0.0f, 0.0f, -5.0f);

Scene->ActivateCamera("Player");

That scene creates a line of twisting boxes with a light rotating around the first one (the light is rotating because of code in a listener not listed here). Screenshot (not too exciting, but just an example):

Thanks for everything. Edit: Fixed a lot of things with lighting, it's pretty much implemented now (besides optimizing and such) Yay, fun!

[Edited by - okonomiyaki on June 2, 2006 3:28:01 AM]

Sr_Guapo

Jun 1, 2006 at 9:49 PM

Looks like a great resource! I may browse through it for some ideas for my own engine.

m4gnus

Jun 2, 2006 at 4:59 AM

looks like extremly clean code(compared to my engine,which is also my first cpp project and looks like a spawn of hell regarding cleanliness). This is actually the first engine's source that i downloaded.

regards,
m4gnus

Erty

Jun 2, 2006 at 5:50 PM

Looks nice and clean from what I've looked at so far. I'm not very experienced with engine programming so I can't really comment on or criticize it but I'll keep on going through it in search of good (and bad) stuff :)

okonomiyaki

Jun 3, 2006 at 12:49 AM

Thanks guys! I have a ton of stuff still to do with it, but I think I'm off to a right start. My projects used to look like a spawn from hell, and I've been trying to code more clearly and cleanly.

DirectXFreak

Jun 3, 2006 at 1:36 AM

Hey, this is great stuff. I'm very impressed with your design and code cleansliness. It reminds me of Ogre! Great work! =)

deffer

Jun 3, 2006 at 8:41 AM

Looks like a tone of work done. And it's just ~6k LOC!

A few random thoughts:
1. Is it based on Enginuity?
2. You put message pumping into your input class, not a window class. Interesting.
3. Did you already put it to use in some project? I mean, did you find in practice what parts of the engine are very useful and handy, and which seem like a bargain (if any)?

What, IMO, it lacks in terms of C++:
1. const correctness - I guess you completely left it for now[grin], or, with all the internal caches possibly updating in anytime, it is plain impossible (think: mutable keyword).
2. exception safety - this is much harded, possibly would require a major engine redesign.

What is a must in terms of graphic engine:
1. textures - y'know.
2. render to texture - swapping buffers with the device.
3. matrix stacks - make the app code much cleaner. If you ever used GL, you know what I mean.
4. programmable shader support - y'know.

Can't find anything more right now...

Good luck!
~def

C0D1F1ED

Jun 3, 2006 at 8:45 AM

Looks like an excellent start!

Just a suggestion: Instead of adding 'AX' to every class name, consider using an 'ax' namespace.

Basiror

Jun 3, 2006 at 8:59 AM

Yes the AX is a finger breaker on the long run ^^

One thing I have realized is, you use a lot of pointers.
In my projects I make a lot use of stl containers and references, I am just sick of checking for null pointers everywhere, although they aren t required since references do the job pretty well

Other than that, its good that people release their sources from time to time, its always interesting how other solve problems thx

b2b3

Jun 3, 2006 at 4:50 PM

Looks very nice and clean.

What did I miss language-wise:

Use namespace instead of AX prefix.

Use references instead of pointers if possible.

Do not use so many macros. Eg. you have 4 LOG_SYSx, 4 LOG_APPx etc. macros.
Add some comments [smile]. When I wrote my engine and got back to it after almost year it was pretty hard to remember what I did and why without comments. Later I spent few weeks adding comments to almost 12 KLOC of code, which was not too pleasant. So add them while it is not so much work and write new comments as you add new features. This will be especially important if you plan to publish source somewhere on the web.

What features did I miss:

Textures. You definitely need them.

Statistics about engine (how many polys were submitted during frame, fps, memory usage etc.).

File import, that is, you engine should be able to import some simple scene files (you can for example use ASE files from 3ds max because they are very easy to parse).

Programmable shaders. Then you can remove FFP completely.

Also, your test application is linked with debug version of DirectX dll. I don't have DX SDK installed so I was unable to run it.

okonomiyaki

Jun 3, 2006 at 5:47 PM

Quote:
Original post by DirectXFreak
Hey, this is great stuff. I'm very impressed with your design and code cleansliness. It reminds me of Ogre! Great work! =)

Thanks! I have studied Ogre's code some so I'm sure that's reflected in my code.

Quote:

Looks like a tone of work done. And it's just ~6k LOC!

A few random thoughts:
1. Is it based on Enginuity?
2. You put message pumping into your input class, not a window class. Interesting.
3. Did you already put it to use in some project? I mean, did you find in practice what parts of the engine are very useful and handy, and which seem like a bargain (if any)?

1. A few parts, yes. The profiler is pretty much straight from those articles. The basic kernel design is the same too, though I changed it around some. I think that's all I got from it.
2. Point taken, I guess I was thinking that it was 'input' and the input class is updated each frame, not the window. I moved the code to the window class though, and the input tells the window to update itself.
3. I haven't made anything big enough to say that this works well. I've just coded most of this in the past couple weeks (I've been creating a framework over the past year, but hadn't coded much, but I didn't update the dates at the top of files so ignore them). Within the next month or two I will be working on a game that will use this though, and I'll get back to you on that.

In all the test scenes I've done so far though, I've found it extremely nice to have the scenenodes (which control transformations) separated from the entities, and any connection is through the Entity interface. It doesn't care what type of entity, or even if any, is attached. That way I can create mesh, light, and camera entities that all can be attached to the scene and be affected by the matrix hierarchy. I can create a spot light and connect it to a dangling wire, and it will automatically rotate and sway with the wire. Or hook a camera to moving nodes to create cutscenes.

Quote:

1. const correctness - I guess you completely left it for now, or, with all the internal caches possibly updating in anytime, it is plain impossible (think: mutable keyword).

You are correct, it's missing [smile] The only time I used const was to optimize parameter passing (const reference). I thought about this early on, and decided to forget about it for now. Honestly, I don't see how you can use const objects because, like you said, almost every object needs to update internal caches and properties every time. So it may not be great, but I have more important things to worry about.

Quote:

2. exception safety - this is much harded, possibly would require a major engine redesign.

I'm not a huge fan of exceptions. Almost everything is checked by checking the return value of the function (AXFAILURE means it failed). I absolutely hate try/catch blocks and I think the should only be used in "exceptional" situations - places you didn't expect to be. If you can expect a scenario where something might fail, that's not an exceptional scenario. I use try/catch in a certain exceptional situations, like requesting a setting that's not there. But this is just my humble opinion, and I don't think I can say this is a universal truth.

Quote:

What is a must in terms of graphic engine:
1. textures - y'know.
2. render to texture - swapping buffers with the device.
3. matrix stacks - make the app code much cleaner. If you ever used GL, you know what I mean.
4. programmable shader support - y'know.

1 and 2 - definitely. Those things should be pretty easy to implement because it's just adding them into the resource manager, and they don't affect design that much.

3 - I create my own matrix hierarchy. I have used GL, and that's essentially what I do. As you traverse down the scenenode tree, the node grabs its parents matrix and starts off with it. This achieves the same effect as pushing and popping, in my opinion very simply and it only takes one line of code. I push everything to a render queue for sorting, and after everything is processed the queue is executed. When a chunk is rendered it has to be able to pull down one matrix that defines its global world transformation. I can't use stacks with the renderer.

4 - Definitely. As with textures, all I need to do is implement the functions in the renderer the create and set them. Then I can start writing AXShader's that create the shaders and set them. I was mainly focusing on abstracting the shaders, which I succeeded with the AXShader interface. It's just a matter of writing the shaders now.

Thanks!!

okonomiyaki

Jun 3, 2006 at 5:59 PM

Quote:
Original post by C0D1F1ED
Just a suggestion: Instead of adding 'AX' to every class name, consider using an 'ax' namespace.

Quote:
Original post by b2b3
# Use namespace instead of AX prefix.

Yeah, that's something I wish I did from the start. For some reason I chose to do it this way.

I do think it's annoying to write two extra colons when using it though, for example, instead of

AXSceneNode *Node;

you'd have

AX::SceneNode *Node;

(edit: actually, you could use 'using' too... so bah.

But that's a minor point. I should have gone with namespaces. If I get frustrated with something sometime maybe I'll go through and convert it to take my mind off things :)

Quote:
Original post by Basiror
One thing I have realized is, you use a lot of pointers.

Quote:
Original post by b2b3
Use references instead of pointers if possible.

I used the const reference passing scheme quite a bit. But you are right, for many things, I used pointers. There are probably some areas I could change, but I like pointers. I like being able to check for NULL instead of forcing an object to be there. But in general I probably could tone them down, mainly with the math library.

Quote:
Original post by b2b3
# Do not use so many macros

I had a lot of macros? I think the only place I had them was for the logger, the ones that you mentioned. And I think those are justified because I love making it so easy to output to the screen... hmm.. I'll think about it though.

Quote:
Original post by b2b3
Add some comments

Yes, yes, I'm ashamed... I tried to be good about that, there are some comments in there (mainly in the header files). Sometimes I feel like if others are looking through my code I want it to be mysterious because it makes it cooler, but that's just stupid [smile]

Quote:
Original post by b2b3
What features did I miss:
...
Also, your test application is linked with debug version of DirectX dll. I don't have DX SDK installed so I was unable to run it.

Yeah, all those features will be implemented. I just needed to get a basic framework down before I felt comfortable really expanding it.

And thanks for pointing that out, I forgot to change it to the release.

Thanks everyone!

hplus0603

Jun 3, 2006 at 6:09 PM

Quote:
Shaders are written separately and meshes can be assigned any shader at any time.

I find that, in general, that's not really the best way to do it. Shaders are more important as driving the lighting model, than they are driving the surface model. Thus, I would recommend adding concepts of "light kind" and "pass," and making a shader be tied to a specific light kind for a specific pass.

For example: "spotlight" during the "transparent z-sorted" pass, would be one specific shader. Also, instead of dealing with skinning shaders as a separate kind, deal with regular shaders as a skinning shader with a single bone and single index/weight assignment, to cut down one axis of shader bloat.

You _could_ add surface shading in there, too, but then you quickly run into a very large set of variations needed. You could perhaps overcome that by using fragment concatenation, but I believe shaders mostly belong in lighting, and only rarely in surface shading.

Good luck!

TheC00L1

Jun 3, 2006 at 7:55 PM

You need to add physics, collisions stuff.

abso

Jun 3, 2006 at 9:36 PM

Collision detection, yes. But physics isn't really part of a graphics engine, but a dynamics engine. While the two are certainly interdependent, they are themselves separate engines.

TheC00L1

Jun 3, 2006 at 11:04 PM

Yeah the topic was specifically on gaphics, but in the discussion of adding, i figured a game'd no good without the physics. But if you jus want it for the purpose of rendering, and plan to have the physics done elsewhere, then that's cool too.

okonomiyaki

Jun 4, 2006 at 12:18 AM

Quote:
Original post by hplus0603
Quote:
Shaders are written separately and meshes can be assigned any shader at any time.

I find that, in general, that's not really the best way to do it. Shaders are more important as driving the lighting model, than they are driving the surface model. Thus, I would recommend adding concepts of "light kind" and "pass," and making a shader be tied to a specific light kind for a specific pass.

For example: "spotlight" during the "transparent z-sorted" pass, would be one specific shader. Also, instead of dealing with skinning shaders as a separate kind, deal with regular shaders as a skinning shader with a single bone and single index/weight assignment, to cut down one axis of shader bloat.

You _could_ add surface shading in there, too, but then you quickly run into a very large set of variations needed. You could perhaps overcome that by using fragment concatenation, but I believe shaders mostly belong in lighting, and only rarely in surface shading.

Good luck!

Interesting. I'm still hashing through how the shader system will work. I thought about concatenating HLSL code so that multiple shaders (if linked) compile down to one HLSL program. But I'll think about this more, because what constitutes each shader could reduce the amount of shaders needed.

So what if you wanted to do bump mapping, for example? I'm a little confused where you would do the surface shading. I'm also a little confused on what you meant about the skinning shader, could you explain that a little more?

Quote:

You need to add physics, collisions stuff.

Yeah, like abso said, I'm not even thinking about physics right now. Integration with a physics engine shouldn't be too difficult though, and I hope to find a good physics engine to power it. I don't think I have time to write phyics too (if I ever want to write a game with this).

iMalc

Jun 4, 2006 at 12:57 AM

Quote:

#define SafeDelete(x)		if(x) { delete x; x = NULL; }#define SafeDeleteArr(x)	if(x) { delete[] x; x = NULL; }

Deleting NULL is safe, so you can safely change these to:

#define SafeDelete(x)		{ delete x; x = NULL; }#define SafeDeleteArr(x)	{ delete[] x; x = NULL; }

Or if you're familiar with the best way to do multi-statement #defines...

#define SafeDelete(x)		do{ delete x; x = NULL; } while(0)#define SafeDeleteArr(x)	do{ delete[] x; x = NULL; } while(0)

Don't change your SafeRelease of course though!

[cool]Very very impressive!!![cool]

Enselic

Jun 4, 2006 at 12:58 AM

Dude, that engine code looks totally pro! Clean code and a good structure. A great insirational source!

jkleinecke

Jun 4, 2006 at 1:22 AM

I sent you a PM.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.