Five years have now passed and I have an engine to show for it! Even better, I recently finished a small AI Tank game that runs on top of my engine. While the programmer art leaves plenty to be desired, the game runs well and is quite a bit of fun. Furthermore, it proved that my engine is, indeed, a functional game engine. A proud moment indeed!
Along the way, however, I learned a great plethora of lessons; some from having been told repeatedly, others from making rather painful mistakes. So, I've decided to write them down so that perhaps others can learn from them as well, without making aforementioned painful mistakes!
Therefore, without further ado - herewith the various lessons I learned while building my game engine. They are listed in no particular order - no one lesson is more important than the other. If you have any comments, please reach me at grafalgar@gmail.com. I look forward to any feedback!
Lesson: Forethought goes a long way
Before starting to write code, spend a little time planning your code. Think about what you're going to do, how you're going to do it, what impact it makes on your project and how it will affect your code in the long run. This is as opposed to following your first instinct, coding as fast as you can, and hoping for the best. While the latter may get you results sooner, you will end up throwing away or rewriting most (if not all) of your code when new projects come along.
Forethought includes maintaining a general structural overview of your game/engine/etc, and whenever you want to create a new system, really paying attention to where you want to put it. Do you want the Render engine to manage GUIs? Or do you want the GUI to sit on top of the Render engine? Where do Physics fit in, and how does it hook into game objects? What about multithreading? There's no right answer here - it all depends on your goals and what you aim to achieve. It may take you a little longer to get to those goals by not taking shortcuts, but you'll thank yourself later!
So do spend a few moments to carefully consider what you're going to do, make sure you're aware of the pros and cons, and that you can live with the cons! When you add a bit of thought to what you're going to be building, your code generally results in a more stable, coherent codebase that will be much easier to maintain in the long term.
Case: Before I started the engine, I drew a simple diagram that showed how I was going to architect my game, engine, tools, etc. It showed exactly where I wanted physics, GUI, rendering, AI, etc. That same diagram has helped me decide how to insert new systems so that the engine still "makes sense." Since that time I've added new systems as the need for them arose and I still referred back the original design, modifying as necessary, keeping my goals in mind and staying true to the original architecture as much as possible. Sure, there have been times I needed to modify my original design where I found it to be insufficient. But again, planning and forethought comes into play once more: Am I sure I want to change the design? What are the pros and cons? Are there any alternatives? Will this benefit me, and my engine, in the long term?
Anyhow, I now have a fairly elaborate engine with networking, physics, messaging, AI, GUIs, and so on. The best part? They're all happily working together because of the aforementioned forethought.
Lesson: Too much forethought will kill you
So with all of that talk of forethought you also have to be careful of too much of it. At the end of the day, there is no such thing as the "perfect system" - so don't kill yourself trying to invent it. As I've said already, be aware of the pros and cons of your system, learn to accept the bad with the good, and move on. If you just can't tolerate the bad then don't be afraid to rewrite.
Sometimes, many times, it helps to just take your best guess at the problem and run with it. Maybe you'll learn that your shot-in-the-dark was actually spot on and you couldn't have dreamed of a better outcome! Maybe you'll learn you just wasted a week's worth of work and need to start all over again. In either case, you did something and you learned from it. Many new programmers spend countless hours over-engineering a problem and being driven crazy by not finding the best solution with no flaws. Just take the plunge already!
Case: I spent hours, days, weeks trying to figure out how to architect the perfect network layer in my game, without ever writing any code. Consider also that I had never written any network code to begin with. But, I was determined to come up with most robust, flexible, and fast network layer if it was going to kill me. Finally, out of frustration of having wasted lots of paper and having no code to show for it, I just took my best approach from the bunch and ran with it. Now, granted, what I ended up with was very far from ideal. In fact, I can tell you that it's pretty crappy. But.. I wrote something, learned from the mistakes, and moved on. If I was adamant about finding the perfect solution, I would still be banging my head against paper and have nothing to show for it. Instead, I have a networked tank game that taught me many good lessons about network layers!
A tank game made to demonstrate AI called "tAInk"
Lesson: Don't be afraid to rewrite systems, delete code, or start over
Sometimes people don't want to rewrite systems because they spent months, if not years, developing them. Us programmers, man, we get very attached to our code. It's not surprising - code isn't easy to write, and heck if you're going to abandon all your hard work! It's a moment of pride when you realize that your tens of thousands of lines all magically work together!
The problem is, if you grow too attached to your code, it won't get a chance to grow. You'll end up dealing with crappy systems you wrote years before, along with their frustrations and annoyances. By adding more crappy code to said crappy code, supporting the resulting crappier code with even crappier code, eventually you'll end up with a big pile of crappy code that you end up abandoning as a whole out of frustration and starting anew, just in order to start "fresh" and "clean." If you think abandoning one small system is heartbreaking, try it with a full-blown engine that took years to develop and shipped several commercial games. Yes, many game companies go through the exact same motions: eventually the engines become atrocious, and they spend large clumps of cash to build new technology from scratch.
Don't be afraid to rewrite systems if you're not happy with them. Even if the system is "good enough", still rewrite it if you come up with something better. There's a nice word for this: "Iterative" Programming. Don't fear it!
Case: The networking layer I just mentioned? Yeah, after a few months of struggling with my rather poor solution, I scrapped it, and started over with what I learned in the meantime. My solution is still not perfect. Hell, it's nowhere near professional network layers, but it's better than before!
In addition to rewriting the network layer, I rewrote my editor, game-logic systems, object-hierarchies, physics, messaging, GUI, Rendering, and so on several times. I have no idea which iteration of what system I'm on, just suffice to say that with every rewrite I grow progressively happier with the results.
Lesson: Separate your game logic from your engine
By separating your game logic from your engine code, your engine remains relatively "pure" and reusable. If you don't keep them separate, you end up with an "engine" (emphasis on quotes) that must be stripped and cleaned up before every new project. Each time you start a new project, the process of stripping and cleaning becomes more and more difficult. It's always surprising to see how much game code makes its way into an engine, and how difficult it is to get rid of it.
What do I mean by mixing engine code and game code? Well, if your rendering system checks an object's health to determine whether or not to render a particular mesh red, then you've mixed engine code and game code. When you try to create a new game that doesn't involve health, you'll have to figure out how to strip that code from the render system.
Remember - programmers are lazy, and never trust yourself (or anyone) to not take a shortcut if it's easy to take. You'll be much happier if you can just reuse your engine without any modification/cleanup in your next project.
Case: When I wrote my first game I went into it not knowing what a game engine was. After many months of struggling, I finally had something to show but everything was intermingled in the worst possible way. Game objects knew about levels, levels knew about game objects, sprites knew about input (don't ask), level tiles know about monsters, oh it went on. When the game was finished, there was no way to separate the engine from the game. It was all or nothing. Needless to say, that was a whole lot of abandoned code simply because I could not reuse it for a new project.
As a result, separating the game from the engine (and keeping it that way) was one of the first rules I made before writing or even touching code. I was going to keep the engine and the game logic code separate even if it killed me! To enforce this, I put my game in a separate project than the engine, and made the game project statically link against the engine lib. In this manner, even if I wanted to include game logic from the engine, the linker would complain (circular link dependencies). So, not trusting myself, I actively blocked a shortcut. As a result, my engine is 100% game-code free.
Lesson: Do build a game, or several games, while building your engine and tools
Have you seen people boasting about their awesome-tacular engines with the bestest-of-the-best rendering, physics, and networking yet have no game to show for it? Have you tried using said much-hyped engine to build a real game? You'll quickly find that just making a game involves a great deal of work that ends up having very little to do with the "cool" systems, like Graphics, Physics or Networking. How do your game objects interact? How do you set properties, like Health, Speed, etc? What about points? How do your game objects interact with other systems?
By developing a game alongside your engine, you are forcing yourself to use all of the pieces of your engine together, for a single purpose. And let's face it; game engines are large and unwieldy - with way too many systems and subsystems! How can you possibly know they all work together without having something to show for it? Writing a game is entirely different from writing an engine and I argue that a game engine without a game is nothing more than a tech demo. Now, there's nothing wrong with writing a tech demo! Just be sure not to mistake a tech demo for a game engine, until a game has been made out of it.
In addition to building a game alongside your engine, I urge you to write several games or applications alongside the engine. What does that accomplish? It will force you to think about how your engine code is shared amongst different games, without stomping on each other. So, for example, you'll implement bump mapping for use by Game 1, and now Game 2 will refuse to render. It's up to you to figure out how change the engine such that both will work. It's very difficult, and time consuming, but very beneficial and ultimately invaluable in building up your engine to be a generic engine.
Case: During my engine development, I built several small "games" to test the engine: One involving a three-wheeled buggy racing around a track; one involving a cannon shooting big ol' cannon balls around planets; my tank AI game; and a 2D gear-themed adventure thingy. Additionally, I wrote several previewers and test applications, all sharing the same engine. Even my editor runs on my game engine. Since each project / test app / game / etc runs nicely with the same engine, it proves (to me at least) the versatility, robustness and flexibility of my engine. It's a proud moment to see all that come together, let me tell ya!
A another game made with the game engine
Lesson: Know and identify throwaway code
Not all your code needs to be perfect. Not every line needs to be a shining beacon of technical excellence. You'll never finish anything. That doesn't mean you have license to write poor code all the time either. It just means that you need to be aware of the times when you write "subpar" code, why you're doing it, and what you will do about it later on.
Part of this is also knowing the consequences of writing poor code. If you write poor code, and forget about it, you'll pay for it (often, dearly). There are a number of times where I'll track down a bug only to find throwaway code I wrote months earlier and promptly forgot about.
A good example of throwaway code is to have a temporary stub-system in place to get another (dependant) system working. Let's say you're working on a making a particle system, but nothing to render the particles yet. No matter, write some stubs and draw little cubes in the world where your particles are meant to be. It's dirty, hacky, and potentially spaghetti .. but it helps you get your particle system done, right? When the particle system is complete, go back and rework the rendering so that that, too, is correct. It's kinda like duct-tape: excellent in the short term in order to reach larger goals, but remember to go back and do it right!
Case: When I wrote my GUI system, I wanted to test button presses and such but my input system wasn't created yet. So, instead of shifting focus and building the world's-bestest-input-layer, I hacked something together in 30 minutes that would do the job, and continued fleshing out the GUI system as needed. And by hack I mean hacked. I directly polled DirectInput from within the GUI manager and fed it straight to the buttons, which in turn analyzed the data and acted accordingly. No abstraction, no flexibility, nothing. But it got my GUI button input handling working rather nicely. When the GUI was where I wanted it to be, I went back in and reworked the input without impacting the GUI very much. So, I wrote some throwaway code, achieved a larger goal, went back to fix said throwaway code when I could apply full focus.
Case: Now here's where forgetting about throwaway code bit me in the hiney: In order to get 3D mesh collision working, I had to make a really nasty hack that tightly coupled my physics engine and my rendering engine. It was a nasty hack, but it helped me achieve my goal of getting trimesh collision functional. Unfortunately, I completely forgot about the hack. Months later I changed my vertex format in the renderer, and my physics broke completely! I could not figure it out, and continued to not figure it for two days! I finally tracked it down to the hack I put in, and even discovered a big ol' comment telling me I had to fix it or I'll see some nasty bugs later!
Lesson: Tools. Love them. Make them. Love making them
Programmers are lazy. Programmers like to take shortcuts whenever they (we) can, because we have a goal to achieve and doggonnit we're gonna get there! So, who needs a fancy tool if you can just jot down some settings in a .cfg file? Psh!
Uhm .. yeah .. so that's terrible. Let's get out of that habit, shall we?
Tools are awesome, and the more you write, the more you'll love them. Do develop an editor that lets you easily create levels, game objects, apply textures, write scripts, preview your game, etc. It's a tremendous amount of up-front work, but when you're done you can create new games much faster than before. Additionally, when you bring more people onto your project they'll be able to hit the ground running much faster, instead of relying on you to teach them how each individual text file needs to be hand-tweaked to make anything work.
Case: Once my engine had something up and running I built an editor to create levels, game objects, physics-rigs, effects, and so on. It has allowed me to create my game assets with little frustration and has saved a ton of time in just initial setup. My editor has a long way to go, and it's far from either done or perfect, but it has saved me so much time it's insane.
So yeah - don't skimp on the tools. Besides, do you really want to end up maintaining what ends up being hundreds of unique text files? :)
A game editor tool for the "tAInk" game
Lesson: Use Version Control, even if you're the only developer
So you're the only person working on your game, you know where everything is, and you don't have anyone stomping on your toes while you're working. You don't need version control, do you?
Well, let me ask you this - have you ever made a large amount of changes to your game, radically changing how everything works, only to realize toward the end that you've made a big mistake and you have no idea how to recover from it? Or, alternatively, you've introduced a nasty bug - but cannot remember the changes you've made to cause it? Or your IDE crashes and you lose half your code? If any of this has not happened to you, don't worry, it will and you will cry long tears knowing you have no historical backups of your code!
Version Control allows you to keep incremental backups of your work just in case you need to recover old source code, or see exactly what it is you changed, or pinpoint when some bug got introduced. It also serves as a nice backup in case you somehow lose all your work in a hard drive crash or something similar.
Case: When I first started my engine, I was going to work on it with a friend of mine. We set up a subversion server and went at it. He's moved on though, but I never let go of the subversion server. I found it immensely useful to double-check my changes, revert to backups, and so on. A month or two ago I lost everything through a hard drive crash ("Tick of Death" anyone?). If I had not had all my work on the subversion server, I would have lost it all. Can you imagine? Five years worth of engine work - gone. Yikes! Well, fortunately for me all I had to do was reinstall the subversion client on my new hard drive, sync up, and all my work was right there! I only lost whatever work I did not check in before the crash.
That concludes my short-ish list of lessons I picked up through the past couple of years of game engine development. Unfortunately I had to learn many of these lessons on my own, sometimes painfully so. I hope that they are helpful to you and will prevent you from making the same mistakes I have, or rather make them much less painfully!
