TDD in game development?

1.) Is TDD applicable to game development? I'm guessing the answer is yes for large projects/teams, but I don't know about small project games like what I've mentioned I've done/am working on.

It's applicable to development, so game development would not be ruled out...but it is something that you'll find is practiced more widely outside of games than in, although there is no particular reason for that.

It can apply to small projects just as appropriately as it can to large projects. Larger projects are in some ways more difficult if the entire team is not using the technique, because other team members might break your tests or you'll keep the tests very small and based around what you author only, possibly at the sub-feature level.

But do remember there is more than one way to boil an egg. Some people get similar benefits by doing other things and some people just frankly have a preference for one thing over another. There is no single correct methodology and if there is a best one, it's often a mixture of things anyway.

2.) If it is, what kinds of tests would one run?

A test, which will likely be a set of smaller tests but collectively considered that validates whatever feature you're implementing. These will be tests that confirm you get an expected output for the given input, Fake and mock tests that simulate real use of the code, either in full or in part and so on. A test that takes you through the entire 'use case' of the feature or code, possibly from initialization, gathering the results/output and validating them as being valid and/or within certain constraints and testing even the destruction of the system to finish.

The general idea is that the tests also define your requirements, so TDD is often considered efficient because you're not authoring a whole lot of imaginary requirements or documents elsewhere, you're doing the requirements, the tests and the code pretty much all in the same place which many people consider is productive.

Also, it can also include tests that test memory and performance impact...particularly if you have specific requirements there.

If you are working on an object, you might run a test every once in a while to make sure the object is in a correct state at any one time, sometimes people try and keep an object in what is known as a class invariant state for example which basically means it's never broken between operations, or at least that kept to a minimum. I used to work at place where as part of their coding standard, each object had an 'invariant' function that would be asserted to be true at various points. Such a function can play a role in these tests.

Basically...whatever tests you want, but enough so to validate against your requirements. Anything less is not really TDD, although that doesn't necessarily mean less is not still valuable.

Is it a good idea/worth learning? etc[/quote]

It's a good idea, but like many good ideas they often represent more of a personal preference and/or is going to be based on your own personal situation, the sort of code your are authoring and so on. Some people take to concepts like this well and others just feel it gets in the way, or they might not benefit from it much so it's nothing but overhead. It's productive and productivity is a good thing, but you need to figure out whether the productivity it offers you is valuable and/or enough to justify it.

Start small, definitely more at the class unit test level rather than testing higher level features and the entire output of a program or specific feature. This will let you dabble in it without diving in head first. i.e. if you're interested, do enough to figure out how much personal value this is to you and whether or not you want to spend more time on it.

Try and figure out whether it'll be productive for you at both the small and large scales i.e. at class level,more of at the class unit test level or at the feature level...or both (again thinking in terms of many small results sum to a big result). You'll have to also maintain, develop and iterate the tests you author so you need to understand whether that's overhead for you or not.

Ultimately, you need to determine that you can accept the overhead and consider it tolerable, understanding whether the net result of all the work still yields productivity or some other benefit that has value to you. You might actually suffer a productivity loss from any concept such as this, but consider that to be okay given the quality of code coming out of the other end. Again, that's a very personal decision and assessment. Programming is all about making trade offs and not all have black and white results.

Bear in mind that it can help to put together an extensive framework to run these tests. I don't recommend doing that initially, but there is a point where if you take this on running many little tests all the time by hand becomes cumbersome so do consider that if you run with this, it is quite inevitable you'll spend time automating the entire process. That will payoff in the long run, but there'll be a period in time where you're not working on new code and putting together a framework and build system.

I am a big proponent of TDD, and I've used it extensively since about 2003, on and off when it is appropriate.

1.) Is TDD applicable to game development? I'm guessing the answer is yes for large projects/teams, but I don't know about small project games like what I've mentioned I've done/am working on.

Writing software is a cost. Money is a cost. Your time is a cost. Your sanity and emotional well being are costs. You must learn to weigh and balance costs in your life.

TDD incurs a large cost up front, it is almost a 1:1 cost equal to the cost of development. There have been several studies on the costs of TDD, and they all tend to converge on a 50/50 split. That TDD cost pays for the ability to maintain your code into the indefinite future without worry. The payoff is the "long tail". The longer tail you have, the more benefit you will see.

TDD for stuff that will stick around forever is a good thing. We use TDD and non-TDD unit tests for various libraries such as memory managers, tools, and more. These things will stick around for a long time, and the investment in TDD will pay off over the many years the software will be in use. In this situation it is more cost effective to have instant zero-cost tests that were expensive to create.

TDD for stuff that is transient is a bad thing. We do not use TDD for game code. It is a poor investment for most game code; it is far less expensive to have an army of QA testers and a six-month alpha/beta cycle than to effectively double the development time from 18 months to 36 months.

For personal projects, especially where you don't have the ability to hire an army of QA testers, TDD is a great thing.

2.) If it is, what kinds of tests would one run?

One of the better TDD tutorials is the creation of a Tetris clone. It is available here and is written in Java. It can be easily ported to the language of your choice if you want. That tutorial provides about 30 different TDD steps, then asks you questions to come up with the next 15 or so steps, then lets you finish off on your own. I've worked through it before and found it to provide good advice along the way. A few of the TDD Yahoo Group contributed to the example, (including several well-known names in the TDD world), and simultaneously making your first text-base Tetris clone as learning TDD is a fun way to go.

I am a big proponent of TDD, and I've used it extensively since about 2003, on and off when it is appropriate.

[quote name='EpicWally' timestamp='1349385023' post='4986902']
1.) Is TDD applicable to game development? I'm guessing the answer is yes for large projects/teams, but I don't know about small project games like what I've mentioned I've done/am working on.

Writing software is a cost. Money is a cost. Your time is a cost. Your sanity and emotional well being are costs. You must learn to weigh and balance costs in your life.

TDD incurs a large cost up front, it is almost a 1:1 cost equal to the cost of development. There have been several studies on the costs of TDD, and they all tend to converge on a 50/50 split. That TDD cost pays for the ability to maintain your code into the indefinite future without worry. The payoff is the "long tail". The longer tail you have, the more benefit you will see.

TDD for stuff that will stick around forever is a good thing. We use TDD and non-TDD unit tests for various libraries such as memory managers, tools, and more. These things will stick around for a long time, and the investment in TDD will pay off over the many years the software will be in use. In this situation it is more cost effective to have instant zero-cost tests that were expensive to create.

TDD for stuff that is transient is a bad thing. We do not use TDD for game code. It is a poor investment for most game code; it is far less expensive to have an army of QA testers and a six-month alpha/beta cycle than to effectively double the development time from 18 months to 36 months.

For personal projects, especially where you don't have the ability to hire an army of QA testers, TDD is a great thing.
[/quote]

Maybe it's the mentality among game developers, or perhaps the nature of games itself that TDD hasn't been adopted in the game development as much as it should've been.

Games naturally follow the cycle of develop-release-forget. Most games don't need maintainance after release. The shelf life of a game, even that of the AAA titles, probably wouldn't last longer than 1-2 years before being thrown out into the used bin for $5. Patches aren't released unless it's absolutely necessary.

Enterprisey products have longer shelf-life, continously being maintained, improved, and released. This makes TDD a more suitable approach for this kind of products.

MMO or online games, where there are plenty of updates, fixes, and patches following the initial release, I would assume benefit a lot from TDD. Though, I haven't worked on an MMO professionally, so I wouldn't know the kind maintenance it'd need to keep it up.

Frob - Thank you for the link to the Tetris by TDD tutorials! I got through the first section, and I'm enjoying it/learning a lot from it. I already did my own Tetris clone a while ago, so to go back through it in a different way like this is pretty neat. I can't wait to get through the rest of it!

I just realized this is about Test Driven Design, not Technical Design Documents! (Slapping self on forehead)

1.) Is TDD applicable to game development? I'm guessing the answer is yes for large projects/teams, but I don't know about small project games like what I've mentioned I've done/am working on.

Yes. It's directly applicable. I'll shamelessly plug my book on the subject:
http://tinyurl.com/agdbook

For art and design, there are emerging equivalents (there was a great presentation about "art unit tests in Maya" at GDC 2012, but I have not links to it). Pairing applies to other disciplines, but also not a lot of documented "good practices".

2.) If it is, what kinds of tests would one run?

Apart from the typical unit tests for code, for decades, developers have found ways to have games "play themselves". Many times innocuous, minor changes to properties or elements in the game (e.g. physics changes, prop placement) can have tremendous, unanticipated influence on gameplay....kind of a "butterfly effect". Having the game play itself across the levels and platforms with as many variations you can and frequently as you can helps. Usually the limitation is size of the server infrastructure a studio can afford to build. I've seen companies with huge investments here (usually MMO devs).

Hope that helps!
Clinton Keith
www.ClintonKeith.com

I just realized this is about Test Driven Design, not Technical Design Documents! (Slapping self on forehead)

TDD is Test Driven Development. You still need a regular design.

It is also called things like Test-first development, and "Red-Green-Refactor".

Basically the point is to have a goal, then create a test that tests for it. The test will fail (red), then you implement the code and ensure all unit tests pass (green), then clean up the code (refactor) ensuring that all the tests still pass.

It is an agile development method as the implementation is not fixed and specified, and can vary over the course of the project.

The point is that code has unit tests at every step of the way, and those tests always pass.