Are Game Developers 15 Years Behind?

It depends which company you work at, obviously.

e.g. At the games studio I'm currently working for, whenever I check-in code, it's compiled and tested by a remote machine before being pushed to the trunk (meaning I can't break the build). At a previous games studio, any commits to certain libraries were automatically added to a task-tracking system flagging the need for them to be reviewed. I've never had those luxuries in corporate software jobs.

I've worked at a top 50 corporation (not games) that was far worse -- Emailing ZIPs for version control and still laughing at OOP as some academic ivory tower concept... So I've got an equally bad stereotype of PHB's and corporate code-monkeys and "architects".

The blog post is a bit silly, in that it's pondering why a stereotype doesn't hold up under analysis.

Shhh.. don't tell the guys who wrote WoW or EVE online or any massive multiplayer game which supports over a million concurrent users in a realtime virtual environment, handling millions of e-commerce micro transactions daily.. they are doing it "wrong".. obviously their "15" year old coding style just can't compete with the new fangled "agile" methods to todays programmers

Game development is like any other software endeavor, there are those who use best practice and do it successfully and those who don't. You can't hold up the failures and say the entire industry is doing it wrong or backwards. Hold up the best and try to make that statement? Most software projects fail, no matter it be game developement or creating a new billing system for Medicare or putting a probe on Mars.. They all have the same chance of failure because ultimately humans are only so clever.. Once software grows beyond the ability of an individual or a few individuals to predict its behavior accurately they fallback to methodology, testing, abstractions, more testing and debugging.. You can only do so many cycles of that before you just have to ship it or cancel it..

Game developers used the agile team based methods well before its adoption by mainstream. Game development has always been collaborative small teams, artists, designers, programmers and producers. And the short development cycles force them to connect and update their progress daily. This is even reflected in the layouts of game development studios, usually they are open based work place arrangements designed for quick access to each developer and facilitate micro-communication and flexible small sub-teams working on sub-tasks.. etc..

Unit testing, Agile development, etc.. I've seen them used in the real world, the good teams adapt the methods to their work style, the poor ones just pay homage to the superficial and abandon it at the first difficulty.. Ultimaitely it boils down to the skill, competency, and experience of the developers and their passions more so than any methodology, though the best teams will have a formalized methodology..

-ddn

It depends which company you work at, obviously.

e.g. At the games studio I'm currently working for, whenever I check-in code, it's compiled and tested by a remote machine before being pushed to the trunk (meaning I can't break the build). At a previous games studio, any commits to certain libraries were automatically added to a task-tracking system flagging the need for them to be reviewed. I've never had those luxuries in corporate software jobs.

I've worked at a top 50 corporation (not games) that was far worse -- Emailing ZIPs for version control and still laughing at OOP as some academic ivory tower concept...

The blog is a bit silly, in that it's knowingly trying to deconstruct an assumed stereotype, and then pondering why those false assumptions aren't holding up.

Many game developers say that game development is more like this even though your experience is the opposite.

Also, there are plenty of game developers defending that difference saying that game development should be or must be like that.

It's reassuring to hear that your experience is that game development is better engineered than the stereotype. I can see why you would feel that the reputation is not deserved.

I wouldn't say that gamedevs are 15 years behind, but 10 might be more accurate if for no other reason than the unholy obsession with C++. Combine that with the relative immaturity of middleware tools and qa processes (development processes have actually improved a bit here)...

I wouldn't say that gamedevs are 15 years behind, but 10 might be more accurate if for no other reason than the unholy obsession with C++. Combine that with the relative immaturity of middleware tools and qa processes (development processes have actually improved a bit here)...

There's a lot of discussion here on reddit about it. Some disagree that it's true, others defend it as true because it must be.

Are you a professional game developer?

Anecdotal evidence and lore from the internet do not reliable data sets make.


This entire article is built upon a very flimsy straw-man image of game developers, which in my experience simply holds no water. Every successful studio I know about (in the sense of being privy to their development practices and habits, not just "oh hey I played their game once") is far more advanced in terms of engineering process than you make them out to be. Now, I don't mean that to be an attack on you or your article; I just mean that you're basing your opinions on faulty data, and therefore the resulting critiques are basically misguided at best.

Highly reactive, prototype-based, iteration-driven development models have been central to most mainstream game development for at least 15 years. We were doing "agile" many years before The Manifesto. We left behind waterfall almost as soon as the ink dried on the ill-fated paper that accidentally made it popular. (Well, to be accurate, game development never really did embrace waterfall or similar BDUF methodologies very seriously; it went from hack-it-until-it-works in the early days to a more iterative system in the '90s. A few places have tried BDUF over the years but I'd be surprised if any of them survived very long, at least not without changing things up pretty drastically after the first expensive failure.)

In terms of methodology, games are probably actually one of the best sectors of software to be in. They're demanding as all hell in many ways, and that forces us to be up to par on how we get things done.


Now, if there's one area where game development might lag, it's in tools adoption - or, rather, was several years ago. Centralized task coordination and tracking software, centralized documentation systems/wikis/etc., and so on have different degrees of buy-in in different corners of the industry. However, while this was spotty when I first did investigations into the matter about 8 years ago, I've been refreshing my own personal data lately and it turns out that this has all but disappeared as an issue in game development. Ironically enough, basically every studio I know of (again in the sense of knowing how they do business, not just passing familiarity with a name) is using... JIRA. Many of them also use Confluence. I find it rather humorous that someone at Atlassian of all places is questioning the practices of some of their most loyal customers


However, that only applies to process-related "meta" tools. Game development has had better end-to-end in-house tool pipelines than most commercial software for decades. For instance, at my current workplace, we're shuffling literally several terabytes of assets and code around, all in automated tool chains, and spanning over a dozen different software packages, libraries, and custom "glue" apps. We have this down to a finely tuned science, not because of some idealistic notion of process, but because we have to. We can't do our jobs without it.

I think about trying to introduce that level of complexity and sophistication to the more traditional IT-type shops or shrinkwrap application shops I've been at, and all I can do is laugh. Their poor heads would explode Now granted, my personal experience doesn't span some of the larger-scale apps out there, but still; what we had was a far, far cry from what game studios have done for ages.


I had to pull some serious teeth to convince one former employer to even introduce bug tracking software and incremental software releases. We sold a shrinkwrap application that you've probably heard of (well, if you have kids anyways). In the end I wrote our own custom bug tracker in my spare time because they refused to spend money - even a trivial amount - on buying an existing package. When I left, there was still substantial resistance to the very notion of tracking our bugs, and several years later I have it on good authority from inside sources that they virtually stopped doing any kind of formal defect tracking or update planning. They begrudgingly started doing it again as part of contractual obligations to the company that bought them out when they started to flounder.

There were speed bumps getting my current (game industry) employer to adopt some more modern practices and tools as well, to be fair; but nothing like that other company. And once we had things in place and running smoothly, nobody has looked back. For a while we had a staff member whose entire responsibility was to manage the defect and task database, and monitor our automated regression testing systems. I've never talked to a non-games software shop that even knew what regression testing was, let alone practiced it.


I can now name, off the top of my head, probably half a dozen companies who have better pipelines, better processes, and smoother adoption of best practices than my current employer (which isn't to say my current employer is bad at any of that stuff). All of them are games studios.



Now... in the interests of fairness, I'm countering your anecdotal ideas with more anecdotal ideas of my own, so I'm not trying to claim that I have the empirical standard of all evidence on my side I just wanted to provide a counterexample to some of your assumptions and illustrate that the stereotypical cowboy-coder image of game developers is, frankly, totally unfounded.


Maybe in 1983.


Not today.

I really don't get it what does it mean "to be 15 years behind". I always thought that being up to date was related more than anything to releasing stuff for the most modern hardware.

@chromofo, what do you think it means?

Thanks ApochPiQ for your detailed reply!

Anecdotal evidence and lore from the internet do not reliable data sets make.

I find it rather humorous that someone at Atlassian of all places is questioning the practices of some of their most loyal customers

You are right. I don't publish much evidence of the truth of the stereotype, but then there are two problems with that.

Firstly, you have to be honest, it's a well-known stereotype. It does exist and I think anecdotal evidence is proof enough of that.

Now as to whether it's true... teams who adopt these tools and practices are in abundance in the game industry. I acknowledge that. I'm very much on the side of Atlassian's customers. What I want to know is how come there are so many teams who believe these practices and tools are not suited to their industry? I've had PLENTY of discussions with AAA title developers and teams who acknowledge it's a problem but I am not going to publish their names!

However you don't need me to. You can just look around at the comments on the reddit thread for example. These responses boil down to a few standards:

1. How dare you! or How would you know? (yours)
2. You are right, I know it sucks.
3. You don't understand - games are not suited to this. (I address this in my article but I acknowledge it's tl;dr)

Maybe in 1983.

Not today.

Glad to hear it!

For any absurd set of thinking, you can probably find a non-zero percentage of the population who is utterly convinced of its truth. (There's a really nasty, below-the-belt comment about religion hidden in here someplace )


Why do some developers think that clear-win best-practices are not applicable to games? I dunno. Maybe you should ask them... to me it seems like an exercise in rationalizing someone else's delusions. Fun for psychiatry majors, but not my personal favorite hobby

Firstly, you have to be honest, it's a well-known stereotype. It does exist and I think anecdotal evidence is proof enough of that.

It's well known for all programming shops AFAICT.

Phrases like "agile" and "best practice" themselves have been ruined by the hordes of "certified" analyst architect systems engineer programmer sigma-6 black-belt wizards and their pointy haired bosses who can talk the talk but not actually program, motherfucker. Then you've got the mavericks who only care about programming, motherfucker, but have no appreciation for formal methodology. Most people are (thankfully) somewhere in-between the two extremes.

If you haven't heard of the former, you've probably been too busy giving out copies of 'who moved my cheese' or getting certified in the latest fads, and if you've not heard of the latter, it's probably because you program solo

These stereotypes are common to all software people, not just games ones. See thedailywtf for fodder.

For any absurd set of thinking, you can probably find a non-zero percentage of the population who is utterly convinced of its truth. (There's a really nasty, below-the-belt comment about religion hidden in here someplace )

True unfortunately.

Why do some developers think that clear-win best-practices are not applicable to games? I dunno. Maybe you should ask them...

I am asking them, but you keep answering

Are you a professional game developer?

No, I am not. I am comparing my experience in bizdev at shops I would consider slightly above average to my general understanding of gamedev houses (which admittedly comes second hand from the more renowned members here), and a few educated guesses about the... less traditional gamedevs that comprise a lot of the casual/mobile gaming world.

What I want to know is how come there are so many teams who believe these practices and tools are not suited to their industry?
[/quote]

I suspect because of time crunch. Practices are percieved as process, process comes with overhead, and we barely hit our release date last time. This is a common driving force in bizdev against process improvement, and I can only imagine that the increased time to market pressure in gamedev exacerbates it.

I've never talked to a non-games software shop that even knew what regression testing was, let alone practiced it.
[/quote]

Even the terrible QA departments I've worked with know what regression testing is and dedicated time to it.


So while I don't deny that some bizdev houses (especially those that don't sell software or some software as a service sort of thing) are truly ass-backwards, my experience in run of the mill bizdev just doesn't hold with some of the stories here.

Oh, I certainly didn't mean to imply that all bizdev type shops are as bad as the example I gave. Nor do I claim to have a representative set of data for that section of the software world. Just offering counterpoints to show that the stereotype cuts both ways: sometimes, the "real world software" guys are just as bad as the popular image of game developers, and vice versa.

e.g. At the games studio I'm currently working for, whenever I check-in code, it's compiled and tested by a remote machine before being pushed to the trunk (meaning I can't break the build). At a previous games studio, any commits to certain libraries were automatically added to a task-tracking system flagging the need for them to be reviewed. I've never had those luxuries in corporate software jobs.

I've worked at a top 50 corporation (not games) that was far worse -- Emailing ZIPs for version control and still laughing at OOP as some academic ivory tower concept... So I've got an equally bad stereotype of PHB's and corporate code-monkeys and "architects".

Ditto on both counts.

Non-tech companies tend to operate on the assumption that they are not software shops. At one non-game company we had 5 software developers (out of 30 employees) and we were finally able to convince the owner that he needed to properly fund a proper versioning system and backup process. His business -- he kept telling us -- was to provide services to the customers, not write programs. Yet he employed five people to do exactly that. We wrote software, managed databases, kept CRM systems running, and more. He balked at some of the practices we kept trying to push, which is part of the reason I left that job. I've found that while the pay is better, treatment is far worse than the industry.

Game companies, and most tech companies, understand that software is their life. They understand that the developers are basically spinning gold from the ether, and they need to treat them well. They embrace software development best practices because they recognize they are in the software business.




His sources are anecdotal evidence from a few of his peers. He also has links to a few places that have anecdotal evidence for his claims.

Who did they talk to? If they include basement-made projects, the projects in the "Help Wanted" section that starts as an XNA template app with a few crappy drawings and then dies a week later, includes the hundreds of thousands of flash apps that litter the web, and also includes the major studios, then I would agree that collectively that group of people largely ignore modern software design principles.

However, I wouldn't call those people the game industry. I would limit it to the profitable studios, and the few studios who follow the profitable practices but then die for other reasons.

Based on my experience with multiple profitable studios, he is off base.

For a blanket statement, it is definitely wrong. Software companies will vary wildly on how 'advanced' their software practices and technologies are both in and out of the games industry. I know in general, Japanese developers have some archaic practices in place that one of my friends who recently moved to Japan faced a massive uphill struggle to get some decent pipelines and practices in place. Look at the differences between the major console providers SDKs and the gap can be quite large.

When I was working at EA, we had continuous integration going round the clock every time new code was checked into the trunk and data was submitted. Errors in the build got sent to developers who checked in data and/or code. Daily discs were getting burned. Anyone could pull down executables and built data for any platform at any time from the build server. We had tools to merge two separate Perforce depos due to outsourcing security concerns.

I left the industry since then and the only area that I would say the Games Industry is behind in is investment in quality QA.

(I think everyone here who has responded have been involved in games at one stage or another).

[quote name='ApochPiQ' timestamp='1301972883' post='4794468']
Why do some developers think that clear-win best-practices are not applicable to games? I dunno. Maybe you should ask them...

I am asking them, but you keep answering


[/quote]



Well, let's look at how you're going about asking the question...

- You insinuate that game developers are ignorant or unwilling to adopt best practices
- This places those of us who know better on the defensive immediately, wishing to correct your misinformation (because, let's be honest, it's kind of personal at this point)

- You present your anecdotal stereotype as if it were fact, and ask some speculative questions in that light
- This immediately grabs the attention of anyone with training in formal logic, who is going to smell straw man all over the place and want to correct your illogic

- You post in the Software Engineering forum of a game development site, and on programming.reddit.com
- These are two communities full of people who are far more likely to have already embraced best practices in the first place, and they're going to give you their experiences, not hypothetical speculation from trying to understand everyone who hasn't bought into the practices at all



So to me it seems only to be expected that you get the results that you do. You're going about it entirely wrong.

If you truly want to know why some people think games are an inappropriate venue for engineering practices, you need to do three things:

- Find a better outlet for asking this, not loaded with people who are going to answer the way I did
- Present the issue in a neutral light. Ask if games are appropriate for engineering discipline, and ask people to justify their answers
- Refrain from making assertions of any kind, because this will immediately trigger "someone is wrong on the Internets!" syndrome from a substantial fraction of your readers


Frankly I'm just as curious now why someone would be naive enough to think that engineering discipline has no place in the games business; unfortunately, the best place I can think of to find such people would be at a support group for chronically failing game developers. I'm really itching to know who it was that got you started on this line of investigation in the first place, so I can mock them publicly... although I can certainly understand your hesitation to drop that name

Considering close association with Atlassian, I find this write up to be disappointing and shallow. It touches on some broad generalizations of publicly available information. Except that what is public represents only mostly hobbyst development which only rarely brings any advanced insight. The article would carry same merit if it simply called all business and enterprise development a bunch of PHP script kiddies and Java code monkeys.


Console development is bound by NDAs and licenses are given only to proven developers. All of them are forbidden from discussing any details in public. Simple consequence is - there is no publicly available insight from anyone doing any kind of console development, aside from a random blurb, meaning PS3/XBox/Wii simply don't exist on the web. iPhone is Nickleodeon. Most development is trivial and irrelevant as far as business practices or quality goes. Cheap is the name of the game. Big projects with $500/hour development rates are part of corporate development and internal branding or related efforts, so they are, again, not present on the web.

Big publishers are probably ahead of most bizdev shops. A simple google for "$(AAA publisher) + agile/continuous integration" or "MMO agile/contiuous integration" brings up articles like this or this. Searching around gamasutra brings articles like this. These are companies where per-product investment is $100 million and revenue measured in billions.

The final misconception is regarding priorities. Source code is a solved problem. Not only that, but entire physical development is increasingly irrelevant as far as big picture goes. Entire development budgets can be as low as 30%, that includes creative design, testing, coding and asset creation. Success of large publishers is almost completely dependent on marketing and franchise or licensed IP.

Then there is innovation which puts bizdev into stone age. Toolchains which allow real-time versioned collaborative development on Xbox360 with scenes comprised out of billions of polygons - all real-time and interactive. How many businesses would consider one million database rows "huge".


The article illustrates a huge clash of cultures. But it has nothing to do with script kiddies learning in garages.

JIRA is de-facto toolchain for mostly OSS development stacks, primarily Java centric. That ecosystem is open, homogenous and fairly standardized. It's used almost exclusively for various forms of CRUD development. The toolchain it presents is well suited for both the culture and problems that need to be solved. The number of openly available frameworks and libraries facilitating all aspects of development is enviable.

But when one goes out of that industry, the ecosystems are closed and proprietary. While that does have slightly negative effect on quality it also skews the perspective on what goes on. People involved in such development simply do not engage into broad public discussions nor are they allowed to. Another industry where this is prevalent is industrial automation and CAD. Reason are similar - underlying toolchain is always proprietary and trade secrets and other legal implications of discussing such work freely pose too much of a risk. Evaluating these industries in same manner would be comparing them industrial automation by Arduino project.

The other aspect ignored is the actual needs. In aerospace engineering, deficiencies in CAD tools are irrelevant. If a tool does not have "select all", someone will be employed to click on all elements. LabVIEW is a pain, but solves actual needs. Bioinformatics is a mess (this is one field to look into, they really love the insight, but one needs a degree in medical or related field + CS) but pharma is big money, so throwing more manpower is still an option. Gamedev also sees abundance of labor - starry eyed kids, willing to work 90-hour weeks - they are cheaper than retraining top-of-the-line experts, whose time is valuable and expensive.

Article ignores the reality of business - in most industries, automation of certain tasks is too expensive. General IT, with falling profit margins need to minimize expenses on custom development and infrastructure. Minimizing labor makes sense. But elsewhere, just throwing more people at it works. Automation is only introduced when required or when the field becomes a commodity. This is why so much manufacturing is still done by people - they are simply cheaper. Agile does not necessarily mean machine automation.


Very few still have time to try to figure out the precise definition of the word "Agile" because ever since they adopted "agile" practices, software development is a solved problem.

Forgot to mention this in my previous post: http://blog.ccpgames.com/alli/files/2010/08/Ottarsson_GlobalAgileGameDevelopmentNoPassportRequired.pdf

GDC presentation from CPP (Eve Online) who use Agile on a global scale.

The major issue we have in game development are the deadlines. We can't think how we are going to code and design it, because we need to get it done fast.
In the company I work we do both, software and games. And both department are very different on that matter. Software side have more time and money, for simpler applications. The game side, there is the publisher on one side that want the game out for that period of time and the funding on the other side that is too low.

If we fix that basic fundamental issue, then maybe game developers will have more time to unit test and all that stuff.