Using native GUI controls for games

Quote:

Original post by Kylotan Quote: It is pretty much par for the course for game designers who know nothing about Human-Computer Interaction to throw away the style guide for whatever platform they're coding on and instead develop their own unique interface for their games with visual, tactile, and response time characteristics unique to their game. This drives me insane.

I like this guy already. If only using native widgets would solve all UI problems in games.

Quote:

In response I mentioned that basically DirectX and the Windows GDI just do not cooperate, but in fairness I haven't tried this or read about the subject for about 5 years now. Is this still the case - is there no efficient way of rendering them under DirectX? Is it practical to use Windows controls in games? What about the new stuff as part of .NET?

I'm pretty sure that even Avalon or whatever it's called won't do what game writers want

Quote:

And I don't suppose there's any more luck with the various Linux widget sets?

Doing this on linux involves writing a small bit of glue code between the GDK and SDL/GL. This hasn't been done, yet, however. It's a tiny bit more complicated than it sounds.

Quote:

My second point was that players probably don't want to see the Windows interface from within a game, but he doesn't necessarily agree. Thoughts please!

Theming, at least for GTK, would take care of this completely.

Quote:

My second point was that players probably don't want to see the Windows interface from within a game, but he doesn't necessarily agree. Thoughts please!

Definetly Agree. As games get closer to becoming a real simulation (more realistic), I don't think that players want a Win32 MessageBox informing them of some information - which I think would only draw away from the game's experience. This is the opinion of ONE PERSON, I would be surprised if real game companies haven't done some amount of testing to see whether consumers would like games to look and feel like Windows Applications.

I could be wrong, and maybe many people want their games to feel just like they're using Windows or Mac, but I'd be surprised.

His main beef seems to be with the cursor, and, specifically, the fact that not all graphics cards can support the hardware accelerated cursor overlay in 3D mode, so games resort to rendering their own cursor using graphics. Couple an often less-than-stellar frame rate, with the 3-frames-ahead rendering of Direct3D, and you get a pretty laggy experience compared to the standard Windows GUI cursor.

Now, this guy obviously isn't a graphics programmer, or he would know about these things. As an end user, he has a right to complain when things feel bad. However, he should complain to Microsoft and the hardware vendors, for limiting their (old) hardware, and implementing benchmarking optimizations (render-ahead), rather than game developers, who have to play the cards as they are dealt.

To be fair the guy also went on a bit about tooltips and mouse button assignments not acting as he expected, but what it seems to come down to is that the guy who writes the column is a software developer with no game development experience. He quite rightly sees that making your own GUI is reinventing the wheel, without realising that high performance graphics APIs give you a totally different axle to the usual. And, yes, I expect that the immersiveness aspect may well not strike a chord there either, since he prefers shorter game sessions, etc.

Taking the discussion in a different direction, how can we get around this. Can we, for instance, use viewports to render in-game aspects, platform GUI toolkits to render widgets, and theming to make them look snazzy? I mean, most of the UI components he's talking about (which, I presume, do not include HUD elements in FPS/action titles) do not update that frequently.

Any takers?

it should be a good exercise for every graphics programmer to write his own gui

at university we are currently working on a java interpretor and we are implementing the console version into a gui

well for me and a few comrades this has been a job for 15 minutes but most of the people there don t know the difference between a membervariable and a static membervariable nor that there are static memberfunctions


also the concept the parent/children hierarchy of gui elements is totally new to them
on irc everyone asks me how to do this and that ok so i explain everything and the result is they have no clue about the programming language they are using


back to the topic:

i think most developers don t want to use foreign code in their projects
i have a personal dislike to libraries like GTK since they don t use the hungarian notation which makes their code pretty ugly and difficult to read for me

another point is that many libraries use procedural programming which makes gui programming to complex
i am going to write my own gui implementation as soon as i need it
i have a concept, i know some gui libraries from GAME SDKs + MFC ....
so i don t expect any difficulties
another point is that most libraries don t feature everything you need

So, here's my thought...

First, let me start by saying that I'm not very fond of most of the standard windows controls. The thought of actually using these controls inside of a game seems like a horrible idea to me.

Perhaps before we get into games, we should reflect on the use of standard windows controls in regular applications. If you look closely at any professional windows application made in the last 5 years, you'll notice that many of the controls in there are custom controls. Sure, the round buttons and animated docking windows are obviously custom, but the rest of the program may look fairly "normal". But look closer and you're likely to find that the toolbar with that buttons that glow and pop up as you move your mouse over them are custom too. And so is that dropdown list with the flat look. And now that you think about it, that standard windows menus don't have bitmaps next to each entry and they certainly don't allow you to drag-n-drop to re-order the items in the menu. Yes, that too is a custom control.

The reality is that the standard windows controls are serviceable. That's it. They don't look pretty, and they seldom offer any frills. Indeed, these controls, for the most part, do little to aid the users ability to focus on the controls that are important or to make working with them a breeze. Instead, they do the bare minimum to present data to the user and allow them to edit it.

To make matters worse, extending these basic controls is often problematic. For example, let's consider one of the oldest, and most often seen, controls, the text box. Let's say that you want to create your own version of the text box that has a black background and white text, so that it matches the visual scheme of the rest of your app. No problem, you can create a derived control and override these. However, upon testing this new control in your app you'll quickly notice that the default blue color, which highlights selected text, is really hard to see against the black background. After numerous hours of frustration you'll eventually find that this can't be changed. You either have to handle all of the user interaction and drawing for your text box or make some kind of sacrifice. Or, you might turn to the RichEdit control, which I believe does allow this kind of customization, but it has it's own set of problems.

In my experience, customizing the standard controls to look and work the way I want them to has been a failure, more often than a success. Even when it is possible to do so, the solution is often ugly and quite possibly more work then it would take to just create your own version of the control from scratch.

So, why am I rambling on about the minute quirks of customizing windows controls? Becuase if you try to use these controls in a game, you'll NEED to customize their appearance. Throwing the standard buttons, sliders, and combo boxes into your game won't just detract from the overall immersive feel of your game, it'll make it look downright silly. This means that most of your controls will be need to be completely custom anyway. And if you're going to do that, the benefits of using GDI and the remaining customizable windows controls will most likely be outweighed by the cost in speed and troubles in getting those two worlds to play together.

However, having said all of that, I absolutely agree with the spirit of that guy's blog. As I alluded to above, many of the custom controls in windows apps today don't necessarily make it obvious that they're custom controls. Many simply provide subtle, yet beneficial, tweaks to the look and feel of the standard controls. Said another way, creating custom controls shouldn't imply a license to thow out all standards in the guise of misguided innovation. If you recall any of Kai Krause's(Famouse for Kai's Power Tools plugin for Photohsop) more abstract user interfaces, you'll know what I'm talking about.

In the end, creating really great custom user interfaces, whether for a game or for a regular app, isn't that hard. It just takes a bit of care. Afterall, like most innovations, it's often more of an evolutionary task than a revolutionary one. The key to success is to study existing implementations of what you're creating. Disect them, and figure out what makes them nice. Then figure out where they fall short. Now, create your own take on this which blends the expected with your own touches that make it even better. And don't forget to keep standards in place that are expected. For example, if you have undo/redo capability then Ctrl-Z/Ctrl-Y better be the hotkeys for this. Similarly, if you create your own fancy text box, you better make sure the user can still use Ctrl with the arrow keys to navigate by word and double click to select a word.

If you make sure to follow these steps it shouldn't be that hard to create an interface that fits into your game nicely that also feels just as familiar as "standard" windows apps :)

-John

^ Completly agree, and well said to boot.

I've heard this argument (The original one, that is) far too often and it's almost always from people who have an extensive experience in software development as opposed to games development. The amount of times I've seen an out-of-place, ametuerish-looking control that was only justified by 'It works, doesn't it?'...

Games need custom UIs to complete the experience. The second you see a windows-looking button or the stock-standard scrollbar, all of your immersion (Not to mention aesthetic enjoyment) of the game will be completly thrown out the window.

Some people have mentioned that this could be circumvented with skinning, but the problem here is that you're going to have to work incredibly long and hard to not only get them to work with your fullscreen hardware driven game and make them look/function right, but then if you want to port it to another platform all that tedious skinning work you've done will all have to be redone for the target OS. And what happens when you're requested to do a PS2 port? What UI are you going to use then? Oh, you're going to have to write one from scratch, which you could have done in the first place and saved yourself a couple of weeks of headaches.

Although I do understand where he's coming from with a user-interface perspective. Knowing which buttons to activate on mouse-up and which ones to activate on mouse-down (Before any GUI fundamentalists jump on me, there ARE a lot of circumstances when a mousedown event is more intuitive than a mouseup event, normally in effects-heavy UIs), and doing things like page-thumbing on scrollbars (And actually being able to thumb in the first place... whats the point of having a thumb bar if you're just going to use the arrows anyway!?).

In my personal experience, the whole design of the UI should be given to the artists BEFORE the programmers, allowing for an aesthetical consistency. While I'm aware that there are a lot of GUI-concious programmers out there, there seems to be an even amount that couldn't give a rats arse what they used as long as it was less hassle.

A GUI is going to be your players first experience with a game. Make it a good one, by any means necissary.

Anybody got any opinions on the technical aspects though? I mean, there are owner-drawn controls and themese etc which allow you to have a reasonable degree of control over how a control is drawn, while not surprising the user in terms of behaviour. Is there a reason why this is impractical beyond the immersion arguments?

Quote:

Original post by Kylotan Anybody got any opinions on the technical aspects though? I mean, there are owner-drawn controls and themese etc which allow you to have a reasonable degree of control over how a control is drawn, while not surprising the user in terms of behaviour. Is there a reason why this is impractical beyond the immersion arguments?

Itegration and re-intergration (And possibly a full rewrite on platforms that dont support the controls in question) is probably the best reason. If you're going to have to re-write them all anyway, why bother tinkering with someone elses controls at all?

That and getting them to work with your renderer well, and the tedious process of coercing the controls to look and act exactly how you want them (I'm intensly picky about how controls look and react to user input). Learning MFC is far from a trivial task, the Windows GDI is just god-awful to work with and .NET, IMHO, isn't really the best environment for games at the moment, especially if the codebase is already written in another language. Point is, you can normally write your own widgets in half the time it would take to learn how to use most of these correctly, and get them to run.

Edit: Also, final reason, I hate using SDKs for things I could write myself. DirectX is an exception due to its incredibly vast complexity (And the idea of trying to write my own driver to interface with the graphics card and get it to behave on all machines is just rediculously stupid), but using someone elses custom control wrapper irks me, because I always like to know exactly what's going on and know that if I need to make a major change it's going to be possible to do. I'm sure there are lots of other programmers that feel similar.

cross-patform GUIs are a pain, for sure, but skinning should not be a problem, if you already have good support for cross-patform rendering. The UI itself, which should be platform independent (both supporting joysticks AND mouse controls), should also suport multiple, concurrent inputs (split screen controls), multiple resolutions, aspect ratios (wide screen, or 512x512 for the PS2), and split screen, which is a problem for 4 player games (you only get to play with 256x256 portions, and it's got to be readable even on a crappy TV).

That's another layer to think about, on top of which dialog items should be implemented (lists, text boxes, buttons, radio butons, text inputs, scrollbars, ...), and the problems that inherits from using Joypad controls/keyboard instead of direct pointers (mouse or joystick ocntrolled pointers).

Sure, a couple of buttons take minutes to implement, but if you consider scrollable, dynamic lists of focusable items (for like list of options to tweak), trees of items, drag and drop (inventories for RPGs), and the whole FX management as well, focusing the right items, modal dialogs (controllers are pulled, or network cable, that sort of things) ... then it gets a lot more complicated. When doing menus for multiplayer games, those sorts of thing are necessary, and having a re-usable, skinnable, scripted croos-patform system is not just a comfort. It's got to be easy to code (or else, what's the point) too.

Where I work, there is no such thing, although we specialise in cross-patform online games, with support for split screen. I can tell you, everytime a new project starts, the U.I. is ALWAYS a nightmare, and we endup with 3 or 4 concurent but relatively similar systems to do different things (HUD, in-game menus, front end, loading screens...).

Strangely enough, we have an in-house editor (It's a layer on top of the game so you can drop in and out of the game and into the editor straight away), with a quite complex DirectX menu interface, strictly only for the editor, instead of the standard, well known Windows Interface, go figure.

I agree with hplus0603 about cursor rendering. Hardware mouse pointers can't generally speaking coexist with hardware 3d rendering. So games render them themselves (also some games use pointers which are fancier than the standard ones). But due to low framerates and double / triple buffering, there will always be some lag.

But as far as game widgets are concerned, I think that game designers *WANT* their own style of widgets. It adds to the game.

If you're making a fantasy game, use a fantasy font and olde-worlde widgets like scrollbars made of victorian wrought iron.

If you're making a futuristic game, use polished chrome bits etc.

Obviously some native GUIs are skinnable but that's quite OS dependent (EVEN between Win32 versions) and potentially unpredictable.

Games' fonts tend to be fancy and multicoloured and designed for theme purposes, but OS fonts tend to be designed for legibility. I'm not sure whether modern OSs can render multicoloured fonts.

Also stuff like RPG inventory screends tends to be designed to be easy to work with.

I also sort of agree with Peter Molyneux in "Black & White" - where he dispensed with the GUI entirely, not putting any icons, buttons, menus or anything else on the game screen at all. Of course the mouse gestures system was a bit tricky too.

Mark

Sorry, last post was mine.. seems I wasn't logged in... grrr....

GTK is skinnable to the most extreme degree. Take a look at some of the various themes. Porting GDK to GL would really work well. You'd get all the benefits of the toolkit as well. Copy & Paste. Accessability features.

I wish I had the time or the knowledge to do it. It'd solve a lot of problems.