HTML5 for game UI?

I'm looking for a "poor mans" Scaleform solution

Scaleform sucks. Even non-poor companies are often looking into alternatives.

and after discovering Awesomium I am interested in the idea of using HTML5. I'm interested in any experiences of people who have done this, and what animation tool they used. Google Web Developer seems rather basic and doesn't support custom components, Adobe Edge apparently uses jQuery for all animation resulting in slow and large animations.

I haven't seen any decent tool. The big thing stopping us from looking at this at all right now is that:

a) Most of the tools are 32-bit only and at best have only fledgling 64-bit support
b) Platform support is terrible, mostly just PC-only
c) There are LGPL bits in WebKit/Blink, making it impossible to legally use on some platforms even if ports existed
d) Performance implications of multi-process model is scary, though not necessarily bad
e) HTML5 is general-purpose, meaning that like Scaleform/Flash there's a huge area of functionality HTML5 offers that you simply can't use in a game, and like Scaleform you will probably end up with a huge over-complicated UI library that you barely make actual use of outside of prototyping

Overall, the idea of using HTML5 is neat, but it's not going to happen in a large scale until someone writes some high-quality non-WebKit-derived middleware optimized for games. I'd like to see something like RAD Iggy but for HTML5; Iggy is a Scaleform-like Flash library that has a very stripped-down feature set, taking the approach of "if a game UI can't use it, don't include it." It's AS2 and missing gradients and so on that Scaleform supports but almost nobody actually uses in a production game due to the performance problems with using them.

You can maybe look at libRocket. It's just a basic HTML4-ish and CSS2-ish parser/renderer that you have to animate and control yourself (it has no JS engine and no CSS transformations/animation support). It's quite buggy and poorly implemented in my experience but It Works(tm) for smaller indie/hobby projects well enough.

CEGUI has also received some recent love and might be a good choice, but I haven't looked much into the recent versions. It looks like it has animation support and a GUI editor.

I happen to be browsing some other game dev news last night and I came across this article from Facepunch Studios, the guys behind Garry's Mod and Rust.

They are rewriting Rust and using HTML 5 for the UI.

http://www.pcgamesn.com/rust/rust-rebooting-heres-why

It seems as though they are embedding the chromium web browser into the engine:

The UI is one element that was always going to change. The default version was a placeholder. The new version, which is due to be functional and working in the experimental branch around this week, was rebuilt in HTML. That’s made development significantly faster.

“Because it's HTML it's really artist friendly, so shit gets done a lot faster,” Garry tells me. “It renders in another thread, so it has no performance impact on the game at all. It's a regular chromium web browser, so we can use a lot of existing technology like bootstrap, jquery and angularjs to make development faster. We're hoping that people will be able to skin it using regular CSS.”

You might want to ping them and see if they can give you any pointers on how to implement your UI system in a similar fashion.

Quite honestly I cannot image why, given a choice, would anyone ever want to use something as mysterious and complex and unfitting as HTML for game UI. I work with it every day, making websites, and it's not even so good for that.

• The rather neat CSS concept is brutally overshadowed by the abysmal choice and design of the properties it controls. Positioning alone depends on at least 5 of those (display,float,clear,vertical-align,position), many of them having multiple values and conditional shadowing of other properties, which makes the amount of possible combinations somewhat unwieldy.
• Animation issues do seem to be partially solved by external libraries, though the sandboxed environment does not allow for anything advanced without some serious effort.
• Memory usage happens to be a serious issue without decent tools. Leaks are extremely easy to create and persisting UI forms gradually may begin to suffer from that. Nothing wrong with making errors, it's a natural part of the process. Debugging them gets to be a pain with what the browsers have to offer at the moment, though.

Haven't used the Flash toolkit but from what I've read about it, artists can draw and animate stuff easily there so it makes at least some sense to use it.

I do have to confess though, I'm working on a UI library myself - sgs-ui. MIT-licensed, lightweight, only real dependency is the SGScript scripting engine. Can't really suggest it for heavy projects at the moment (it simply doesn't support the massive set of text rendering features and regional quirks yet) but it's about to cover small-to-medium size projects quite well after about a week when I'll finish fully separating UI control code from the only backend that currently exists.

Video demo showing most current features: http://screencast.com/t/yPZ7nV3zYKL

Code for the demo, all of it: https://github.com/snake5/sgs-ui/blob/master/examples/sgs-sdl.examples.sgs#L158

There are no other definition files but the nature of the scripting engine allows to pull every structure or utility function out to some other script file to clean things up. For example, to load control layout from a JSON file, simply add/include the JSON library and do this:

UI_BuildControls( FRAME.root, json_decode( io_file_read( "mylayout.json" ) ) );

---

I also noticed that CEGUI was mentioned before... well, for a comparison, this is CEGUI code:

https://bitbucket.org/cegui/cegui/src/b1d05ad3449859d0676c807cb6b961f312d26ff9/samples/GameMenu/GameMenu.cpp?at=default

Considerably more complex and depends on a lot of external definition files, here's one of them:

https://bitbucket.org/cegui/cegui/src/b1d05ad3449859d0676c807cb6b961f312d26ff9/datafiles/layouts/GameMenu.layout?at=default

- needless to say, far from the best use of XML - to replace property attributes with subnodes for doing exactly the same thing.

as mysterious and complex and unfitting as HTML for game UI.

WYSIWIG editing, lots of tooling, choice of a few mature layout engines, lots of documentation, powerful features (dependent on the layout engine).

I'm not interested in coding HTML, but in using a WYSWIG editor + Javascript to quickly iterate on UI, it doesn't matter if HTML is mysterious or complex, because I am never going to see it.

Haven't used the Flash toolkit but from what I've read about it, artists can draw and animate stuff easily there so it makes at least some sense to use it.

Yeah, but the latest HTML5 tools are allowing the same thing. Even Adobe is moving away from Flash to HTML5. Adobe Edge Animate is the HTML5 equivalent of Adobe Flash Pro. In fact the link I posted previously shows this and some others.

They are rewriting Rust and using HTML 5 for the UI.

Looking into this a bit further today, it looks like Chromium forked Webkit to Blink, I am going to see how far I get with that, I'm not sure I would want to try to integrate the whole of Chromium!

WYSIWIG editing, lots of tooling

The best editors are really quite horrible. HTML was never meant to be edited like that and it shows really well. You will have to see each and every bit of it because most things the WYSIWYG editor does will be a gross misunderstanding of intentions. I have worked on a in-browser WYSIWYG editor myself (had to fix some long-running usability issues in TinyMCE) and it just amazes me how anyone could even think that it is the future. It was already tried a long time ago and didn't work out. Why should it work now, when the standards have only gotten a lot more complex?

WYSWIG editor + Javascript to quickly iterate on UI

Any scripted UI toolkit offers pretty much the same thing. This is not only not exclusive to HTML but same or better with any other scripted UI toolkit. Mostly due to the fact that there are many features that HTML simply does not support (reusable layouts, behavior callbacks, compound coordinate system, proper vertical alignment - all that would seriously reduce the necessary amount of nodes) and cannot be added there.

Even Adobe is moving away from Flash to HTML5.

I didn't really understand that decision. Perhaps they were further from technological stability in Flash than I thought, or lacked the resources to make it happen. Maybe they just decided to ride on their strengths (editing apps) and let others carry the weight. Their latest 3D stuff was quite bad from what I've heard so it would make some sense to judge future Flash performance based on that. Either way, that doesn't really say anything solid for HTML5, except that Adobe executives believe in it more than they did in Flash. Judging by the performance of both software stacks, it's purely a judgment call at this point.

So what do you think of the editors in the first link I posted? They seem to offer the same features as Flash but generate HTML5, CSS and Javascript.

They aren't really comparable to in browser editors designed for making simple web pages I don't think.

I tried out Google Web Developer, and it isn't great, but I did manage to make some nice tweened animations in it in less than 5 minutes. And it actually uses HTML and CSS as its data representation, rather than just generating it as a build/publish step.

My point was that the problem isn't as much the tools as it is the technology. Though I did test both Google Web Developer and Edge Animate and here's some things I noticed:

• There is also no pipeline for passing unknown amounts and kinds of data to it. Best example for this - any options menu. The UI controls need to have data from game settings files passed to them. There actually need to be some UI controls to reuse and apply styles to. None of that could be found.
• Scalability of UI may very well be an issue since I didn't find any way to make it scalable, so one might have to make do with the zoom/pixel ratio properties. Also, things sure get busy when multiple screens or sliding panels are required. Editing such things won't be easy so sticking to fully separated menu screens might be an option.

See, making animations isn't the issue here. Both tools deal with that just fine for most games.

They aren't really comparable to in browser editors designed for making simple web pages I don't think.

Yes, but only because these things use a very limited subset of HTML5. All block elements are absolule-positioned and the textboxes don't seem too advanced themselves, using only the most basic inline nodes. I didn't find a way to add tables or floated images, for example, which are useful for many UI designs.

They seem to offer the same features as Flash

No, far from it, and this was quite a funny revelation for me. I didn't find a single way to design forms - add data input controls, wire them up to some actions. All that still had to be done by hand, in plain HTML/JS/CSS.

What I did today, that may be of some interest for anybody else attempting this:

Investigated the Chromium Embedded Framework, a simple way to integrate Chromium in to any application.

CEF3 is provided as prebuilt binaries, but unfortunately not for VS2012. So I had to build my own.

Like usual its an open source project using an obscure build system, and a custom project generator. Luckily they have a simple option which will, with a bit of tweaking, do the whole get->generate->build->deploy step. See https://code.google.com/p/chromiumembedded/wiki/BranchesAndBuilding and look for Automated Method section.

NB: There is an error in the automate.py script, in that it references svn.bat, which doesn't exist. You can modify it to point at svn instead and then it works fine.

Make sure to get from one of the stable release branches (I used 1750).

It takes about an hour to get and build. Also you have to specify various options to get what you want out of automate.py. My build command looked like this:

set GYP_MSVS_VERSION=2012

svn checkout http://chromiumembedded.googlecode.com/svn/branches/1750/cef3/tools/automate cef3\tools\automate
python cef3\tools\automate\automate.py --download-dir=cef3 --url=http://chromiumembedded.googlecode.com/svn/branches/1750/cef3 --ninja-build --verbose --x64-build --force-build

Obviously you need Python and SVN already installed and in the PATH to bootstrap this.

This gets you prebuilt libs, required dlls, and a couple of sample client applications. It also includes a project that can be used to build a wrapper library with custom build settings (for instance if your own project uses /MT you need the libs built with the same flag).

The framework exposes all you need to communicate from C++ to Javascript and vice versa by way of extensions (for the former) and executing Javascript commands (for the latter).

Edit: actually extensions can only be used in the render process, CEF provides an asynchronous callback system for calls from Javascript to C++ (https://code.google.com/p/chromiumembedded/wiki/GeneralUsage#Asynchronous_Bindings).

Interaction is handled simply by passing mouse and keyboard events to browser objects.

Obviously CEF3 allows offscreen rendering otherwise it would be useless for this task, and it also supports transparent backgrounds for offscreen rendering so you can composite UI into your framebuffer.

A very useful find was this thread relating someones experience of doing the same thing I am doing: http://www.ogre3d.org/forums/viewtopic.php?f=11&t=79079.

Next is Edge Animate. I found out there is a fair amount of support for this on the web, including a nice video on how to dynamically generate content (

). Edge also has the concept of Symbols (reusable components) which is going to make menus and such like much easier. Unfortunately the concept seems to be an Adobe one, I suspect other editors probably don't have the same facility (maybe something similar though). The UI is familiar to anyone who has used Adobe Flash CS, animation is powerful, adding script is easy, and minimum data for a page isn't excessive (200kb for my test page).

So far so good I think, but if I run in to any show stoppers I will update this thread.

Dartlang (API) / StageXL (lib)

This outputs HTML5 but has the same key assumptions as would Actionscript 3. Makes it very easy to manipulate the stage.

UI is almost fun to work with :)

I happen to be browsing some other game dev news last night and I came across this article from Facepunch Studios, the guys behind Garry's Mod and Rust.
They are rewriting Rust and using HTML 5 for the UI.

http://www.pcgamesn.com/rust/rust-rebooting-heres-why

It seems as though they are embedding the chromium web browser into the engine:

Garry Newman is using Coherent UI in Rust, which is HTML5-based multi-platform user interface engine for games. There are already hundreds of projects using Coherent UI which proves the stability of HTML5-based plugin in game UI. You can check Planetary Annihilation by Uber Entertainment, which UI is 100% powered by Coherent.

What I did today, that may be of some interest for anybody else attempting this:
Investigated the Chromium Embedded Framework, a simple way to integrate Chromium in to any application.
CEF3 is provided as prebuilt binaries, but unfortunately not for VS2012. So I had to build my own.

Integrating CEF is a good alternative, especially for small projects. We've investigated it in a details and wrote an analysis on What should developers consider when using Chromium Embedded Framework (CEF) in their games? I would love to hear your feedback on it. Please find it here

Integrating CEF is a good alternative, especially for small projects. We've investigated it in a details and wrote an analysis on What should developers consider when using Chromium Embedded Framework (CEF) in their games? I would love to hear your feedback on it. Please find it here

Yeah I came to the same conclusions as you after looking into CEF for a few days. The lag, and the cost of pixel transfer make it fairly limited for all but the simplest UI. The system you have is exactly what I would like, but unfortunately way out of my price range for a hobby project. I also looked briefly at Awesomium, including downloading and running their demos, and they barely offer anything beyond what CEF offers as far as I can tell. They still use multiple processes, and can't handle rendering more than one fullscreen target at a time.

My solution is probably going to be rolling my own layout and rendering system, with Javascript V8 integrated. Start with something simple and then developing it in parallel with the rest of my project. Obviously it won't cover all HTML5 and CSS3 features, but then they aren't all required for a game UI. I do think that Adobe have it right with Flash and Edge Animate, with respect to their "everything is animatable, and all actions are animations" approach. Once I started thinking of UI as a state graph with animations along the edges it starts to seem like a very powerful and coherent approach (although maybe it is an oversimplification for some scenarios).