[java] Java based UI for AAA games.

I guess. I mean, you can host the Java VM inside an application, if I remember correctly. I'm not sure exactly how you communicate with the VM, but I assume it's possible. I don't know why you'd do it though. It seems like you're just going out of your way to build a really terrible UI if you do it. You're a million times better off just using WxWidgets or something.

You can communicate with the VM through native (C/C++) code, but it's ugly and it has a bit of a learning curve. I would take Promit's suggestion and use something like WxWidgets instead of pulling in something as massive as a full JVM.

You can't use AWT or Swing for anything performance-central, since you have little control over repaint events, so you'd be writing the UI from scratch, and then it would likely be a nightmare to interface it since you'd have to somehow share the rendering context between Java and your game (likely limiting you to OpenGL), or calling native functions to draw your UI, meaning half your UI library will be in Java and half in C++ no matter what.

The only commercial game that embedded Java that I am currently aware of is Vampire: The Masquerade - Redemption, but I am sure there were others. But for UI, it's more trouble than it's worth, unless you plan to write the entire game in Java. Java does not offer anything UI-specific that you can benefit from. You can implement the observer pattern cleanly in Java, but this is something that can be solved with multiple inheritance in C++ as far as I understand. Take inspiration from AWT/Swing for the types of classes and methods you might need, but code the thing in C++ or better yet, use some existing UI library.