Proposing an Open-Source Dynamic Physically-Simulated Character Animation Library

I'd also love to see something like this. I've used CAL3D before in past projects and aside from the pain in the butt exporters, the API itself is fairly clean and simple to use. But combining CAL3D and a physics library would be a great replacement for Euphoria, at least for those who can't afford it.

I would find such a project quite interesting as well, but at the moment I have way too much to do working on my bachelor thesis.

Anyways, here are a couple of links to articles that might be of interest:

http://www.cs.ubc.ca/~van/papers/#2007-siggraph-simbicon
http://www.cs.ucla.edu/~pfal/papers/cag01.pdf
http://homepages.inf.ed.ac.uk/tkomura/casa05.pdf
http://graphics.cs.williams.edu/papers/DynamoVGS06/
http://www.kuffner.org/james/papers/komura_vrst2004.pdf
http://grail.cs.washington.edu/projects/charanim/
http://www.cs.ucla.edu/~vector/files/SCA07_Control.pdf
http://graphics.cs.brown.edu/games/Dynamo/SIGGRAPH06/sig06-world-space-sketch.pdf
http://www.cs.ucla.edu/~pfal/papers/siggraph2001.pdf
http://www.cs.ucla.edu/~pfal/papers/phdthesis.pdf
http://graphics.cs.ucr.edu/papers/zordan:2007:IDR_DRAFT.pdf
http://grail.cs.washington.edu/projects/styleik/styleik.pdf

Hey again all, my apologies for the delayed response.

I'm ready to start getting moving on this project. My thoughts are that it be hosted on SourceForge. If anyone has a better idea, please shout it out. The license should be unrestrictive except, I think, for encouraging collaboration on developing the library. This area is admittedly under-researched on my part, but unless anyone provides a good argument for a different license (if you have one, PLEASE do speak up), my thoughts are that the project be started under the LGPL license. Otherwise I fear indefinite procrastination due to not knowing what license to use, and that would be a shame!

Again, my current thought is to base the library on the well-established open-source library CAL3D. It's too early to be sure that this will go smoothly, or if we'll find a need to make considerable changes to CAL3D, or even a need to define a separate character animation system if it turns out that CAL3D simply won't work...though I don't expect that will be the case (and indeed I hope not). Since I personally only have direct experience with ODE, if I'm to start the project myself, that would be where I would begin...though I would plan soon to switch code over to be based on PAL, since it seems to be the most extensible move. If anyone thinks there's a better way to do things regarding dependencies, do please speak up.

As for what exactly would be included in the project, my thoughts are that it at its core would be an intermediary interface between the character animation library and the physics library, with data structures reflecting the bones and model found in the character animation library and the joints and geometry found in the physics library. Without all of the design details worked out exactly, the idea I have is to have this layer act as a kernel through which all information relating to animation can flow, and which can act as a filter.

For example, it should be feasible to set up a system where canned animations are played, but filtered through the kernel before being visualized by being written back to the CAL3D structures. What the kernel would do with this information is communicate with the physics library as appropriate to establish that the animations can be performed as requested -- you can think of this process as redirecting the movement through the physical world before executing it proper. If bone geometry collides with a wall while attempting to perform the animation, the final animation presented would be altered accordingly, while still attempting to continue to play to its completion (unless the application, upon learning of the change, chooses to discontinue the animation). The end result would be, for example, trying to punch through a wall but instead being stopped by it.

Pumping animations through this kernel, when properly tuned, should similarly allow one to easily enough play any number of standing animations that will automatically adjust to standing on arbitrarily shaped terrain under a model's feet, bending knees and ankles as needed in order to continue to maintain balance. (In practice though I'm sure this will also require some sort of feedback AI generating animations to be blended in). This filtering should of course be adjustable, as there may be times when you want to use completely physically-based animation, times when you want completely canned animation, and times when you want them to interplay, in yet unanticipated very specific ways I'm sure.

In addition to that, we'll need some means for specifying exactly how one wants to build the physical representation of the body. Though it's easy enough to generate, I'm sure in most cases we do NOT want a one-to-one correspondence between CAL3D bones and physical bones, and going with a default assumption that every joint is a ball-and-socket joint won't do either! So we'll need some way to build a physical bone/joint hierarchy, probably stored in an easy-to-edit and easy-to-merge text format (xml I expect). Once this file format is well-established I expect it will be appropriate to build a nice wysiwyg tool for taking a CAL3D model, examining its bone structure, and building a physical bone structure from that -- and exporting in our established file format.

Finally, the top-most level of the library (and arguably the funnest part) will be the artificial intelligence interface -- logical patterns for generating animations procedurally. We'll have PID controllers and the like for controlling characters to maintain constraints via feedback loops, and I imagine that probably the first example in this department (once the rest of it has all been sufficiently implemented) will be a little dude standing that you can push on, but who will react in order to simply maintain his balance. Here we can have fun also mixing in some randomness for more lifelike animations...and indeed, at this level writing character animation logic may be analogous to writing short shaders to flavor animation. More sophisticated environment interaction will be an interesting research problem, and will probably best be handled at yet a higher level of AI processing, perhaps to be tackled in a separate library all of its own.

Now all that being said, before we get this thing a proper home, I'd prefer to have its name squared away...so I'd like to begin a discussion to that end. I personally am big on acronyms, and two potential names that I've come up so far are REACT (Robust Emergent Animated Character Technology) and DASH (Dynamic Animation Simulation Hierarchy). If we choose to play directly off of DANCE (a good name, but alas already taken by those maddeningly-selfish researchers! Prohibiting commercial use? Damn them and their selfish altruism!) my first thought is PRANCE (*PRocedural* Animation aNd Control Environment).

Thoughts?

Here's a news update for anybody still following.

Currently I am favoring the name REACT (Robust Extensible Animated Character Technology), a variation of one of the afore-mentioned name possibilities. I am also leaning towards the MIT license. Though I might perhaps like a license that explicitly encourages people to contribute back to the project, there's really no way to force people to do so, and I have beef with some aspects of the LGPL (largely that the dynamic-linking aspect won't hold up well for most embedded systems such as many game console platforms). The MIT license is simple, and imho a bit more clearly worded than the BSD license.

Now, the only remaining issue before beginning the project proper is determining the most favorable site to host the project at. Right now I'm looking between sourceforge and google code projects. They each have their strengths and weaknesses...I'm just not yet sure.

Suggestions/commentary?

I would agree with the notion that an MIT/BSD-style License would be preferable compared to LGPL. Personally I'd choose BSD over MIT but the difference is quite small.

I don't really like the name REACT (seems a bit nonsensical), but I don't have any better suggestion either.

Anyways, the interface of Google Code seems much cleaner than Sourceforge, and SVN is standard, so I'd go for Google Code.

PS: How would CAST (as in theater) do as a name? (Character Animation & Simulation Toolset)

Actually the dynamic linking limitation was relaxed in LGPL version 3.0 . But I would agree that BSD is better than MIT as far as open-source licenses go. Subversion is also offered as an option on Sourceforge.net also.

PAl as I heard is not really that useful yet. My experience is with PhysX, it's an interesting project and similar to something I'm planning to R&D once I can clear my work too.

Quote:

Original post by Void PAl as I heard is not really that useful yet.

Did you try it? Abstracting away the actually used physics library is always better. Especially in something like a character animation library. Ideally the user should be able to use any physics library in conjunction with the character animation library.

Quote:

My experience is with PhysX, it's an interesting project and similar to something I'm planning to R&D once I can clear my work too.

Dunno about PhysX, but Bullet seems nice (and is open source)

Hi,
I also have interest in working on something similar and seeing what can be practicallz achieved with current technology and processing power, although i was planning to limit this to a theoretical (make use of dead time) mind exercise (http://www.gamedev.net/community/forums/topic.asp?topic_id=500206) because unfortunate of lack of time... but in any case im keeping an eye on this ...

I'm happy to see the growing interest in this project. For licenses, I have reread the BSD license again and I must say it is growing on me. Just to be clear though, is the only real difference between the (current) BSD and MIT licenses the "don't use our names to endorse your product" clause in BSD? Because I've seen MIT with a similar clause appended...Are there any other real differences that I may not have realized? My curiosity is piqued since multiple people here have expressed a preference for BSD over MIT, so I want to be sure I understand why.

As far as names go, it may be worth explaining that the name REACT was proposed as referring to a character responding dynamically to a situation, and mentioning that I've been revising the acronym a bit on recommendations -- the current iteration was Robust Extensible Animation Control Technology. (The Control is in there referring to control theory, as I realized the "Animated Character Technology" didn't in any way suggest that anything dynamic was actually going on.) I must say l0calh05t, I do like how Toolkit fits in well describing this library (as there still is some assembly required, but it'll give you what you need to do a lot), so I'd then further adapt the name to be Robust Extensible Animation Control Toolkit. CAST is good too -- I like the stage reference. What do other folks think?

As for hosting, I'm starting to lean towards google code projects now. It sounds much easier to deal with and faster to get moving than source forge, and I kinda like the idea of encouraging competition by trying out the new competitor. The only thing I'm potentially concerned about is the lack of support for some advanced features, such as being able to do server-side build steps on commit...It would be nice to automate some unit testing and to auto-generate and host documentation and the like. It's possible that google may yet unveil this support sometime soon though, and the project could always switch hosts later on if deemed appropriate.

Before I move on for the day, here's a simple diagram I whipped up that maps out some of the (high-level) design I had in mind so far for the library:


The crux of the dynamic animation library (the one being proposed here) is the kernel, which manages the internal state of information and acts as a central intermediary between all other bodies of code that seek to manipulate or request that information.

I realized that much like how the library does not need to be dependent on any particular physics engine, it likewise does not have to be dependent on any particular character animation library -- thus enter the bridges. Implementations following the design pattern of the same name, these would be bodies of code that connect two distinct libraries, responsible for directing information between them, so that the two libraries need not depend on each other. With the exception of the controller bindings (although, if there do perchance happen to already exist a few standards for defining controllers, as may be the case amongst researchers, it would be quite cool to have a bridge for each...) these bridges would be add-on libraries that are not part of the core but that are still essential for functionality (like, say, drivers in an operating system). I expect that they would still be part of the repository code base, but would be built as separate libraries to be optionally linked in as needed. Some application would then depend on the core (consisting of the kernel and core drivers) and any other necessary non-conflicting drivers, and whatever other libraries those drivers connect to.

Note that we could just as easily have a bridge to PAL as to any specific physics library, so we wouldn't require folks to depend on PAL if they don't like it (and prefer to go straight with Havok, for example).

Thoughts?

I really like the Bridge idea. I think that would work perfectly, as long as the Kernel is completely independent of any particular bridge implementation. What exactly are the Controller Bindings for? I'm not familiar with what that exactly does.

That's my intention -- that the core library (the piece including the kernel) would only define interfaces for the bridges, and make no further assumptions about them other than that they implement the bridge interface. At the moment the plan is for three types of bridges, which I'm sure would require separate interface designs (though there may be some elements they share in common) -- those to skeletal character animation libraries, to physics engines, and to controllers.

About the Controller Bindings, these basically serve as the link between the kernel and the controllers. The controllers are bodies of code that drive dynamic animation, and really could take any of a number of conceivable forms, though I had been assuming that they would quite frequently be scripts -- hence the use of the term "bindings." I realize now that it's probably better to just follow the more generic term and file this as just the third type of bridge, since you might want your controllers to be compiled code or in any previously conceived format.

The design of this interface is definitely still in the research phase, a lot of which being wading through research papers to see how other people do things and to try to assess what sort of standards are already out there. I'd definitely appreciate any help in this area (thanks already for those links, l0calh05t!). I know DANCE has their own format, which seem to be arbitrary hand-coded controllers or simple descriptions of static poses that straight-forward PD controllers (feedback loops that take derivatives into account) are used to drive between, with python scripts to compose the various aspects of the simulation together (models, collision data, actuators, etc). What NaturalMotion uses for Euphoria is a mystery to me, though I understand one thing they do is run many generations of genetic algorithms to "grow" effective animations.

One thing that the DANCE people write about is requiring that the controllers have full read access to the current model state, and that they implement an interface for assessing if pre-conditions are met, and for determining whether the controller is succeeding or failing (so that a higher-level controller, if you wish to arrange a controller hierarchy of some sort, could assess whether to switch behaviors). My first thought is that they be simple agents that in the abstract just reflect on the current status of the model at large (and any other data they are privy to) and issue commands to the kernel that call to apply torques at joints as desired.

Thoughts?

For character animations to take off there have to be a few key elements:

Standard skeletons, standard file formats, compatible mesh formats (preferably those used by easy to use free tools) .

An easy to use skeletal animation editor (including nifty interfaces like the graphical cyclic pattern maker, and joint action limiters)

A place people can publish their (now standard) animations so that they can be reused by others (ditto for the scripting for multisequence actions where associated animations are tied together for real uses).



Like so many others I have a need for hundreds of animated actions, but with all the work that goes into creating one I cant have enough time to do just that (ontop of the rest of the simulation engine/AI system which is the primary project). As far as Ive seen the 'industry' hasnt figured out any standards or organized a way to open up the creation process to amateurs.

There are thousands of common gestures and actions to be created and having the creation process made easier and within the reach of the multitude will do for animation what paint and photoshop did for 2D texture production.
Having them publish openly allows incremental development where one persons efforts are incrementally improved by someone else.






A seperate issue: There is some interface between the physics engine and the terrain mechanism (since the surfaces are required to be defined for the contacts/collisiones and friction factors, etc... That information requires standard terrain information components for the objects to interact with.

I love this idea, and I registered just to post my support for it. I'm not at a point yet where I feel like I could really help out, but I at least wanted to show my interest. Please keep us updated on your progress!

Also, if this helps . . . for something like this to be really useful to me, it would need:

1) Easily changeable animation and physics libraries. (Your bridge concept seems to take care of this . . . for example, I know you're targeting Cal3D and ODE, but I would want to use this for Horde3D's animation component and Bullet, and with your current concept it seems that would only mean writing new bridges, which should be fairly straightforward.)

2) A simple C-based API. Internally, you can have as much object oriented C++ as you like, but the most useful game libraries are the ones with the simplest APIs, and having a simple declarative C API is especially useful. First, it makes it easy to work with the library from other langagues (almost any language can call C easily, but calling C++ is considerably more difficult and sometimes impossible) . . . personally, I would be working with the library from a .NET context, and I know there're also sizable communities of Python and Java game developers. Second, having a simple exposed API actually encourages good programming practices internally and encourages plenty of thought on good encapsulation and abstraction. For an idea of the kind of API I'm talking about, consider Horde3D's (http://www.horde3d.org/docs/_api.html, good) vs. OGRE's (http://www.ogre3d.org/docs/api/html/classes.html, bad) APIs. As a result, developers have turned out Python, D, .NET, Java, etc. bindings for Horde3D that are only a few hundred lines of code, whereas wrapping OGRE has been a huge ongoing project for years that has never been fully successful (with the exception of the fairly mature Python-OGRE).

3) Some kind of functionality to support inverse kinematics solving. There's not much out there now for this in terms of opensource but commercially useful game libraries. Havok Complete seems pretty much perfect on this front, but of course it's not commercially available for a reasonable price: http://www.havok.com/content/view/19/32/. Some potentially useful links:
http://cg.cis.upenn.edu/hms/software/ikan/ikan.html
http://www.cours.polymtl.ca/roboop/ (this is actually LGPL'ed and possibly useful, although obviously targeted at robotics rather than games)
http://sourceforge.net/projects/dynamechs/
http://www.kuffner.org/james/papers/ (nice IK papers and some alternative methods)

Mostly I just wanted to show my support . . . this wasn't supposed to come off as a list of demands! Anyway, I really hope you keep going with this project, and I might be very interested in helping out at some point in the future.

Quote:

Original post by CoreyWhite 2) A simple C-based API. Internally, you can have as much object oriented C++ as you like, but the most useful game libraries are the ones with the simplest APIs, and having a simple declarative C API is especially useful. First, it makes it easy to work with the library from other langagues (almost any language can call C easily, but calling C++ is considerably more difficult and sometimes impossible) . . . personally, I would be working with the library from a .NET context, and I know there're also sizable communities of Python and Java game developers. Second, having a simple exposed API actually encourages good programming practices internally and encourages plenty of thought on good encapsulation and abstraction. For an idea of the kind of API I'm talking about, consider Horde3D's (http://www.horde3d.org/docs/_api.html, good) vs. OGRE's (http://www.ogre3d.org/docs/api/html/classes.html, bad) APIs. As a result, developers have turned out Python, D, .NET, Java, etc. bindings for Horde3D that are only a few hundred lines of code, whereas wrapping OGRE has been a huge ongoing project for years that has never been fully successful (with the exception of the fairly mature Python-OGRE).

I believe a C++ API would be much better/cleaner. It would still be possible to add C bindings/wrappers (this is how SFML does it, and imo that's the best way of doing it)

Quote:

I believe a C++ API would be much better/cleaner. It would still be possible to add C bindings/wrappers (this is how SFML does it, and imo that's the best way of doing it)

I disagree, but I'm not experienced enough to provide a strong argument. You may be right. Everything I'm about to say is my opinion, based on an admittedly incomplete understanding.

I certainly don't think there's anything innately wrong about a C++ API depending on what the library is going to be used for, but if flexibility, cross-platform support, and ease of developing bindings for other languages are important, then a C API seems a better choice to me. On the other hand, if the C++ API were to stick to core language concepts (which means even avoiding things like STL in the API, since its implementation can vary so much) then it should be possible to wrap it from most languages.

My experience with this comes entirely from a C# perspective, and I can't say too much about how things might vary for other languages. I do know, however, that a small C API (like Horde3D's) allows bindings in the .NET language using Platform Invokes rather than requiring a full-blown managed C++/CLI wrapper. Why is this important? Because the Mono project, which makes C#, VisualBasic, Iron Python, etc. portable to Linux and OS X does not support C++/CLI and has no plans ever to do so. This is why OGRE cannot be successfully ported to Mono, and therefore can't be used from a C# context on Linux or Mac OS.

If C#/Mono support isn't a priority -- and in fairness, there's not necessarily a reason that it should be, although I do think C# is an up-and-coming language for indie game development -- then this isn't much of a concern. (Although it does mean the library would probably be useless to me, personally.) As I said, I don't know how much of that applies to Python or other languages. On the other hand, a C API also makes it easier to port something to new platforms (iPhone, consoles, etc.) that don't necessarily support the STL and other standard C++ libraries. I also firmly believe in encapsulation, even at the library level, and restricting to a C API and making a small, simple interface a high priority helps hide information on implementation and makes things much easier for end-users.

I think you guys are basically arguing for the same thing really -- to have an internally C++ library with a C API trimming. I've seen this done before, as in ODE for example, and I definitely see why a C API would be a lot easier for other languages to bind to than a C++ API would be. However, I don't see the harm in exposing the C++ API as well as the C API, so that C++ programmers would have the option of using the C++ API directly. So I think the way to go is to essentially go ahead and write it as a C++ library (making extensive use of the modern engineering practices we know and love like the various design patterns), but continually maintain a C wrapper API that other languages can more easily bind to. In the end I don't expect the kernel to be very big -- it's mainly just a housing for the various bridges and the central pump for data flow -- so maintaining this C wrapper won't be so daunting a challenge as, say, wrapping up a massive scene graph library.

While C#/Mono support hasn't explicitly been set as a priority, I certainly wouldn't want to stand in the way of improving the accessibility of this library. To that end, I will definitely add the instating of a C API wrapper (not replacing but in addition to the directly exposed C++ interface) and the option of building custom containers instead of using the STL to the list of priorities for this project.

In response to your other points CoreyWhite, Horde3D and Bullet would indeed be great libraries to bridge to. Once the framework is in place and published, I hope the community at large will assist in developing all the bridges we could ever need. (: They'll be standardized and maintained within the same repository as the core. I see inverse kinematics as a control problem, so it's beyond the scope of the kernel, but possibly not the core at large. It might be most appropriate to handle problems like IK in a higher level control library that the kernel would bridge to, and since the exact scope of the control aspect of this project hasn't been fully fleshed out, I can't really say how it would best fit in for now -- certainly getting the infrastructure in place and working is the more immediate priority...But I'm sure it will find its place, and it's likely that it will be in the same repository.

Going backwards to wodinoneeye, you bring up some interesting points about standards. As far as I know, this is entirely new territory, and much work has yet to be done to arrive at such standards. Perhaps here is a good place -- what ideas do people have? My first thought, for standardizing an animation format, is to have keyframe animation, where each keyframe is a list of torques/forces applied to joints. This would probably best be coupled with some sort of judgement thresholds that deem whether we're remaining successful or if something has gone awry and we should abort.

With regard to physics and terrain, it seems to be common for physics engines to each have their own standard way of creating collision meshes for heightmaps (and other irregular 3d geometry). I'm confident that this problem is already taken care of and does not need to fall within the scope of this library.

Finally, I've some good news -- I may be able to get funded for this project! I work as a programmer in the Serious Games industry, which means I write training and simulation software comprised of 3D real-time environments (basically non-entertainment video games). I've been discussing this project with my employer, who definitely sees the use of ragdolls in our applications, and who's been gaining interest in being able to have characters dynamically perform some agile maneuvers. So, they might potentially be willing to let me work on this project on company time, essentially funding its development. It's not at that point yet, as they would prefer that I write something tailored specifically to the engine we use, Delta3D (which internally uses CAL3D and ODE). However, my intention is to whip this up and then massage it into a more general purpose library, and then have it publicly published.

[Edited by - Danathus on July 17, 2008 3:49:29 PM]

Where is the project link？I support you to do more about it.:-)

anything ever come of this? its an ambitious project so i suspect no.