🔒 Locked 🎮 Unity

Skeletal animation with COLLADA

Started by havsmonstret Apr 13, 2015 at 6:37 PM 2 replies 5.4k views

Original Post

Apr 13, 2015 at 6:37 PM

Hello. I'm trying to implement skeletal animation and loading my model and animation from COLLADA files. I've read all the data from the file and I've got my bones and vertex weights set up. Currently I set all the bones' transformation matrices to the identity matrix and then translate the root bone to see that that works.

I apply all the bone transformations with the vertex weights in my vertex shader (GLSL):


mat4 animationTransformation = TRANSFORM_Root        * vertexWeights0 +
                               TRANSFORM_Head        * vertexWeights1 +
                               TRANSFORM_Shoulder_L  * vertexWeights2 +
                               TRANSFORM_Arm_L       * vertexWeights3 +
                               TRANSFORM_Shoulder_R  * vertexWeights4 +
                               TRANSFORM_Arm_R       * vertexWeights5 +
                               TRANSFORM_Upper_Leg_R * vertexWeights6 +
                               TRANSFORM_Leg_R       * vertexWeights7 +
                               TRANSFORM_Upper_Leg_L * vertexWeights8 +
                               TRANSFORM_Leg_L       * vertexWeights9;

gl_Position = vertexTransformation * animationTransformation * vec4(vertexPosition, 1.0);

And then I modify the TRANSFORM_Root matrix and upload it:


animation.getJoints().get("Root").setTransform(new Matrix4(new float[][] {
    new float[] { 1.0f, 0.0f, 0.0f, 3.0f * (float)Math.sin(t) },
    new float[] { 0.0f, 1.0f, 0.0f, 0.0f },
    new float[] { 0.0f, 0.0f, 1.0f, 0.0f },
    new float[] { 0.0f, 0.0f, 0.0f, 1.0f }
}));
shader.updateUniform("TRANSFORM_Root", animation.getJoints().get("Root").getMatrix(), true);

This works fine and the whole model translates nicely along the x-axis.

What I don't understand is what data to actually use for what from the COLLADA file.

Each bone has it's own transform in the skeleton definition (visual_scenes). Further I have a bind shape matrix (read from bind_shape_matrix) which is just the identity matrix in my case, a "skin bind-pose" matrix for each bone (read from library_controllers -> skin). They are specified as "inverse bind matrices". I also have animations for each bone, specified by a list of times and matrices. I understand I should interpolate between these different animation matrices.

But how do I put all these matrices together for uploading to the graphics card? I've read a couple of tutorials and they are quite unclear on how to do this (or don't use COLLADA at all), most just have a "bone transform" which is used to transform the vertices. But how do I calculate this transform from the bones' transformations, the skin bind-pose matrices and the animation matrices?

Here's the COLLADA file if anyone wants to take a closer look: https://gist.github.com/anonymous/da2c3f57140c1a5beb1b

Sorry for the nasty code.

Thanks in advance!

NumberXaero

Apr 14, 2015 at 1:35 AM

Im going to refer to bones as joints, where a joint is a local xform (transform, xf), which when combined with its parent joints world xform, produces the joints world xform, Where a transform is often a matrix, or individual translation, scale, rotations that make up the xform.

Youll have to sort out the matrix conventions based on what you do in your code and what you get from the file. I would get it working on the cpu and then moving the skinning to the gpu when you have it working.

joint_local_xf = given from file, an offset from the parent joints xform (no parent, local = world)
joint_world_xf = parent_joint_world_xf * joint_local_xf

So you have........

original_vertices = model consisting of a list of vertices (and indices)
bind_shape_matrix = from creation package, when the model was created and bound to skeleton
inv_bind_pose_matrix[] = array of matrices, 1 for each joint (assuming, or identity)
weight_list[] = list of weights associated with a joint, how much the joint affects the vertex
animation_key_times[] = array of frame times, when you land between two key times, you interpolate using some method
animation_key_data = lists of matrices that refer to what a joints transform should be at a given time, sometimes stored as matrices, often easier to work with as separate translation, scale and rotation

After the model is loaded, you have the original vertices.
The bind_shape_matrix is applied every frame and doesnt change, so you can apply it once at load, save the result and use it for skinning.
bind_shape_vertex = bind_shape_matrix * original_vertex
Each joint (bone) has a xform from the scene, this is probably the pose for frame zero (first frame). You can think of these local xforms as the local xforms to use for the animation if the joint didnt move during the animation.
Ex. An arm up in the air, but only the wrist joint moving left to right, the elbow and shoulder would be up in the air at the start, but wouldnt animate, only the wrist joint would change.

During the scene loop

1) advance the animation time, clamp or wrap time to stop or repeat, scale to speed up down, etc
2) use the current animation time to interpolate between two animation key frames local transforms producing a new local transform for a given joint
this is where data as vec3s and quaternions for rotations might be easier to work with, rather then matrices
interpolated_anim_data_xf = lerp(anim_data_xf[n], anim_data_xf[n+1])
3) set this interpolated local xform from 2) as current for the joint (replacing the one from the frame 0 pose from the scene, or not maybe if the joint has no animation data)
joint_local_xf = interpolated_anim_data_xf
4) update the joints world xform using its new interpolated local transform, root -> leaf
joint_world_xf = parent_joint_world_xf * (interpolated_)joint_local_xf
5) create a skinning matrix for each joint from the new world matrix
joint_skin_matrix = joint_world_matrix * joint_inv_bind_pose_matrix
6) do the skinning using the bind_shape_vertices[] and each joint_skin_matrix

ForEach bind_shape_vertex[] v
{
skinned_vertex_pos = vec3(0.0, 0.0, 0.0)
ForEach joint j affecting vertex v
{
skinned_vertex_pos += (joint_skin_matrix[j] * bind_shape_vertex[v]) * joint_weight[] // this is done positions and normals, remember to normalize if things look weird
}
skinned_vertex[v] = skinned_vertex_pos
}

7) Draw the skinned_vertex[] list

Thats the general idea of how the data is used

Buckeye

Apr 14, 2015 at 2:24 AM

mat4 animationTransformation = TRANSFORM_Root * vertexWeights0 + ...

Correct me if I'm wrong, but it appears that you're performing a matrix calc for each bone for each vertex. If so, that's very inefficient and, either on the CPU or GPU, is a waste of computing time and power. I.e., it's likely that most of the vertex weights are 0 anyway.

A more common approach (in the shader) is to multiply the vertex position by only (for example) the 4 bone matrices that affect that vertex with the largest (normalized**) weights. That is, for each vertex, determine the indices of the bones with the 4 largest weights, and store those indices and weights as part of the vertex structure. That resorting should be done for the model file (and written out to a custom file to be used for your app) before it's loaded into your app. It needs be done only once.

** By normalized weights I mean: select the 4 bones with the greatest weights for that vertex. Rescale them such that weight1 + weight2 + weight3 + weight4 = 1.0. Then store the first 3 weights. The fourth weight can be later calculated as 1.0 - wt0 - wt1 - wt2.

You then store all the bone matrices in an array in the shader and end up with something like the second code set here. In particular, note that the bone indices and weights are input per vertex, rather than vertex weights per bone.

havsmonstret

Apr 14, 2015 at 7:02 PM

mat4 animationTransformation = TRANSFORM_Root * vertexWeights0 + ...

Correct me if I'm wrong, but it appears that you're performing a matrix calc for each bone for each vertex. If so, that's very inefficient and, either on the CPU or GPU, is a waste of computing time and power. I.e., it's likely that most of the vertex weights are 0 anyway.

A more common approach (in the shader) is to multiply the vertex position by only (for example) the 4 bone matrices that affect that vertex with the largest (normalized**) weights. That is, for each vertex, determine the indices of the bones with the 4 largest weights, and store those indices and weights as part of the vertex structure. That resorting should be done for the model file (and written out to a custom file to be used for your app) before it's loaded into your app. It needs be done only once.

** By normalized weights I mean: select the 4 bones with the greatest weights for that vertex. Rescale them such that weight1 + weight2 + weight3 + weight4 = 1.0. Then store the first 3 weights. The fourth weight can be later calculated as 1.0 - wt0 - wt1 - wt2.

You then store all the bone matrices in an array in the shader and end up with something like the second code set here. In particular, note that the bone indices and weights are input per vertex, rather than vertex weights per bone.

Yeah, you're right. That is just hard coded currently so I could get a simple animation up and running and get it to work before I optimize and generalize it.

-- snip --

Thanks! That's exactly what I needed to get it up and running!

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.