Skinning Doom MD5 on the GPU efficiently?

sorry this is no help, but im pretty sure in doom3 they would do skinning on the cpu, due to the fact with lighting + shadowing, each mesh perhaps will be rendered multiple times. something to consider if youre gonna do lighting/shadowing yourself.

each weight doesn t have its own position.
Usually you have a UBYTE4 (4 * 1 byte) indexing the 4 bone matrices used for this vertex stored in a matrix array. This bone matrices array will be passed to the vertex shader through uniform parameter.
Then you have 3 floats representing the weight of each bone, the 4th one is given by w3 = (1.0 - w0 - w1 - w2).
This way you have an overhead of 3 * 4 + 4 = 16 byte per vertex.

That's the conventional way of doing things, yes. The problem is that is not how D3 models are stored! Take a look at this excerpt:
weight 162 3 0.2192307562 ( 10.5320158005 -8.3126058578 6.2153153419 )
The fields are index, joint index, weight value, and position. Each vertex references up to four of these.

Now you begin to see why I'm a little confused? I know D3 does CPU skinning, that's not news to me, but surely this came up at some point?

I don't think it did come up. The Doom 3 meshes were *never* meant to be skinned on the GPU, hence the format is layed out in a way that is not friendly to GPUs, but presumably is friendly (or at least useful) to the way they need the info.

Ok, I'm just getting back to this.

So, that seems really helpful, thanks. Unfortunately, I didn't understand a critical part of what you said.

Quote:

And just calculate the inverse bind pose matrices from the initial pose of the model

What is the inverse bind pose matrix, and how do I find it?

The bind pose is the pose that is stored in the .md5mesh (with arms and legs straight out). Take the bone matrices for this pose and inverse (transpose the rotation matrix) them and you will get the inverse bind pose matrices. I use these matrices for normal transformation.

Quote:

Original post by Anonymous Poster - John

Are you John Carmack ? [smile]

I did the MD5 skinning, animating on the CPU and was content with it since I felt that it will eventually end up being slower on the GPU.

Quote:

Original post by Anonymous Poster This seems to be what Doom 3 does then.

It depend maybe of the configuration, because Doom3 run on a Geforce2.

Quote:

Original post by Anonymous Poster Lol no :) I'm John

is that your front name or your back name?

Quote:

Original post by Anonymous Poster Lol no :) I'm John from www.emotionfx.com :) (shameless plug) :)

Oh well, hi again [grin] Dustin and I actually spent most of a dinner talking about D3's insanity [lol]

Quote:

About the inverse bind pose matrices. Bind pose is the pose of the model which was used to link the mesh to the bones. So often this is in the well known T- or Jesus pose :) This is most likely also the pose in which the MD5 models are stored.

It is, yeah.

Quote:

So you could calculate the inverse of the world pose bind pose matrices.

When I'm doing the inverse, do I do just the inverse of the rotation, or do I use the inverse of both rotation and translation?

[Edited by - Promit on May 27, 2006 8:10:29 PM]

Ok, so I think I'm starting to get a clear picture of what's going on.

Basically, for any given joint J in any given frame we have a target orientation matrix T. Our original data is actually in the bind pose orientation, call that matrix B. We need to transform all of the vertices from B, which is where they have been kept in the vertex buffer, to T. The way to do that is to use B^-1 to get our joint to the identity transform I, and then transform it by T to get the final result vector R. Thus the whole equation is R = T * (J * B^-1). I assume that we can precompute J * B^-1 when loading and store the vertices as they would be if all of the joints were at the origin, rather than storing them in the bind pose. That will save us the multiplication by B^-1 when we actually to render.

Is that all correct, or did I make any mistakes?

(argh, gamedev forums didn't understand the for loop)
(reposting now after registration) :)

I'm not sure what you mean with J.

Let's assume:

B = bind pose matrix in world/model space
W = world/model space matrix of the joint
V = vertex position in bind pose

The skinning algorithm works like this:

    V' = (0, 0, 0);    for (int i=0; i<numInfluences; ++i)        V' += (V * (Inverse(B) * W)) * influences.weight;

Now V' contains the skinned vertex position. You do this for every vertex in the mesh.

As you can see, the Inverse(B) * W is shared between all influences effecting a given vertex. So this can be precalculated before entering the skinning loop.
Also if you assume there is no scale, you could use the matrix transpose instead of the inverse.

This assumes the weight values of all influences for a given vertex sum up to a value of 1.

So what you need to do in order to get the MD5 skinning on the GPU is to ignore the vertex offset values it stores for each influence, and calculate the matrix B after you loaded the data. You can then apply forward kinematics to calculate the world space matrices of the bind pose for each joint. This will give you the B values. The W values are the transformation matrices of your joints in world space. These values change over time. The B value is constant over time, so does not change. Also the V value does not change over time, as it are the vertex positions you load in from the file (in bind pose).

Just let me know if you need any more help. You can also email me of course, but maybe the forums are better as someone else might find this info useful :)

- John

Quote:

Original post by Buckshag Just let me know if you need any more help. You can also email me of course, but maybe the forums are better as someone else might find this info useful :)

*chuckles* they are, namely myself as I plan todo this at some point in the nearish future, so thanks for the details from myself as well [grin]

Edit: oh, and welcome to Gamedev as a registered user [smile]

[Edited by - phantom on May 30, 2006 9:38:07 PM]

not to be a necromancer,

but about a year and a half ago I implemented md5 skinning on GPU... it was painful, this is how I did it:

each vertex depends on a set of weight points. I put an (artificial) max on the number of wieghts a vertex could depend on: 8. now the "vertex" arrays became the wieghts (thus one vertex needed 8 GLSL attributes)... I stored the point weights as follows:

.xyz --> position in space (relative to parent joint)
.w --> integer part= which joint, fractioanl part*2= magnitue of the weight

now the ugly parts... some md5's have lots of joints so you need to partition the mesh so that each triangle depends on at most a certian number of joints (I set that number to be 50 joints. That breaking of the mesh into submeshes was painful.

at the end of the day the attrbiutes for the GLSL vertex shader were:
8 weight thingies (vec4)
1 info thingy (number of weights and texture co-ordinate)
1 normal (vec3)
1 tangnet (vec3)
1 biTangnet (vec3)

12 attributes used total.... I guess I coul dhave goe for 12 wights... but I did not see the point to it....

I sorted the weights by how much mass they contribute to the vertex, so I cheated and just made the normal, tangent and biTangnet multiplied by the joint of that weight...

there is no need to calculate the inverse of the joints at any step... the heirarchy assures you of this...

painful lessons that I learned doing this:
1) convert te Quaternons to matrices then send them to the shader, though the matrix is 9 floats (and possily takes 12 floats in the shader) chaning the quaternonin into a matrix at shader time is just not a good idea.

2) the state of using arrays in GLSL is dicey, it is ok for uniform variables but you get into trouble for attributes (which now nVidia's driver says is not cool to do) or local variables... basicly I wanted to write a loop:

for(j=0;j{
place+=grossstuff[j];
}

the diver would crash, but if I did

for(j=0;j{
if(j place+=grossstuff[j];
}

it was ok, basicly because the GLSL compiler unrolls the loop at compile time... same ugly story in Cg (the Cg compiler would jsut say "internal compiler error).... that pissed me off sooooooooo badly....


and what do I get for my troubles? I can display about 21 or so MD5's (HellKnight from Doom3) at a frame rate of around 25FPS(I leave vblank on because that is how people shoud play).... hardware is GeForce6600GT (128MB) and AthlonXP 3000+, 1GB RAM, Linux OS. and I know the killer is the geometry part as mutilating the fragment shader does not change the frame rate (i.e. just a texture with no lighting gives same perfomance as doing bump mapping and specular lighting, with 12 lights!)


on the other hand, md3, I can display an incredibele number... lets see... over 50 at about 50fps (Klesk quake3 model).... the fragment shader is the same (12 lights)....admittedly the Klesk model has fewer poly's but stilll... my performance was a lot worse on a GeForece5900FX, where I had to rearange the shader source just *so* to get it to fit into the GPU... (shudders)...

has anyone gotten any success of drawing LOTS of md5 at the same time? I am wanting to say draw about 20 with ligthing (but no shadowing) at 60FPS on midrange hardware... my only thoughts at this point fo rme to improve perfomance is to bight the bullet and say that a weight point takes 5 floats (thus the joint information would be packed on 2 of the remaining attributes) thie way I would not have to use floor or fact.

Best Regards

-kRogue

P.S. the link of 2 is the exact place where I learned to do md5 skinning.

P.P.S (edit) I went ahead and looked up the code in Quake4 how they did md5 GPU skinning.. get ready guys, if I am right, it sucks: they put the wieghts (all of them) into uniform variables (I think it bugs out at 90 or so) here is the shder code (I think) looki n the file q4base/pak001.pk4/glprogs/mdr5_whatever.vp (whatever can be basicfog1, renvbump1, ect.... so to draw an md5 they have to do a lot of draw calls, looks like lots of calls... I could be wrong though as I never bothered to learn the asm style shaders (but maybe I should have as it lets me choose all the low level pakcing junk my self since the GLSL and Cg compilers seem unable to handle that)...




[Edited by - kRogue on June 10, 2006 2:13:50 PM]

Quote:

Original post by kRogue P.P.S (edit) I went ahead and looked up the code in Quake4 how they did md5 GPU skinning.. get ready guys, if I am right, it sucks: they put the wieghts (all of them) into uniform variables (I think it bugs out at 90 or so) here is the shder code (I think) looki n the file q4base/pak001.pk4/glprogs/mdr5_whatever.vp (whatever can be basicfog1, renvbump1, ect.... so to draw an md5 they have to do a lot of draw calls, looks like lots of calls... I could be wrong though as I never bothered to learn the asm style shaders (but maybe I should have as it lets me choose all the low level pakcing junk my self since the GLSL and Cg compilers seem unable to handle that)...

I think these shaders are for models stored internally as MD5Rs, and I remember reading on doom3world.org's forums that MD5R are used in the XBOX version only

[Edited by - Starfox on June 11, 2006 2:29:52 AM]

wonder what those shaders are for then.... hmmm... makes you go hmmmm...

The original post's here: http://www.doom3world.org/phpbb2/viewtopic.php?t=12953&highlight=md5r

yet weirdly enough, the MD5R shaders, in the PC version at least, are all ARB VP1 programs...