Skip to main content
gamedev.net
🔒 Locked

optimizing object outlining using stencil buffer.. [suggestions requested]

Started by jdaniel 2 replies 2.1k views
Original Post
jdaniel
jdaniel
Hello, I am currently working on an engine and I am responsible for implementing object outlining to show the user what was just selected. Everything works fine, but performance is negatively affected when an object is selected and thus outlined. Basically, what I am doing is the following: 


glPushMatrix();

  glClearStencil(0);
  glClear(GL_STENCIL_BUFFER_BIT);
  glEnable(GL_STENCIL_TEST);
  glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

  // render original into stencil
  glMultMatrix(transMaatrix);
  glCallList(mydisplaylist);

  glStencilFunc(GL_NOTEQUAL, 1, -1);
  glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

glPopMatrix();

glDisable(GL_LIGHTING);
glDisable(GL_TEXTURE_2D);

glLineWidth(3);
glMultMatrix(transmatrix);

  // now begin a complete rendering loop rendering
  // the object in wireframe ...

     glBegin(GL_LINE_LOOP)
       glVertex3fv(....
     glEnd();

glEnable(GL_LIGHTING);
glEnable(GL_TEXTURE_2D);
glDisable(GL_STENCIL_TEST);
All of this is compiled into its own display list and is called if there is a currently "selected" object. It works beautifully, just as I want. I like the fact that the outline width stays the same no matter where the user places the camera. However, I lose considerable performance when calling this display list. It seems to directly related to the number of vertices in the orginal object. I am aware, for a selected object I am rendering three times - once into the stencil, then using the wireframe against the stencil, and then finally the actual object. The GL state changes do slow things down slightly, but I am unaware of how to remove any of the state calls. Any suggestions or pointers to appropriate papers would be greatly appreciated. Thanks, jdaniel
Erik Sintorn
Erik Sintorn
Drawing wireframes is very slow. A scene that runs at 800 fps on my GF6800 slows down to 200fps when I render it in wireframes instead. Anyway, what do you need the stencil buffer for? Couldn't you just draw the object a little bigger than usual, with a solid color and no lighting an then draw it normally on top of that?
jdaniel
jdaniel
It seems that by first drawing the solid object without lighting and then drawing the object on top would not allow consistent line width. As the user moved the camera closer to the object, the line outlining the object would appear to grow in width.

By using the above method with the line width set to 3, the outline stays a consistent pixel width regardless of where the camera is placed.

Something seems wrong with the optimization, however. I think you may be right in the fact that I do not need the stencil buffer to do this.

I think I saw some GPGPU code that does edge detection at one time. Does this seem like something that could be optimized by writing a shader?



Erik Sintorn
Erik Sintorn
Hmm.. I suppose you could do edge detection with shaders like this (a variant of the extrude shadow volumes in hardware trick):

For each edge, send along the triangle normals of the two neighbouring triangles.

If both triangles are front facing, or both are backfacing they are not part of the outline and you could move one of the vertice to the position of the other so it won't be drawn.

This will draw a bit more than just your outline, but it should cull away most of the edges.

I'm not sure it wouldn't be faster to somehow figure out the size of the solid color object from the MVP matrix instead.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.