2D rendering with Windows GDI on top of 3D OpenGL

Is it a realtime application?
I would assume some flickering using the method you describe. What exactly do you mean with doesn't work?

I think GDI only draws to system memory on Vista/7, so that might be your problem. You could try rendering to a GDI back-buffer, and then drawing on top of that. http://msdn.microsoft.com/en-us/library/ms970768.aspx has some information. Do you use PFD_SUPPORT_GDI when choosing your pixel format?

Yes, it is realtime in the sense that we're rendering as the user is interacting with the app, we're not pre-rendering scenes or anything.

Yes, we do see flicker. This is annoying, but we're able to live with that.

By "doesn't work" I mean the 2D rendering gets all messed up. For example, we render simple TextOut labels on top of some of the 3D objects and instead of the text you just get a rectable filled in with what looks like an invalid buffer. Just rendering wierdness where we're not getting clean 2D rendering, but there's clearly something messed up.

We've looked at rendering to a back buffer, problem there is you lose HW acceleration (at least thats was my conclusion when I looked into doing that). With HW acceleration we, of course, get a HUGE boost to FPS so going back to SW rendering doesn't seem like a viable option. We do use SW rendering to a back buffer for printing as an example, but there you're just rendering 1 frame and you're done.

I'll double-check the PFD_SUPPORT_GDI and see if we're picking a pixel format that has this on the problematic machines.

We do only see the problem on Windows 7 (haven't tested Vista as we've gone from XP right to Win7) and your comment about GDI and Vista/7 is interesting. Note, though, that on some Win7 machines, this GDI rendering works just fine. So its not something that never works, it appears related to video card/drivers.

1, You have to let go of the context with OpenGL before you can draw on it via GDI (wglMakeCurrent (NULL, NULL)). Note that in some cases (Vista for example) the window gets refreshed from the system maintained back buffer when OpenGL disconnects from it, causing your rendered image to get overwritten with something else. To work around this problem I do a screen capture with glReadPixels (slow with some Ati drivers) and directly load my rendered image up on the window using BitBlt before I release the context with OpenGL.

2, PFD_SUPPORT_GDI will disable hardware acceleration so I would not recommend using it.

3, I think the best solution would be to use Frame Buffer Objects to render into offscreen memory and BitBlt that on the screen before you start drawing with GDI. Using this method you'd be able to render into offscreen memory and still retain hardware acceleration. (OpenGL has some awkwardness here since you'll still need a window to create an OpenGL contex even though you don't want to render on screen. But a hidden dummy window will make it.)

Thanks, I'll give those suggestions a try. Out of curiosity, do you know anything about use of PFD_SWAP_COPY?

http://msdn.microsoft.com/en-us/library/dd368826(VS.85).aspx states that PFD_SWAP_COPY is a hint, that may or may not change the behavior. That page also has information on all the other flags available.

Peti29, if using a frame buffer for rendering, you mentioned being able to then blit the contents of the frame buffer onto the window. Isn't a frame buffer a GL construct? How do you access the image on the frame buffer from Windows APIs? Guess I need to do some more digging on frame buffer objects, but that's my first question, i.e. how does that connect to the Windows API world and how do I access it to get it on the screen if its a GL construct.

Well, the frame buffer object is apparently an extension to OpenGL. I'm using Visual Studio 2005 SP2 and my GL headers don't have this extension defined nor do they contain the functions I see in the below article. Is there something I need to get to be able to use extensions like this? I've only done the basic core GL before, not used extensions.

http://www.gamedev.net/reference/articles/article2331.asp

I don't think PFD_SWAP_COPY is relevant here. It tells the driver to copy the back buffer's content onto the front buffer (so that the back buffer remains unchanged) instead of exchanging them (in which case the back buffer receives the former contents of the front buffer). But that only affects the working of wglSwapBuffers and as Erik wrote the driver may disregard that flag anyway.

You are right, you still need to call glReadPixels when obtaining the contents of the FBO, but that supposed to be faster than capturing from the screen. (And what's more important most drivers are unable to draw under an overlapping window, so if some other window is over your app's client area, or your window is partly out of the screen, your captured image will contain invalid parts there when capturing from the screen.)
Before calling glReadPixels you have to set glReadBuffer (GL_COLOR_ATTACHMENT0_EXT); to specify the FBO as the source for the capturing.

Yes, FBO is an extension (at least prior to OpenGL 3.0) so you'll have to manually get the function pointers of it (with wglGetProcAddress). Yeah, that's a bit ponderous...

If you never used extensions nor FBO maybe you should try to simply disconnect the OpenGL context from the window before you draw on it with GDI. Most probably that method will work you just fine and it is a lot easier that way.

I've thought about this more and, although this might be a viable way to do the offscreen rendering, in our scenario, I think we'd have to end up 1) rendering to the frame buffer, then 2) copying from the framebuffer to an offscreen bitmap, then 3) doing our GDI rendering to the offscreen bitmap, then 4) copying from that bitmap to the window DC.

This would be needed to avoid the flicker with the GL scene showing up then the GDI, then the GL scene, etc. And there are 2 copies in that process as well.

I think the other alternative we might have to go with is just rendering everything, 3D and 2D overlays with GL. This wouldn't require the GL extension, wouldn't require the extra copying, wouldn't have the flicker, etc.

What exactly are you trying to do with GDI? Draw text, render windows controls?

I ask because the above are far too convoluted to be useful - there are much easier ways depending on what you want...

We render text, lines, rectangles, filled rectangles, rounded rectangles, etc. Just basic stuff like that for annotating 3D in the plane of the screen. We're not providing rendering of a window system or anything like that. Just basic annotations.

That should be very quick to do in OpenGL instead. Font-support is built-in to WGL with wglUseFontBitmaps, see for example: http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=13. If you want to draw rectangles etc. in pixel-space, use glOrtho to set the projection matrix, and you can easily draw with 2D pixel-based coordinates. There's also glLineWidth that can help you easily draw thick lines.

Erik, do you get anti-aliasing with wglUseFontBitmaps?

We do have some cases where we render 3D text in the scene already with OpenGL and we've used wglUseFontOutlines for that. The problem there is you don't get anti-aliasing so the text looks pretty rough.

There are two really simply options available -

1. Look at texture mapped fonts (possibly with kerning information)

2. Use GDI+ to render to a memory bitmap (which allows text & shapes) and then use glTexSubImage to update a GL texture. This is by far the simplest for your needs. Just remember to watch for alignment issues - see here http://msdn.microsoft.com/en-us/library/ms970772.aspx & here http://msdn.microsoft.com/en-us/library/ms970765.aspx

mark ds, I assume I'd setup the offscreen bitmap with an alpha channel; will GDI+ then do antialiasing making use of the alpha? I wonder if GDI+ would also anti-alias other geometry (lines, curves, rounded rects)? Then each frame (after rendering) I update the texture and setup the UV coords so it lines up with the screen and draw a simple rect? Sounds like an interesting approach. I wonder though what the performance would be like since you have to copy from the offscreen image to the texture and then render the texture, whereas if we just do the 2D rendering with GL its just some extra GL commands and no additional copies. Granted, we don't get anti-aliasing then, unless wglUseFontBitmaps does anti-aliasing or we use something like FreeType and/or Cairo (suggested in another thread).

Are you actually referring to anti-aliasing, as in edges where the pixels are multi-sampled, or do you mean the fonts get blocks that are larger than one pixel?
Used correctly, the only problem is the same as aliasing on polygon-edges without multi-sampling. Perhaps you have used a font that doesn't have a high enough resolution, try with a true-type font. If you really need anti-aliasing, perhaps FSAA will help, or simply draw the text to a higher resolution texture and scale it down to 50% or 25% with averaging to get somewhat the same effect. I fail to see how such perfect font quality is important when constant flickering is acceptable though, which makes me think perhaps I misunderstand you. There are OpenGL functions to smooth lines and points if you want them anti-aliased (enable POINT_SMOOTH and LINE_SMOOTH with alpha-blending).

well, put simply the text drawn with GDI looks "smooth", the shapes are rounded and not "jaggy". If I render using wglUseFontOutlines using a true-type font, the result looks "jaggy" and rough. Its a big difference between the GDI text and calling the display lists with wglUseFontOutlines.

I'm not 100% sure as to the technical mechanism being used, so I'm just referring to it as "anti-aliasing" in general as I can see a big visual difference.

As for constant flickering being acceptable, obviously its not great, but its not as bad as some of the garbage on the screen we're seeing :)

So, yes, we need it to work and actually render something you can recognize as text, lines, boxes. We can achieve that as is just using wglUseFontOutlines. However, as I said, the resultant text doesn't look good and is really rough around the edges. Big difference from the text you get when rendering with GDI.

Here's another way to put it, when I tried this and rendered both and then zoomed in really close, I could see GDI was doing anti-aliasing, where it was filling in around the corners with different shades of color in relation to the background having the result of smoothing out the curves.

That gives a much nicer look then the straight GL text with wglUseFontOutlines.

To get back to the big picture though, it comes down to the following. We need to render a 3D scene and also 2D annotations on top of that scene (text, lines, rectangles, rounded filled rectangles, etc).

Our current approach is to render the 3D with opengl, then follow it up with 2D using GDI on the window after swapbuffers is called. This is causing weird display artifacts on some Win7 machines with some video cards. It also flickers due to the alternating 3D/2D rendering to the window. The flickering is annoying, but the artifacts are worse.

My question really is, any game (just about) is likely to have some 2D text, # HP remaining, score, etc. So this problem has to have been solved, drawing 3D and then 2D on top. I expect our approach of using GDI is less than ideal and perhaps not even correct, esp in a HW accel environment.

So I'm tossing around possibilities to try and come up with a better way. Doing all the rendering with GL seems like one approach. The only snag there we came across initially was the text looks like crap compared to GDI text (the "jaggy"-ness I mentioned in the above reply). So one thought is that if we solve that, maybe that becomes a viable alternative.

Whether the smoothness from the GDI text is technically anti-aliasing or something else, I'm not sure. It just looks a whole lot nicer than using wglUseFontOutlines :)

Yes that's antialiasing, font-smoothing which is enabled in Windows by default. I don't think you can get that from wglUseFontBitmaps. I don't know if wglUseFontOutlines has any difference to wglUseFontBitmaps. In order to get that you need some other way to draw fonts, and the easiest is probably to simply draw in black-and-white with GDI, and use that as the alpha-channel for a texture. For drawing rectangles etc. I would definitely use OpenGL, as you can get anti-aliasing there, and it's also possible to fake manually by drawing lines with blending around objects.

If you need performance, don't draw the text with GDI every frame, but draw every character to a texture at app start-up (or load a saved image with the font-characters in it) and draw using that. You basically draw one quad for each character in the string. This is how most games do it, as far as I know. It's the way I have always done it. NeHe has example-code and a font-texture for that too: http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=17.

With that you can obviously get any quality you want.

I don't know how much quality you need. This is what I get with wglUseFontBitmaps, which I think looks pretty good, though it's definitely not anti-aliased. Perhaps it's just me that don't care about it, I have even disabled font-smoothing in Windows, because I think it's easier to read without it. =)