🔒 Locked

GL_RGB video is slow using glTexImage2D

Started by spowers Feb 24, 2009 at 3:24 PM 4 replies 7.4k views

Original Post

Feb 24, 2009 at 3:24 PM

The setup: Video stream in format: GL_RGB Using glTexImage2D / glTexSubImage2D to display the video stream to a texture rendered onto a quad. The problem: I have read that using glTexSubImage2D with a non-GL_RGBA texture is slow since the video driver needs to reformat the texture to GL_RGBA. When I do this with video it ends up slowing down the process. I call glTexSubImage2D after each frame is ready to display (~30Hz) and a 1280 x 480 video stream slows the process down to 14 Hz(fps). Changing the glTexSubImage2D to use GL_RGBA seems to greatly increase the performance of the process although it will not render the image correctly since I'm still giving it a GL_RGB stream. I added a simple function that will pad the RGB stream with an alpha channel but that ends up slowing the process down further. The question: Rendering video using openGL is a fairly common task and video rarely (never?) comes with an alpha channel. How do others render the video quickly enough with high resolutions? I tried glDrawPixels but that is much slower. Is there a quick way to pad the RGB stream with an alpha channel?

Fiddler

Feb 24, 2009 at 3:32 PM

VLC comes with an OpenGL video renderer - maybe you can check their code to see how they handle this.

In general, you should try to pad while decoding the stream, not after (as the latter would require a copy of the data, which will be very slow).

Another possibility that may, or may not, end up being faster, is to double-buffer the textures:
upload to texture 0
upload to texture 1 and render texture 0
upload to texture 0 and render texture 1
upload to texture 1 and render texture 0
etc...

If you can upload texture data asynchronously, e.g. by using the PBO extension or by creating a second opengl context in a different thread, this will hide the latency of the data transfer and should be quite a bit faster.

Edit: Also, take a look at this Beyond3D thread. There is some *very* good advice around the middle of it.

spowers

Feb 24, 2009 at 3:41 PM

Thank you for the reply!

My video stream decode process is a simple memcpy() and is done within a library that I do not control and cannot change.. unfortunately. All I have to work with is a pointer to a buffer that contains the new frame in GL_RBG format.

I toyed with the idea threading the padding but I also need to limit the latency of the video stream (its from a live camera) and the display. Multithreading the padding process will not reduce the latency.

I'm interested in the Pixel Buffer Object. Is that native to OpenGL or is there a library that I need to find? Is there a library that is recommended? Would the PBO still require GL_RBGA data?

spowers

Feb 24, 2009 at 3:48 PM

I also have a related question. When I created a static GL_RGB texture and used that with glTexSubImage2D it seemed that my slowness problems went away. I used this static texture exactly like my video stream (calling glTexSubImage2D @ 30Hz) but I never changed the contents of the texture.

Is there some sort of optimization going on that I am not aware of?

Fiddler

Feb 24, 2009 at 3:53 PM

As you may know, OpenGL is separated into a core API (i.e. what we call OpenGL 2.1, 3.0, etc) and extensions. Pixel Buffer Objects are a relatively recent extension to OpenGL that adds asynchronous texture read / write operations.

If your video card has recent drivers, you should be able to use this extension. The link in my previous post points to a PBO tutorial with example code. The PBO specification explains in detail how this extension is implemented (not a light read, but useful nonetheless).

If you have never used OpenGL extensions before, you should use a library such as GLEE to simplify the process of getting the function pointers.

Edit: The graphics card does not support GL_RGB textures directly, which means that the driver has to pad the texture when you upload it. The padding happens only when you change the texture contents - if the texture remains intact, you are running at full speed.

Edit 2: One suggestion is to split the upload process in many smaller glTexSubImage2D calls, instead of one big one. It seems this can help performance, so it's worth a try.

spowers

Feb 24, 2009 at 4:59 PM

Quote:
Original post by Fiddler

Edit: The graphics card does not support GL_RGB textures directly, which means that the driver has to pad the texture when you upload it. The padding happens only when you change the texture contents - if the texture remains intact, you are running at full speed.

Edit 2: One suggestion is to split the upload process in many smaller glTexSubImage2D calls, instead of one big one. It seems this can help performance, so it's worth a try.

So even if I call glTexSubImage2D() with an unchanging buffer it will not re-pad the image? I tried changing random pixels within the buffer and it didnt seem to reduce the performance.

Is this to reduce the load on the CPU cache or is it to yield the thread that it is running on? I ask because my application is not multi-threaded... yet.

I'll give both a try tomorrow when I get back to coding.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.