🔒 Locked

Offloading multiple-frame calculations to GPU

Started by Numsgil Dec 10, 2009 at 5:36 PM 8 replies 1.3k views

Original Post

Dec 10, 2009 at 5:36 PM

Suppose I have some crazy large computations that I expect to take several seconds, even with the GPU's raw horsepower. I can do it on the CPU as a background thread of some sort. But the CPU can only really manage something in the ~1-10 GFLOP range. The GPU can theoretically handle things in the 100 GFLOP range or larger. And if I have multiple computations I could theoretically split them up between GPU and CPU. My "game" (more of a simulation with a visualizer) uses very little graphics horsepower, so it would be cool to offload some of these multi-frame calculations to the GPU. They don't have a kill-by date, so I'm mostly interested in throughput over latency. But I don't want to affect the graphics FPS at all if I can avoid it. I'm guessing the only way to really achieve this is if I could break the large computation in to per-frame chunks that I know will not take longer than a few milliseconds to execute. That is, I could devote part of my rendering budget to these computations per frame. That would probably even work well if I was targeting a console since I would know a priori what sort of hardware and timings to expect. But it doesn't really help in the PC world where users might have anything from an Intel integrated chip to a dual 9800 NVidia. So I'm wondering if anyone has any thoughts on how this might work. My current thought is that I could maybe have a separate context for the computations, but I'm not sure if that would do a good job at load balancing between the high priority but low load graphics vs. the low priority but high load computations.

ApochPiQ

Dec 10, 2009 at 7:12 PM

I'm fairly sure that you can accomplish what you're after using nVidia's CUDA platform; at the very least I know that scheduling CUDA threads is pretty straightforward and powerful, so there's probably some way to sledgehammer things so that you get your standard graphics throughput while using leftover cycles for the extra computations.

Only downside is that you're locked in to nVidia hardware, unless you want to mess with doing a parallel implementation in Stream.

Numsgil

Dec 11, 2009 at 12:47 PM

Okay, so after some furious googling what I think I want to target is OpenCL. Looks like all the major vendors are on board, so I get some nice cross platform behavior. Does anyone know what the minimum specs are for different cards?

I might still decide to target raw shaders, though. It has the advantage that I can target lower end hardware. And GPGPU computing still feels a little in flux.

I'll probably have issues load balancing either way. Does anyone have any links or war stories trying to do anything like this?

the_edd

Dec 11, 2009 at 1:56 PM

Quote:
Original post by Numsgil
Okay, so after some furious googling what I think I want to target is OpenCL. Looks like all the major vendors are on board, so I get some nice cross platform behavior. Does anyone know what the minimum specs are for different cards?

OpenCL isn't limited to graphics boards, at least in theory. Depending on the implementation, it can target CPUs or any other "computation devices" on the system. The API allows you to enumerate such devices and select them in your code.

Apple's OpenCL integration will use whatever CPUs and GPUs are available, for example. I don't know how mature Windows drivers are at the moment, though.

There are some nice introductory podcasts and presentations on OpenCL here. Though it's hosted on macresearch.org, very little of the content is Mac specific.

Numsgil

Dec 11, 2009 at 6:55 PM

Thanks for the link. I'll watch the videos now.

the_edd

Dec 12, 2009 at 9:04 AM

Quote:
Original post by Numsgil
Thanks for the link. I'll watch the videos now.

I find watching them at 200% speed is a good idea. The narrator talks very slowly.

Numsgil

Dec 13, 2009 at 3:17 PM

200% is crazy :) 150% is about as fast as I can go without things becoming unintelligible. Which video player did you use? VLC starts making the audio really choppy at 2x.

the_edd

Dec 13, 2009 at 5:08 PM

Quote:
Original post by Numsgil
200% is crazy :) 150% is about as fast as I can go without things becoming unintelligible. Which video player did you use? VLC starts making the audio really choppy at 2x.

Oops! Yes indeed, I meant 150%!

Adam_42

Dec 14, 2009 at 4:06 AM

Quote:
Original post by Numsgil
I'm guessing the only way to really achieve this is if I could break the large computation in to per-frame chunks that I know will not take longer than a few milliseconds to execute. That is, I could devote part of my rendering budget to these computations per frame.

That would probably even work well if I was targeting a console since I would know a priori what sort of hardware and timings to expect. But it doesn't really help in the PC world where users might have anything from an Intel integrated chip to a dual 9800 NVidia.

So I'm wondering if anyone has any thoughts on how this might work. My current thought is that I could maybe have a separate context for the computations, but I'm not sure if that would do a good job at load balancing between the high priority but low load graphics vs. the low priority but high load computations.

The way I'd go about solving this is to benchmark a computation of increasing chunk size on startup. You should be able to find the biggest chunk size that doesn't take too long quite quickly. You can then safely do that much work (or less) each frame.

Numsgil

Dec 14, 2009 at 11:37 AM

Quote:
Original post by Adam_42
Quote:
Original post by Numsgil
I'm guessing the only way to really achieve this is if I could break the large computation in to per-frame chunks that I know will not take longer than a few milliseconds to execute. That is, I could devote part of my rendering budget to these computations per frame.

That would probably even work well if I was targeting a console since I would know a priori what sort of hardware and timings to expect. But it doesn't really help in the PC world where users might have anything from an Intel integrated chip to a dual 9800 NVidia.

So I'm wondering if anyone has any thoughts on how this might work. My current thought is that I could maybe have a separate context for the computations, but I'm not sure if that would do a good job at load balancing between the high priority but low load graphics vs. the low priority but high load computations.

The way I'd go about solving this is to benchmark a computation of increasing chunk size on startup. You should be able to find the biggest chunk size that doesn't take too long quite quickly. You can then safely do that much work (or less) each frame.

I was hoping to get away with doing computations as monolithic chunks. There's quite a bit of work involved in chunking up the work so you can pause/resume work whenever you want. That's one of the advantages of multithreading on the CPU: you can just manage the worker threads to pause/resume work or yield to higher priority threads, without doing anything extra for the work going on inside the threads beyond some synchronization primitives. I was hoping for that same basic architecture on the GPUs, if possible.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.