ID3DX10ThreadPump How-To

(ab)using the ID3DX10ThreadPump

Anyone familiar with the ID3DX10ThreadPump interface? As the name suggests its new in v10 of the D3DX library so its been around for a while now. From some of the early slides and diagrams I saw this always struck me as an interesting feature in D3DX10 but I'd never really had any need to play with it.

The basic premise is that you can multi-thread resource generation and loading. A real-world use case might be that you detect that your player is approaching the end of a level (or a waymarker suggesting new content is needed) and you start loading up a thread pump to load it in the background. Resource creation on the GPU is still not free, but at least you don't hurt the main rendering loop by having it stall on IO or other CPU tasks.

The aforelinked MSDN page describes where Microsoft chose to expose/implement thread-pump resource loading and chances are you'll of seen the parameter on enough occasions and probably just gone with NULL [lol] It does have this interesting quote though:

Quote:

The data loader interface can also be inherited and its APIs can be changed if one is loading a data file defined in one's own custom format.

As with much of that MSDN page one can't help but visualise it being explained by a posh english gentleman - would one like afternoon tea with one's thread pump? [smile]

Unfortunately from my digging around thats where the suggestion starts and finishes. I didn't find anything on the search engines or mailing lists explaining how to actually implement my own data loader or processor or any example code.

In my case I'm looking to multi-thread my terrain caching algorithm so that I can page in new terrain areas in a conceptually simple manner and have minimal impact on the core rendering performance. I was originally playing around with the joys of _beginthread() but decided I'd give the D3DX library a whirl first. Multi-threaded programming can be an entertaining challenge, but IMHO its too low-level for me to really want to actually mess around with the underlying threads - I like things like ID3DX10ThreadPump or .NET's ThreadPool to handle that fluff for me.

So, first up we have a data loader:

And then we also have a data processor:The only thing worth noting is the rather fugly member function declarations. COM_DECLSPEC_NOTHROW HRESULT STDMETHODCALLTYPE [oh]. Initially I forgot to add these and went rooting around inside the somewhat cryptic D3DX10 header files (d3dx10core.h to be precise) for the appropriate spec. For those that care, it basically boils down to a __declspec(nothrow) HRESULT __stdcall declaration - nothing too exciting really...

I then threw together a simple bit of test code to put it through its paces:


That's pretty much all thats necessary to get it to compile and execute with no errors. Those of you who actually read the code snippets will have already realised two rather crucial details - firstly the processor and loader are all NOP's and secondly that the queue-then-join design doesn't really exploit much by way of parallelism!

The former isn't too exciting to go into as the only reason you're going to be using this technology is if you have some sort of customized procedural loading or some other custom format to work with. That is, not one to be covered in a generic journal entry!

I'll demonstrate the flow of execution shortly, but in a nutshell your data loader should be allocating a chunk of memory in the ::Decompress() function and then destroying it again in ::Destroy().

The wording for ::Decompress() seems incorrect to me as it suggests your implementation is passed a block of data to decompress when its actually the other way around. You ::Load() from disk and then pass out the decompressed data via the parameters in ::Decompress(), something like:

_data = new int[512];*ppData = _data;*pcBytes = 512;

In particular, _data is a private-scoped member variable that can be used in a SAFE_DELETE_ARRAY() call during ::Destroy(). D3DX10 won't tidy up your memory allocation for you, so be careful not to fire-and-forget some chunk of decompressed memory [wink]

The wording for the data loader's ::Destroy() method has an accuracy that should be noted but will be later emphasised. "...to destroy the loader after a work item completes". That is, it isn't called just after loading is done but after the entire end-to-end work item processing has been completed.

Thats the data loader covered, so the next stage is the data processor.

First up is the ::Process() method which has a signature looking suspiciously similar to the previously discussed ::Decompress() data loader method. The chunk of memory allocated and owned by the data loader is fed into the processor via this function.

The data loader should really be dealing with just raw bytes from storage and its in the processor that it starts getting massaged into a D3D-ready state. That is, you'd use this method to transform the incoming bytes into an internal representation you could dump straight into an D3D10_SUBRESOURCE_DATA::pSysMem field before creating a new device-accessible buffer.

Once the data has been processed a call will come through to ::CreateDeviceObject() where we're to actually translate the processed bytes into a GPU-usable resource. A crucial link to realise here is that the pointer passed as the last parameter to ::AddWorkItem() (see above) is the pointer passed into this function. This could be an ID3D10Buffer that you're creating and returning to whatever code scheduled the work item in the first place - you need to be careful here to make sure you don't leak resources!



The above diagram shows the flow of events for a single work item in isolation. Obviously, in a properly threaded environment different work items are likely to be scheduled in different orders so you may observe different combinations despite the fact that each individually behaves in this order.

When executing you'll see, assuming debug output is enabled, something like:
D3DX10: IO thread count: 1, Process thread count: 1
emitted at the time you call D3DX10CreateThreadPump(). I've still got my crappy single core AMD PC hence only getting a couple of threads. My shiny new Intel Q6600 should hopefully have more threads available. As an aside, I'd be interested to know if these values scale linearly with the number of sockets/cores available... (hint, hint [wink])

As far as my investigation has shown there are two sets of threads corresponding to the two types of class provided (data loader and data processor). Items are added to the IO queue for the data loader events, and once they exit that queue they join the processing queue before finally joining a third queue for device creation.

That last detail about a third queue is quite important and goes hand-in-hand with D3D10's multithreading story being much the same as D3D9's. That is, you still want to access the device via a single thread if you can. If you're calling ::WaitForAllItems() like I posted above it'll block the thread until any device creation work is done, and as best I can test the ::CreateDeviceObjects() are called serially so as not to have multiple work items trying to create resources at the same time. Hence the 3rd queue which helps with this.

Other evidence for this is the ID3DX10ThreadPump::GetQueueStatus() which retrieves three counters for the internal state.

Rewriting ::WaitForAllItems() to be your own loop is easy enough, and more than likely what you really want to be doing if you're to use this feature effectively. By taking control of this busy-wait type loop you can get on with other work (for disk based loading or HLSL compiling you're likely to fit in several frames before it returns). There is, however, one very important detail. You must call ::ProcessDeviceWorkItems(). Unlike the previous two queues that step work through items themselves you need to manually iterate forward the 3rd queue - this gives you the control to schedule device-dependent work at a suitable time (e.g. once every N frames, or only N resources per frame etc...etc...)

The following can be inserted into the earlier code fragment instead of the call to ::WaitForAllItems():


So thats about it really [grin]

Given that I don't have a multi-core CPU it's been a bit hard to really evaluate how good this feature is, but I hope to have my new rig built in the next month or so at which point I'll revisit this and maybe look into doing some actual performance measurement.

Before I close this off, it is worth noting two of the limitations that i've come across so far.

1. When you have a number of work items in the 3rd queue awaiting device creation you can't select their priority order via the API. This can hurt if you've queued up a load of speculative resources but then get an urgent need to force one through because you need it on this frame - the best you can do is try and flush the cache via a big call to ProcessDeviceWorkItems() or even a WaitForAllItems(). I suspect you could manually call ::CreateDeviceObject() but I don't know how the thread pump would like that and it'd probably end up calling it again which could cause a memory leak if you're not careful...
   
2. You only get to pass one resource pointer into the ::AddWorkItem() entry point. For a typical geometry creation scenario you're going to want both an index and vertex buffer which would require two ID3D10Buffer pointers. I suppose you could have fun with pointers or structs or mess around with your own public interface to your data processor, but whichever way it is a minor inconvenience!

Anyway, the bottom line is that this seems like an under-documented (outside of MSDN) feature that I'd like to think I've shed some useful light on for your coding benefit [grin]

Comment, as always, are appreciated.