Texture cache \ pixel shader

As far as I know there is one texture cache per texture unit and the only thing you need to do to keep this cache happy is to ensure that all your texture fetches to a particular texture are spatially coherent.

Per texture unit of course !

Thank you, this is more clear.

... But ... is there a bunch of texture unit for all pixel unit ? Pixel shader are ran in paralel, so, what about texture caches in front of that ?

Pixel shader are not ran in parallel. It's a bit more complicated ; on current GPU, pixel shader are processed sequentially, but they process more than one pixel at a time. So the parallelism is between pixels using the same pixel shader, not between pixel shaders.

I think (but I'm not sure) that next-gen cards will be able to process several pixel and vertex shaders in parallel ; all the resources will be mutualized in a unique shader model. Therefore texture units will be shared. The problem will not change anyway ; as long as a texture unit is bound to a memory area, its texture cache is feeded from there and everything is fine.

Quote:

Original post by Niwak Pixel shader are not ran in parallel. It's a bit more complicated ; on current GPU, pixel shader are processed sequentially, but they process more than one pixel at a time. So the parallelism is between pixels using the same pixel shader, not between pixel shaders.

Arf, in fact, that is what I mean by "pixel shader ran in paralel". But your right, It is not exact.

So,
many pixels are processed in paralel, each pixel accessing the same texture units. What about the texture cache coherence ? Is the cache big enough to fit the entire texture ?

Is the texture cache coherence due to the fact that pixels that are processed in paralel are taken by squared blocks (I'm not sure of this) ?

The texture cache is far smaller that usually used textures are. I'm sure you have heard about the pre-/post- TnL cache, the texture cache works exactly the same way just with a diferrent resource. The texture cache is loaded with data that is spatially located around the location read at the previous texture fetch and/or with the data you have recently read. So, If you will scatter you texture fetched all over the texture you will get lots of texture cache misses, that means the pixel you want is not in the cache so the GPU must read it from the texture.

The same applies to the pre-/post- TnL cache, if your indices are properly ordered you can get a ratio of 0.6-0.8 transformed vertices per triangle(triangle strips have a ratio of 1 tranformed vertex per triangle,except the first one of course). That means that most of the time the GPU won't transform your vertices, more likely it will just take it from the cache.

Quote:

Original post by MePHyst0 The texture cache is far smaller that usually used textures are. I'm sure you have heard about the pre-/post- TnL cache, the texture cache works exactly the same way just with a diferrent resource. The texture cache is loaded with data that is spatially located around the location read at the previous texture fetch and/or with the data you have recently read. So, If you will scatter you texture fetched all over the texture you will get lots of texture cache misses, that means the pixel you want is not in the cache so the GPU must read it from the texture. The same applies to the pre-/post- TnL cache, if your indices are properly ordered you can get a ratio of 0.6-0.8 transformed vertices per triangle(triangle strips have a ratio of 1 tranformed vertex per triangle,except the first one of course). That means that most of the time the GPU won't transform your vertices, more likely it will just take it from the cache.

I understand well what you said but it doesn't explain what happend in a paralel architecture where a same texture unit can be sampled by two executions of the same pixel shader.

Imagine you have a pixel shader like this :

inst1
tex2D <--- unit1
inst3

During the execution, it may be done in this order :

pixel(x0,y0) pixel(x1,y1)
inst1 inst1
tex2D <--- unit1
inst3 tex2D <--- unit1
inst3

I'm maybe wrong somewhere, but if this scheme is right, the texture unit number 1 is, here, sampled 2 times by 2 different pixels. So, what about the cache ? There is cache coherence if the pixels are disposed in blocks, but it have significant effect on the usage of textures because the cache coherence must thought by pixels blocks.

Else, there is a texture cache per unit*pipeline...

This is the kind of thing that can be acknowledged or completely disproved only by people employed by GPU Vendors :)

The problem you described sounds interesting, however I still think that there's one texture cache per texture unit. Regarding your question, GPU usually makes sure that pixels that run in parallel are coming from the same triangle, so in common circumstances the texture fetches made by these pixels are spatially coherent( since the interpolated texture coords are coherent from nature). So, you can get heavy cache misses only when doing your own crazy texture reads, but the GPU can not stay prepared for such thinks, so it just assumes that you won't make such thinks :)

Finally, I would like to mention that this is my,and only my, honest assumption which can be totally wrong. I am convinced that it isn't, but I'll be happy when someone will prove me wrong :)

Thank you very much for re-posting.

Seems I did not forget a dumb factor but, I'm conscious that it could be consomptions too.

Quote:

Original post by MePHyst0 So, you can get heavy cache misses only when doing your own crazy texture reads, but the GPU can not stay prepared for such thinks, so it just assumes that you won't make such thinks :)

This is interresting to see that you should really do chaotic things on your shader to have texture cache inconsistency due to the paralel execution. Even shadow mapping (with PCF) is partly coherent for example.

Pixel shader units are usually grouped into 'quads' that operate on 4 pixels at a time. Generally these will be 4 pixels in a square from a single triangle. Cards that have say 24 pixel pipelines will usually be arranged with 6 groups of quads. Most cards only have a single texture cache shared by all the texture units and it is generally quite small - around 32K is common. For a simple case like drawing a large screen aligned quad with appropriate sized mip levels available you'll get very good texture cache use since nearby pixels will generally be processed in parallel together and will all be reading from spatially coherent texels for filtering. The main purpose of the texture cache is to reduce redundant fetches for bilinear, trilinear and anisotropic filtering, not to hold large amounts of texture data.

The cache doesn't have to be very large since many modern graphics cards do a pretty good job of hiding latency for texture fetches by interleaving multiple 'threads' of pixels. On a card with 24 pixel shader units for example you'll get 6 quads execute a texture fetch instruction for 24 pixels in one thread, then when the data is needed for an operation you'll switch to another thread and issue another texture fetch instruction for a different group of 24 pixels using the same pixel shader. The card will keep switching threads as long as there's work to do that isn't waiting on a texture fetch and as long as it has enough resources to handle all the simulatneous thread contexts. Ideally by the time you switch back to the first thread the texture data you requested will have arrived from main memory and there's no need to stall. In other words the texture cache is actually less important than you might think as long as the card can keep busy doing other things.

The main thing to worry about with the texture cache is incoherent memory accesses - this happens when you don't have mip mapping enabled or when your pixel shader is looking up widely spaced texels in adjacent pixels (like with environment mapping using a very bumpy normal map). For regular albedo textures or any conventionally mapped texture where adjacent pixels generally access nearby texels the texture cache will work pretty well.

Excelent ! Thank you for this pretty explaination.

Quote:

Original post by mattnewport On a card with 24 pixel shader units for example you'll get 6 quads execute a texture fetch instruction for 24 pixels in one thread, then when the data is needed for an operation you'll switch to another thread and issue another texture fetch instruction for a different group of 24 pixels using the same pixel shader. (...) In other words the texture cache is actually less important than you might think as long as the card can keep busy doing other things.

1) Is there tricks to improve this ?

Correct me if I'm wrong : the card can either switch to an other thread or it can just perform instructions of the same thread that don't need the texture fetch instruction result ?
For example, I'm thinking about do a texture fetch in the shader code, then do some instructions that don't need its result, and then do instructions that use it. Is this correct ?

2) Is the thread switching really free ?

An other time, correct me if I'm wrong but all the information of the first-thread\pipelines must be saved to allow thread switching ?

3) The last, where it is possible to have that kind of informations [smile] ? Is there paper on the cards architecture ?

The beyond3d forum is a good place to talk about hardware design of GPUs. You can find many information’s in the patent database, too. But as the written in some kind of “patent language” they are sometimes hard to read.

At the moment only ATI use a threading system to distribute work on the units. nVidia use a different method. The ATI system works with a large pool of threads and every time the texture unit or the ALUs/FPUs finished an instruction on all pixels in a thread another one will be assigned. This makes it more like a job scheduling system than a threading system you may know from the CPU. Using many threads makes it possible to execute anything in order.

Yep, Beyond3D is a good resource, both the forums and the articles. There's some pretty in depth articles on both ATI's latest architecture and nVIDIA's, as well as a good article on the Xbox 360 GPU.

There's no one place you can get this sort of information, check out nVIDIA and ATI's developer sites and work your way through some of their documents and presentations. The GPU Gems 2 chapter on the GeForce 6 architecture is worth a look. Other than that keep an eye on the Beyond3D forums and mailing lists like the DirectXDev list.

ATI use the thread pool approach to hide the latency of texture fetches in their latest hardware. I'm not exactly sure how nVIDIA hardware works but it has the same effect in that it's able to hide texture fetch latency by working on other pixels. You're right that latency can also be hidden by doing work that doesn't need the results of the texture fetch in the same thread but generally you don't have to worry too much about scheduling this yourself - the backend compiler in the driver that converts D3D shader code into hardware specific microcode will usually do a good job of rescheduling instructions to hide latency and can take advantage of hardware specific knowledge not available to you. What you can do is try and ensure that there is other work in your shader that the compiler can schedule though - try to have a mix of texture fetch instructions and arithmetic instructions. The exact balance you need for best performance will depend greatly on the hardware though.

On ATI hardware the thread switching isn't completely free - the hardware has a limited pool of general purpose registers and other context for a group of quads and once that's used up the card will have to stall until the texture data arrives. I imagine similar limitations apply to nVIDIA hardware but I don't know the details of how that works. Generally, unless you're working on consoles where you know the exact details of the hardware, you can't do much other than the obvious optimizations of doing as little work as possible in the pixel shader to achieve the results you want.

Thank you for all guys =)

Quote:

Original post by mattnewport snip

Very very interresting.
Thank you for the link too ! I will study it in depth very soon.

I immediately go to the beyond3D forum.