Skip to main content
gamedev.net
🔒 Locked

optimizing cache usage

Started by Dragon_Strike Apr 9, 2009 at 6:16 PM 3 replies 8k views
Original Post
Dragon_Strike
Dragon_Strike
i have two images which im blending... im doing this with SSE and i simply loop over all the pixels and peform the calculations... it looks something like this... 

void foo(unsigned char* pDest, unsigned char* pSrc, int count)
{

__m128i* dest = reinterpret_cast<__m128i*>(pDest);
	__m128i* source = reinterpret_cast<__m128i*>(pSrc);

	_mm_prefetch(reinterpret_cast<char*>(&dest[0]) , _MM_HINT_NTA);
	_mm_prefetch(reinterpret_cast<char*>(&source[0]) , _MM_HINT_NTA);

	int length = count/sizeof(__m128i);
	for(int n = 0; n < length; n+=4)
	{
		// prefetch a cache line
		_mm_prefetch(reinterpret_cast<char*>(&dest[n+4]) , _MM_HINT_NTA);
		_mm_prefetch(reinterpret_cast<char*>(&source[n+4]) , _MM_HINT_NTA);

              // UNROLL 4 TIMES AND DO CALCULATIONS
  
              d = _mm_load_si128(&dest[n+0]);
	      s = _mm_load_si128(&source[n+0]);
                  // DO CALC
              _mm_store_si128(&dest[n], r);

              d = _mm_load_si128(&dest[n+1]);
	      s = _mm_load_si128(&source[n+1]);
                  // DO CALC
              _mm_store_si128(&dest[n+1], r);

              d = _mm_load_si128(&dest[n+2]);
	      s = _mm_load_si128(&source[n+2]);
                  // DO CALC
              _mm_store_si128(&dest[n+2], r);

              d = _mm_load_si128(&dest[n+3]);
	      s = _mm_load_si128(&source[n+3]);
                  // DO CALC
              _mm_store_si128(&dest[n+3], r);
        }
}
for each loop i do a non-temporal prefetch for the data used in the next loop... i then do calculations for all the pixels in the cache line... im wondering if this is the optimal way to use cache? is there anything i can improve? maybe prefetching several cache lines? EDIT: the reason im doing is that when i had no prefetching in my code and looked at the resulting assembly i found that the compiler had not added any prefetching... which is why i am now adding prefetch hints to the compiler
InvalidPointer
InvalidPointer
It sounds good, but then again I haven't done much with manual prefetching yet. Loop nest optimization (see the wiki article) also seems appropriate.
clb: At the end of 2012, the positions of jupiter, saturn, mercury, and deimos are aligned so as to cause a denormalized flush-to-zero bug when computing earth's gravitational force, slinging it to the sun.
samoth
samoth
Prefetching makes sense if and only if the processor has a chance to fetch the cache line before you access it. Usually, accessing RAM (not data in cache) takes upwards of 300-400 cycles on current CPUs[1], so that's what you should at least prefetch as a minimum, else you only add instructions that do nothing.

Also do note that NTA tells the processor that you only intend to access that data once, which means that the processor may purge these lines much earlier. That may be good or it may be bad, and it's impossible to give a generally correct answer (would have to test).
Personally, I am inclined never to use NTA these days, since processor caches are getting bigger and bigger, and eventually at the next context switch the caches will be flushed anyway, so it's unlikely that ages old stuff will clutter the cache for too long. This, however, is opinion-based, not evidence-based.

[1] The guy who wrote 7zip made a CPU and cache test program which you can use to see how much it's on your computer. It's somewhere on the 7zip site under "tools".
Dragon_Strike
Dragon_Strike
ive done some experimenting and ive found this to be faster

	static const uint32 NUMBERPAGE = 128;	__m128i d = _mm_setzero_si128();	__m128i s = _mm_setzero_si128();	__m128i* dest = reinterpret_cast<__m128i*>(pDest);	__m128i* source = reinterpret_cast<__m128i*>(pSrc);	int length = count/sizeof(__m128i);	for(int k = 0; k < length; k+=NUMBERPAGE)	{                // prefetch 128 cachelines *2 = 16,384 kb		for(int n = k; n < k+NUMBERPAGE; n+=4)		{			_mm_prefetch(reinterpret_cast<char*>(&dest[n]) , _MM_HINT_NTA);			_mm_prefetch(reinterpret_cast<char*>(&source[n]) , _MM_HINT_NTA);		}                // unroll so that it works on one cacheline per iteration		for(int n = k; n < k+NUMBERPAGE; n+=4)		{			// UNROLL 1 //						// DO CALC			_mm_store_si128(&dest[n], d);			// UNROLL 2 //			d = _mm_load_si128(&dest[n+1]);			s = _mm_load_si128(&source[n+1]);						// DO CALC			_mm_store_si128(&dest[n+1], d);			// UNROLL 3 //			d = _mm_load_si128(&dest[n+2]);			s = _mm_load_si128(&source[n+2]);			// DO CALC			_mm_store_si128(&dest[n+2], d);			// UNROLL 4 //			d = _mm_load_si128(&dest[n+3]);			s = _mm_load_si128(&source[n+3]);						// DO CALC			_mm_store_si128(&dest[n+3], d);		}		}


im still unsure on what to base NUMBERPAGE on...

ive tried __mm_stream but its ALOT slower than _mm_store

im unsure how Loop nest optimization will work here?

im also unsure how many SSE registers there are... maybe i should load to four different "d"- and "s"-s at the same time and store them at the same time

d1 = _mm_load_si128(&dest[n+0]);
d2 = _mm_load_si128(&dest[n+1]);
d3 = _mm_load_si128(&dest[n+2]);
d4 = _mm_load_si128(&dest[n+3]);

s1 = _mm_load_si128(&source[n+0]);
s2 = _mm_load_si128(&source[n+1]);
s3 = _mm_load_si128(&source[n+2]);
s4 = _mm_load_si128(&source[n+3]);

_mm_store_si128(&dest[n+0], d1);
_mm_store_si128(&dest[n+1], d2);
_mm_store_si128(&dest[n+2], d3);
_mm_store_si128(&dest[n+3], d4);

would that make sense?
samoth
samoth
Quote:
Original post by Dragon_Strike
ive tried __mm_stream but its ALOT slower than _mm_store

The same applies as for PREFETCHNTA. With __mm_stream, you tell the processor: "write this out, and by the way, I will not access it again any time soon". So, when you access it again in any way, it burns 300-400 cycles on a cache miss.
When applied correctly, __mm_stream (example: write data into a mapped vertex buffer object) is better than __mm_store, but when applied incorrectly, the opposite happens.
Dragon_Strike
Dragon_Strike
Quote:
Original post by samoth
Quote:
Original post by Dragon_Strike
ive tried __mm_stream but its ALOT slower than _mm_store

The same applies as for PREFETCHNTA. With __mm_stream, you tell the processor: "write this out, and by the way, I will not access it again any time soon". So, when you access it again in any way, it burns 300-400 cycles on a cache miss.
When applied correctly, __mm_stream (example: write data into a mapped vertex buffer object) is better than __mm_store, but when applied incorrectly, the opposite happens.


i am aware of this... the reason im confused is that i dont access that value again once ive stored it...

for each pixels
i load it "d = _mm_load_si128(&dest[n+0]);"
do the calculation
and store the result (r) "_mm_store_si128(&dest[n+0], r);"

and i dont access it anymore... feels like im missing something

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.