Skip to main content
gamedev.net
🔒 Locked 🎮 Unity

(dynamic)texture cache

Started by f1restar Dec 8, 2005 at 7:09 AM 4 replies 750+ views
Original Post
f1restar
f1restar
Hi, ive been trying to implement a texture cache for a while now. The caching seems to work however ive got some problems mapping the cache/textures to specific coordinates. I have a system that updates textures every frame (not all textures) so what ive done is created a simple ref counted vector of textures. The textures are pre-allocated, so I only update them using glTexSubImage2d (which is way faster than creating a new texture every frame). The problem comes from the fact that a texture is mapped to multiple xy coords, so I need to have a jump table with the index into the cache. The system then searches thru the cache and sees if it can find a texture with the correct image id, if the id doesnt exist it will update a texture in the pool with the new data and id, and give it an low reference count. The mapping doesnt work as I expected and im a little lost in what is wrong. I have 2 questions, is this kind of caching the way to go? There are around 500 textures to be updates each frames, but it can be as low as 5 or as high as 4000. I have succesfully tested it without caching and it runs at 10 to 15 fps depending on how many textures there are to update. I hope that with some caching I can increase it at least to 25 fps. The second question, if anybody can look at the code and see if theres something i've missed or a better way to do it? For the ones that are wondering im part of a collective that is working on a realtime mosaic generator. The generator itself runs at over 40 fps ;) but we want to add a lot of fancy opengl effects to it, so thats where I come in. Sorry for the long code dump ;) 

CTextureCache::CTextureCache() {
	std::cout << __FUNCTION__ << " Creating cache of size " << 64 * 48 * 4 << std::endl;
	for ( int t = 0; t < 5000; t++ )
		xyCoords = new int;
	for ( int t = 0; t < 5000; t++ )
		idCoords = new int;

	for ( int i = 0; i < 500; i++ ) {
		char blaat[ 64 * 64 * 4 ];
		memset( blaat, 0xf0f0, sizeof( blaat ) );
		textureCache.push_back( new CTexture( ( unsigned int* ) blaat, 64, 48 ) );
	}
}

void CTextureCache::update() {
	unsigned long tg_cnt = *( ( unsigned long* ) CObjectManager::getSingleton().getVariable( "tilecount" ) );
	TileGrid* tg = reinterpret_cast<TileGrid*>( CObjectManager::getSingleton().getObject( "tilegrid" ) );

	int tilecount = 0;
	std::sort( textureCache.begin(), textureCache.end(), CTexture::TCompare() );
	for ( int i = 0; i < tg_cnt; i++ ) {
		if ( tg[ i ].changed ) {
			bool cacheHit = false;
			int cacheHitIndex = -1;
			int t;



			textureVectorType::iterator iter;
			int iterCount = 0;


			for ( iter = textureCache.begin(); iter != textureCache.end(); iter++ ) {
				iterCount++;
				if ( ( *iter ) ->tileID == tg[ i ].ID ) {
					cacheHit = true;
					( *iter ) ->m_priority += 5;
					( *iter ) ->x = tg[ i ].x;
					( *iter ) ->y = tg[ i ].y;
					xyCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ] = iterCount - 1;
					idCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ] = tg[ i ].ID;
					if ( ( *iter ) ->tileID != textureCache.at( iterCount - 1 ) ->tileID )
						std::cout << "Not equal!!" << std::endl;
xyCoords[tg.x + (tg.y*64)] = tilecount++;
					break;
				} else {
					xyCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ] = iterCount-1;
					idCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ] = tg[ i ].ID;
					if (textureCache.at(xyCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ])->tileID != 
						idCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ])
						std::cout << "Not matching indexes!" << std::endl;

					if ( ( *iter ) ->m_priority > 250 ) {
						( *iter ) ->m_priority -= 1;

						if  (  (  *iter )  ->tileID !=  textureCache.at(  iterCount -  1 )  ->tileID )
							std::cout << "Not equal!!" << std::endl;


					}
				}
			}

			if ( !cacheHit ) {
				iter = textureCache.end();
				iter--;
				( *iter ) ->x = tg[ i ].x;
				( *iter ) ->y = tg[ i ].y;
				( *iter ) ->tileID = tg[ i ].ID;
				( *iter ) ->updateTexture(
				    ( unsigned int* ) tg[ i ].tileptr, 64, 48
				);
				( *iter ) ->m_priority = 4;
				xyCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ] = textureCache.size() - 1;
				idCoords[ tg[ i ].x + ( tg[ i ].y * 64 ) ] = tg[ i ].ID;
				if ( ( *iter ) ->tileID != textureCache.at( textureCache.size() - 1 ) ->tileID )
					std::cout << "Not equal!!" << std::endl;
			}


		}
	}
	cacheRatio = tilecount * 100 / tg_cnt;

}

GLuint CTextureCache::getTexture( int x, int y ) {
	int coordIndex = abs( xyCoords[ x + ( y * 64 ) ] );

	// 		std::cout << "xycoords " << coordIndex << std::endl << std::flush;

	if ( idCoords[ coordIndex ] != textureCache.at( coordIndex ) ->tileID )
		std::cout << "IDs dont match!" << std::endl;

	if ( coordIndex < 500 ) {
		if ( textureCache[ coordIndex ] != NULL )
			return textureCache[ coordIndex ] ->getID();
	}

	return 0;
}
f1restar
f1restar
Just alittle kick, nobody using a texture caching system in there engine? The caching works, but its much slower than just updating textures everyframe, without searching for ids or reference couting. The frame rate does go up from 1 to about 15 when the references are filled.

Anyway are there other aproches to this? Because this doesnt seem like the most efficient way.

Oh and if a mod reads this? Maybe its better off in graphics programming?
FReY
FReY
Okay, I'm not *exactly* sure of what you're trying to do, but I'll guess. You're trying to keep a list of 'tiles' that are both stored by the driver (in the form of a texture or multiple textures) and by your program (in the form of your texture cache?). And when a tile is updated in your program you want to reflect the change in the associated texture?

Some questions to help me understand better:
* are you trying to store all your tiles in one texture?
* if you're storing tiles in a single texture it may be worth updating the dirty region of the single texture rather than making multiple driver calls to glTexSubImage on the same texture :)
* why are you sorting your texture cache upon update? priority?
* why bother with priority if you need all your textures updated on the same frame?

First things first, in your CTextureCache constructor these lines seem to be extraneous:

    // your original code:    for ( int t = 0; t < 5000; t++ )        xyCoords = new int;    for ( int t = 0; t < 5000; t++ )        idCoords = new int;    // surely you mean this?    xyCoords = new int[5000];    idCoords = new int[5000];



Some things that spring to mind:
* are you regenerating mipmaps every frame? it could be quite time consuming.
* try and profile your code to see what's actually taking most of the time.

Hope I haven't completely barked up the wrong tree and something there helps... ;)
do unto others... and then run like hell.
f1restar
f1restar
Actually its more complex. I allocate a texture pool, with pregenerated (black/empty) textures. I then update them according to there priority, so least accessed texture gets erased/updated.

The idea behind this is that you dont have to search the whole array for commonly used textures. So if an image with id 20 is used 1000 times itll be in the front of the array, and it will find it after only one iteration, instead of say 100. Basicly a simple refcounting cache :)

My problem is, are there better methods to map these sort of coords (the textures are one big image stream, that is mmap'ed). But its all individual textures of NxM (64x48 in this case). Thats why its updating a pool of textures, every textures is different, but I dont want to create a whole new texture, cause that would slow everything down.

So in pseudo code it would be something along the lines of:
loop thru the tilegrid   if a tile has changed       check if tileid is allready in the texture pool      if tileid exists update ref count with +1      if not find least used texture and override that with the new one.


But im having severe problems if I up the size of the texture pool, because it has to loop tru every texture in the pool to see if the id matches. So any suggestions?

Btw is there a way to get the current position of an iterator? As you can see I want to have a jump table, with a simple equation along the lines of x+(y*tilewidth) but for some reason it seems to screw up and updates the wrong position.
extralongpants
extralongpants
Quote:
Original post by f1restar
Actually its more complex. I allocate a texture pool, with pregenerated (black/empty) textures. I then update them according to there priority, so least accessed texture gets erased/updated.

The idea behind this is that you dont have to search the whole array for commonly used textures. So if an image with id 20 is used 1000 times itll be in the front of the array, and it will find it after only one iteration, instead of say 100. Basicly a simple refcounting cache :)

My problem is, are there better methods to map these sort of coords (the textures are one big image stream, that is mmap'ed). But its all individual textures of NxM (64x48 in this case). Thats why its updating a pool of textures, every textures is different, but I dont want to create a whole new texture, cause that would slow everything down.

So in pseudo code it would be something along the lines of:
loop thru the tilegrid   if a tile has changed       check if tileid is allready in the texture pool      if tileid exists update ref count with +1      if not find least used texture and override that with the new one.


But im having severe problems if I up the size of the texture pool, because it has to loop tru every texture in the pool to see if the id matches. So any suggestions?

Btw is there a way to get the current position of an iterator? As you can see I want to have a jump table, with a simple equation along the lines of x+(y*tilewidth) but for some reason it seems to screw up and updates the wrong position.


Have you tried using a map? You could try using std::map, with a key of ID and a value of Texture (or Texture*).

Of course, this data structure wont cache your most used textures (but its a lot faster than finding a matching id in an array).

You may also want to continue using a vector, but keep textures in order of their ID - that is, the texture with an ID of 0 would be at vector[0], and the texture with an ID of 5 would be at vector[5]. Every time a texture is removed, add it's ID to a list of available ID's. Then, when a texture is created, give it an ID in that list, or if the list is empty, give it an ID that is the number of textures+1. It's a little harder to keep up if you're constantly deleting/creating textures, but it should be quite fast. It doesn't really cache textures, but it also doesn't need to.

Good luck.


f1restar
f1restar
I cannot map them directly to a vector because the database can contain >150.000 ids. I could use a hashmap but same problem there how do you contain it to say 100 ids? Delete ids that are not used? Then u need to iterate over the entire map to find ids that are nog used anymore. Thats also the reason behind sorting it based on id usage.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.