🔒 Locked

Resources - Template Specialisation

Started by beebs1 Jun 30, 2012 at 7:25 AM 3 replies 1.6k views

Original Post

Jun 30, 2012 at 7:25 AM

Hiya,

I've implemented a template-based resource management system based on the ideas mentioned by rip-off (here and here). It uses template specialisation, which I haven't used before, so I was hoping some could confirm whether what I've done is reasonable (or not).

The system uses a template class called ResourceLoader, which is used by a separate cache (also a template) to load and instantiate resource objects. Because each resource type needs different data (and parameters) to construct, I've created specialised loader classes for each type.


 // Used to read the actual resource file bytes from an archive/wherever.
 class ResourceLocator;
 
 template<typename T>
 class ResourceLoader
 {
 public:
 
 ResourceLoader(const ResourceLocator *locator) : _locator(locator) {}
 
 private:
 
 ResourceLocator* _locator;
 };
 
 // Simple example: vertex shader loader.
 class VertexShader;
 
 template<>
 class ResourceLoader<VertexShader>
 {
 public:
 
 // Load the named vertex shader.
 std::shared_ptr<VertexShader> load(const std::string& name)
 {
 // Read the shader source from disk (doesn't matter how).
 std::vector<char> buffer;
 _locator->read(name, buffer);
 
 // Construct the shader, passing it's source code as a parameter.
 std::string shaderSource(buffer.begin(), buffer.end());
 return std::make_shared<VertexShader>(shaderSource);
 }
 };

Each resource type needs to be loaded in a different way, so I don't have a truly generic load() method in the ResourceLoader class itself - which I think is the correct way of doing this, although it seems a little odd.

Seem reasonable?

Thanks for any advice. Cheers!

Dario Oliveri

Jun 30, 2012 at 9:10 PM

You can have a generic resource locator with templates, but you still need to create a different loader for each type of file. I usually prefer templates
for container and when some recursive code creation is needed (just used once for that). For all other things I prefer polymorphism (where a different implementation for each loader is explicitly needed since common interface is pure virtual).

If every item you can create can be created with the same constructor...

example:


 class Resource //provide a way to know if file was loaded or not
 {
 protected:
 bool Good; // resource is loaded
 
 };
 
 
 class VertexShader: public Resource
 {
 public:
 VertexShader(AbstractFile * file); //abstract file, allow to load from HD's files but also from other sources (if needed in future)
 
 };
 
 class FragmentShader: public Resource
 {
 public:
 FragmentShader(AbstractFile * file);
 
 };

... then you can just have a simple template loader (don't know how this can be usefull to you)


 template <typename T>
 class ResourceLoader
 {
 std::shared_ptr<T> load(AbstractFile * file) 
 { 
 T * res = new T(file); 
 return std::make_shared<T>(res); 
 }
 };

vreality

Jul 1, 2012 at 10:31 AM

I'm not sure you're gaining much by using templates here, since you're using the loaded data, and you must handle each type differently.

I take a slightly different approach.


class asset_loader
{
  public:
    template<typename T> void Registrar(function<function<void()>(T *)> const & Register);
    unique_ptr<asset_package> LoadPackage(char const * PackageName);
    ...
};

The basic ideas here are:

Assets of various types are generally grouped together into a file to be loaded all at once. I call that an "asset package".
There are various engine sub-systems which will end up owning and using the loaded assets. You associate member functions of those systems with specific asset types, using the templated member of asset_loader. Then, when it's loading assets, it calls those functions to submit each loaded asset to its respective system.
Loading assets produces an asset package object which acts as a sort of smart pointer to all the assets that got loaded. When this object is deleted, it unregisters and deletes those assets.

It's used like this:


asset_loader AssetLoader;
AssetLoader.Registrar<model_asset>(bind(&renderer::Model, &Renderer, _1));
...
unique_ptr<asset_package> pAssetPackage(AssetLoader.LoadPackage("MyAssets"));
// Renderer.Model(model_asset &) will get passed each model_asset in the package.
...
pAssetPackage.reset(); // (or deletion of the unique_ptr) Signals for deletion of all associated assets.

Admittedly this approach is significantly more complicated.

But note that renderer::Model(model_asset &) - which is an example of a function which would receive actual loaded data, and perhaps create a driver owned resource from it - is a unique member of a unique class which owns and processes model data. Loading and lifetime management is handled generically across asset types, but specific data processing is not.

beebs1

Jul 3, 2012 at 8:46 AM

...

Thanks - I've just given that a try, by making each resource take a std::vector of the file contents in the constructor. I've come across a couple of problems though - I'm not too keen on resource classes parsing files themselves (seems like the locator/loader classes should be doing that), and some resources don't fit well into this pattern. An example is a shader program, which needs to be constructed from vertex shader and fragment shader resources and can't be loaded from disk per se.

...

I hadn't considered using function binding to create the API-side assets, that's interesting. For now I'm OK with the resource classes themselves doing it, as I'm using OpenGL for this project and resources aren't creating using driver factories like in D3D - but I'll definitely give it a go when I next use D3D.

I might have misunderstood, but I'm assuming that your asset_package class somehow knows how to parse and generate model_asset objects before passing them to the registered renderer. This is the part I'm having difficulty with - the best way to implement a loader for different types.

Thanks again for the replies!

Cheers.

[Edit] Rayne - I seem to remember you mentioning an article series about asset management was on the way (might have been way back) - did something materialise?

vreality

Jul 5, 2012 at 2:14 AM

On 7/3/2012 at 1:46 AM, beebs1 said:

I'm assuming that your asset_package class somehow knows how to parse and generate model_asset objects before passing them to the registered renderer. This is the part I'm having difficulty with - the best way to implement a loader for different types.

That's the basic idea.

My asset types are defined in code shared by the engine and tool. They're memcpy-able structures, with the exception that they can utilize an "asset_ptr" which can point to any other asset type object or array of objects. In the tool, the asset_ptr<> classes participate in serialization magic such that if you dump any asset object, the serialization code automatically detects any contained pointers, and first dumps the referenced objects or arrays, and converts the pointers to dumped-object indices. The generated asset package includes a list of all included objects and all included pointers, which is used to fix-up the pointers on loading, along with a hash of the top-level object type names, which is used to select the registration function which receives the loaded asset.

A run-time asset_package object, actually contains nothing but a collection of call-backs (std::function's) which will "un-register" those assets from the systems with which they were registered, when the asset_package is deleted. (The actual asset objects may have been deleted during registration. The owning systems handle that, and generate the appropriate un-register callback for each asset registered.)

My scheme is designed to "genericize" asset (de)serialization, so that you can define new asset types, create them, dump them, load them, and use them without maintaining any additional serialization code. So all loading, construction, fix-ups, and handoffs are handled generically by the asset_loader (mostly, without knowing the types of the objects it's handling).

Alternatively, you can maintain custom serializers and parsers, and either load every asset individually, or dump an asset type name hash with each asset in a package, and use that to look up a custom parser.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.