Scene graph and external resources

At some point, the tree is really just a list of nodes, each with a set of data and a reference to their parent node, right? The data in this case is just a reference to the model as opposed to full node data. The difficult part may be how to store the child of the model, since that points to a parent that is in the model file? How do you currently create the tree in this situation?

Let's make some important distinctions:

1. Are your resources loaded statically or dynamically?
2. Are your resources modifiable?
3. Do they move?

Example: embedding a statue in world geometry.

In this case, try to load it at 'map compile time' and bake it in the geometry. Pre-transform everything if needed. In line of principle this is very hard to do (if at all possible) for arbitrary shaders. Just keep it easy.

OFC if you do this, at runtime you just get a big scene graph.

Or perhaps save yourself the work and import as a dynamic model.

Those kind of meshes are often immutable and non-moving. They can be embedded in the world geometry with ease and leverage the whole content pipeline. Your example however is somewhat more complicated as apparently you want to attach to an imported node.

Example: attach rocket launcher to custom player model right hand

Real bad: you don't know what model to load in advance, maybe you won't even have it but you only know the right hand is marked with a specific tag e.g. right_hand.

I'm pretty sure fbx can export skeletal infos or other kind of annotations you can use to extract the information you need. Then you will need a dynamic system to pull out those informations and propagate them correctly.

SVG for example does not have this information. Months ago I used specific syntax to identify the required points and reconstruct the information. OFC this was non-portable, non-standard and in general not recommended but you can do it.

How you do it is up to you to design. I had things I dubbed 'value synchers' which would connect specific logic (even scripts) to GPU buffers. One syncher in would walk an imported mesh and extract special points to drive particle emitters, another could extract accumulated transforms. Inheritance will most likely be your friend here.

Corollaries

• World geometry very special; it usually takes loads of geometry, covers large parts of the screen, most of the time, it is almost never fully visible. In probably needs its own scene graph to scale and a way to be partially drawn.
• Dynamic meshes are often instantiated multiple times, be sure to recognize the difference between a mesh and an instance of the mesh itself. An instance is usually small enough you can either draw or not draw it.
• In my experience, mixing dynamic stuff with static structures is recipe for disaster... but perhaps you might be fully dynamic and still be good enough for you.
• I would suggest against using fbx or any other content-creation oriented format for that matter. They can get really complicated real fast and you are pretty much guaranteed to be unable to support all their features, which implies you will have to keep a list of what is supported and what isn't.
• You always have a moment in which you know what is loaded. Be sure to know where to get that information. Most will come from 'level geometry' file. Some might come from network, you likely have been mangling this dynamic data at some point, just don't throw away the information!

Thanks for replies, guys!

Let me clarify my situation a bit:

1) My resources are modifiable - I can create a save file which uses a resource and then I can modifiy it, and I still want my save be loaded correctly (as much as possible, of course).

2) Resources can be loaded during runtime at any moment

3) Resource instances (nodes with shared graphical data) uses their own nodes (copy of nodes from model) - this gives me ability to animate nodes independently for each instance and still get benefits of instanced rendering.

My very first thought was to just save some path to a resource file along with node data. Also create a section of resource references in a save file. Then on load I can load resources first, put them in separate graphs. In the moment when I'm loading a node with a path to resource I can try find node in the loaded resource by its name and restore graphical information from it (basically - meshes which I need to render). But what should I do if hierarchy of nodes in resource was changed?

I've seen that Unity engine doing a good job when model's asset changes - it is somehow keeps nodes on scene in sync with nodes in resource, but preserving transforms of nodes which were modified by user. I've tried to search some papers about this, but unfortunately - no luck.

5 hours ago, mrDIMAS said:

My very first thought was to just save some path to a resource file along with node data. Also create a section of resource references in a save file. Then on load I can load resources first, put them in separate graphs. In the moment when I'm loading a node with a path to resource I can try find node in the loaded resource by its name and restore graphical information from it (basically - meshes which I need to render). But what should I do if hierarchy of nodes in resource was changed?

 I've seen that Unity engine doing a good job when model's asset changes - it is somehow keeps nodes on scene in sync with nodes in resource, but preserving transforms of nodes which were modified by user. I've tried to search some papers about this, but unfortunately - no luck.

This topic is actually not that much related to game engine (assuming your engine has some sort of scenegraph, and resource management - which most engines have), you are bumping into the are of actual tools. Tools as software don't have that many resources available at all - and are extremely time consuming to build up (and extremely rewarding - at least for me) ...

I do a post from time to time into my blog here regarding my hobby game engine, and most of the time I spend on the project is in tools.

In Skye Cuillin^(TM) ? editor I have managed resources for textures, models, etc. When their file is changed they are hot-loaded and replace original resource, some details are provided here (shameless self promotion):

This works for resources that are directly referenced (textures are the best examples), it will also work for meshes (from engine's point of view mesh is a single geometry). Generally this action can't be undone (because resources are changed outside of the actual software)!

The actual resource (like obj) is a model (not mesh) resource, which - when instantiated into scene - will create empty entity and then push in whole hierarchy from file. In case you would change geometry data for one of the meshes, your meshes in editor would be hot loaded and changed (this is because mesh data is referenced within MeshComponent on the node - they don't actually contain mesh).

If you would change hierarchy itself - then that is a problem!

There are few ways how to solve this:

• Replace EACH node hierarchy that was instantiated from this resource (this will cause issues when you F.e. moved one of the children under different parent, or added different node as child of node instantiated from resource)
• Replace only those node hierarchies that resemble model exactly before the load (which means you need to mark those that should be replaced - and then do so) - this is what actually Unity seems to be doing
• Ignoring the problem and doing nothing with old hierarchies is also a way to solve this problem, a perfectly valid one (probably better than doing the 1st solution)

I currently apply 3rd solution. It is the most straight forward. I'm still considering supporting the 2nd one (if I would hold resource reference in node that was one used as topmost storage when instantiating resource into scene, along with flag whether it is dirty (not to reload) or not (to reload)), then it would work perfectly.

Of course all of this applies only in editor (where actual saved scene is just saved scenegraph, and resource files are around). For compiled scene (for runtime), the scenegraph is still saved .... along with all resources compressed. I'm currently working on this part now.

After some long break in solving this problem I finally implemented working solution.

Each scene node has a pointer to resource from which the node was created, also each node contains pointer to original node that stored in resource. When I'm doing a save, at first I'm saving resource descriptors (just a file name and some meta info), then I'm saving nodes with info about which resource is it using. On loading for each node I'm resolving pointers to original nodes from resource as well as pointers to resource (scene.c and node.c). Pointers resolved by node names - which is quite unreliable because node name can change and then it is impossible to find from which node in resource a node was instantiated, but I'm still do not know better way. In similar way I'm restoring pointers to render data for meshes ( mesh.c) - just find a node in resource and grabbing all render data from it.

But still I need to somehow resolve changes of position/scale/rotation (transform) of nodes in resource so when I'm loading a save nodes on scene will get exact transform from resource. This can be done by saving a flag near node attribute like "has_changed" or something which is indicating that node attribute was changed during runtime. Then on load I can check this flag and if it is not set, then set attribute of node using value from resource.