agnostic fx file based engine

I also think this is an issue.
I want to keep stategroups well organized in what I'm working on now. I found this requires at least some parsing. Luckly, identifying what I (you) need seems trivial.

I must say however I'm still not very confident with .fx files.

Yeah, but I can't help feeling that the fx file should be requesting what it needs from the engine so it would be possible to have the engine completely agnostic about the requirements of the fx file.

Of course this does reverse the roles somewhat in that the author of the fx file has to know about the engine capabilities etc - but it does mean that the engine does not have to be re-coded to cope with a new fx file which has to be a win.

Quote:

Original post by XiotexStudios Now, my original understanding over the hype of fx files is the ability to just drop in a new fx file and that will act as your material - except from my new experience - this isn't quite true. For example - what if the new fx file requires an extra texture to be loaded, your code needs to know this in order to fetch that texture.

Look at some .fx files from DX SDK samples. Some of them use annotations to mark texture variables and give them descriptions in form of strings. Your engine should then look for annotation values and load the textures accordingly.
I can't give you any more general advice, only for your example, since I really can't see any other possible issues here (care to elaborate?).

IMO, that kind of stuff you're (as far as I can see) requesting, does not belong to fx files, but rather to model file, or gui xml(or whatever) descripton.

Quote:

Original post by XiotexStudios It's entirely possible that I have missed some fundamental point about dealing with fx files, but how are people using them realistically. One approach I have seen is to make sure all vertices have exactly the same format (leading to redundant data being transfered across the bus) and implement a common set of variables that each fx file has to be authored to. Even then though to me this seems like still a 'push' technology where your code has to have some knowledge of the possible fx files it has to handle.

Again, this should belong to model description file, IMO. From that, you could create proper vertex declaration. What kind of data your vertex shaders are pulling from the vertex buffers already IS agnostic to content and format of such a buffer. It is true that this can sometimes lead to unneeded data kept in the GPU memory, and some programmers are addressing this, by writing some (basic in form) descriptions on how model data should be rearranged. But since models are completely independent resources from effects (relation is not always 1:1), one cannot modify a model based on what's written in one of effect files associated with it. You should know that kind of needs beforehand.

Quote:

Original post by XiotexStudios Is there a way of using them in a truly agnostic way where your engine does not need to know any details of the fx file and [fx file] uses a kind of 'pull' technology to demand details/assets from the engine?

(fixed?)

If you're anyhow linking one component (engine) to another (effect), at least one of the sides has to have the knowledge of the other side. By this, I mean: predefined set of constants/names/enums/etc., that it uses to communicate.
Either: one side is asking "What do you need?", or another is asking "What do you have?". But somebody has to be asking questions.

Unless you're using standard semantics only, you can't expect your agnostic engine to be able to use a totally random fx file, no matter how complex.

Good luck.
~def

Good reply.

Perhaps I should explain further:

My current setup uses a series a text files to describe a renderable.

The first is the actual 'sprite' file (not to be confused with 2D sprites) that defines the mesh to load and the materials it is to use. Materials are a mixture of textures and a renderstate name which when invoked sets up the render-states/shaders etc to render that sprite. The 'renderstate' name is actually a class and one of the functions of the class is to prepare the mesh into the vertex format needed.

Now, I have lots of these renderstate classes and I would like to replace them all with an effect file type system. What I envisage is a single class that renders the mesh with the fx file - thereby allowing me to reduce the number of code paths my engine has - a bonus!

However, I still need to be able to work out what are the requirements of this fx file in order to allow it to operate. How many textures does it require? Does it need a light vector? Does it need a colour? etc...

I understand the annotation system and I could see how to use it but that seems like a brute-force method of asking the fx file lots of questions when it would be much more elegant to just have the effect file supply a list of its needs - although perhaps this could be done with annotations?

I am aware that a catch-all solution is a pipe-dream but then again so is the idea of just dropping in fx files and expecting them to magically work.

Digging deeper I found in the docs something I have seen referenced but didn't take much notice of... DXSAS.

Looks promising.

I see that you already tried everything I described, hm...

As I see it, there are three kind of things that an fx would wat from the engine:
1. a texture
2. a vector (or a vector array)
3. same as 2, but some with dynamic choosing, for example: 3 nearest lights, or terrain color under a unit being displayed.

Considering 1 and 2.
I'm guessing that not every arbitrary data chunk from the engine could be referenced from the outside. But the engine could introduce some kind of handles to the data that is likely to be referenced (or just a std::map). In an fx file, variables interested in being updated with such a data would have to have annotations, like:
string EngineData = "env cube map"
string EngineData = "sun color";

Then, when loading an fx file, you would get all handles of all such variables and match them with corresponding engine's "external" variables.

Considering 3.
Don't know... (yet[grin])

Good luck!
~def