I am re-implementing my materials and shaders as what I had was an absolute mess. I want to improve it. Currently, there's a question of how to support shaders with different constant buffers (DirectX 11). I'd prefer to be able to reuse one class (or a template of it at least) in order to create flexible shaders. What I have:

class Material
{
...
public:
bool _opaque;					//transparent or opaque queue - different sorting

VertexShader* _vertexShader;		//most important sorting criteria
PixelShader* _pixelShader;

std::vector<Texture*> textures;	//second most important sorting criteria

unsigned char texturesAdded = 0u;		//determines how many textures are added by the shader itself
unsigned int stride = sizeof(Vert3D);	
unsigned int offset = 0u;

D3D11_PRIMITIVE_TOPOLOGY primitiveTopology = D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
}
////////////////////////////////////////////////////////////////////////////////////another file ofc
class Shader
{
	static UINT ID_COUNTER;
public:
	UINT _id;
	SHADER_TYPE type;
	std::string path;
	std::vector<ID3D11Buffer*> _cbuffers;
}

VertexShader and PixelShader inherit from this (which made sense to me as they required different pointer types to directX shader objects anyways, seemed like overengineering to try and abstract everything away).

Obviously, this is simple but it can become a mess... Materials might need to support shaders other than the vertex and fragment/pixel types, but that's fine. I was considering simply inheriting from base Model in case that's required. But what about all the different shaders of the same type that simply want some different constant buffers? It doesn't make sense to create a new class for each one (ooor does it?)

One option is to try and template the VertexShader class with structs/classes of it's cbuffers (example, VertexBuffer<Matrix4x4> for the typical world matrix buffer)- this, I think, would necessitate templating the materials too... which could become messy. Template with base class trick can get around this but... I don't want to dynamic cast and whatnot everywhere... This doesn't look good to me.

Can I make this completely data driven instead? Currently I have a list of buffers and they are initialized in the constructor using a list of D3D11BUFFERFER_DESC objects... but then... how can I make the update function know what to do? Store some kind of array of raw byte data the size of the buffers and just pass in with void*?

Obviously, I could make a different class for each and every shader but that seems like a ton of work. I'd have to pass it a structs (or a ptr/ref to it) of a different kind depending on the cbuffers... that's why I considered templates in the first place! Or am I just overthinking this and it's not actually hard at all. Pls hep!

I was considering calling this function from a renderable object (mesh, volumetric effect or whatever) even though, technically, shader x requires cbuffer y always so it kind of makes sense the update function belongs in the shader class, because the shader class might know about its cbuffers but doesn't have the DATA needed for it. I think this part of design is sound - shader shouldn't really know where the mesh is or whatever else...

I'll try to describe what I have in my system built, it served me well:

1, I have a generic (not in the C++ way) constant buffer class, that is basically a flat byte array, and a vector with the names of single fields in the buffer, the types of those fields and their start offset in the byte array. The update process is twofold, there are built in field type, if those are present the general update function called every frame fills those in ( example: if there is a matrix type field with the name World, the update function will fill it with the owning mesh's world matrix ), the others can be set by hand from code, or can be deserialized from somewhere (level data in this case, the non built in constant buffer fields can be edited in the level editor, and then saved).

2, An instance of the constant buffer is owned by the class that wants to use a material, there is no inheritance there. If you think about it, only the structure of the constant buffer is a property of the material, not the contents. So for example when a material is added to a mesh, the mesh asks it for the structure of the constant buffer it has, builds a new buffer with this data, and then every frame calls the update on the buffer, with itself as a parameter. The buffer's update then reads the fields of the mesh it needs to updates it's built in fields. After this a render command is built, where the shader, and the constant buffer are passed as separate parameters, along with many other things. ( It's a stateless renderer with multiple backends, I'm not calling any API directly from the high level renderer. )

3, A material is a bunch of shader functions bundled together. It contains the vertex shader and a boatload of pixel shaders, for rendering on different pipelines ( deferred/forward/GI baking etc. ). These all share the same constant buffer structure, on one hand the system itself does not allow differences, on the other hand all the shaders are auto generated from the same node graph in the level editor, so there is no option anywhere to create different buffer structures within the same material. In my original implementation, the buffer structure was loaded with shader reflection from the shader files on load time, I've changed that since, so that the structure is saved from the editor, it"s just easier and cleaner.

4, Inheritance is rare in the entire system, mostly I have separate interface classes, and impelementation classes of those, by implementation classes rarely inherit from eachother. Composition on the other hand is rampant throughout the codebase, it just fits the problems much better.

Thank you. Let me bother you a bit longer please, though.

In regards to 1. this is exactly the kind of thing I needed. It shows me that I don't need to keep declaring new structs/classes in my code to match each and every possible cbuffer layout. It worked fine so far but I keep thinking this will be a lot of code clutter

As for 2. "An instance of the constant buffer is owned by the class that wants to use a material, there is no inheritance there."

I think there is a misunderstanding, either from my side or yours. I don't make my material class or renderable/drawable the child of ConstantBuffer or something, of course. They are vastly different. I was rather considering inheriting from the base material class that only has the bare minimum of shaders (VS and PS) to declare other materials that require additional shader types, rather than keeping some sort of map / vector pair with <shaderType, shader> data. This just felt better. Please keep in mind that I've never worked on a commercial game project and this is all trying to get that feel of what is good practice an what isn't based on other things I've done, so I'm completely open to being told that these assumptions are wrong. But I guess this isn't really related to the main question and I'm sorry for ranting on about it in my posts, I sidetracked you.

Instead, let's consider what happens when, in a very concrete situation, there are two different materials with common properties but some differences:
Object A -> Uses material mA such that it has, VS with constant buffer X and PS with constant buffer Y
Object B -> Uses material mB, also only using some VS and PS and no other shaders, but this time the PS needs constant buffers Y and Z!

By your reply, I would not have to do create separate ConstantBuffer classes, as the ConstantBuffer class (wrapping the simple ID3D11Buffer* with some extra data to describe the layout as you said) and i can stick to my std::vector<ConstantBuffer> approach (just wrapping ID3D11Buffer*, which is the major difference here, and moving it to the renderable rather than the material) so it would be "smart" enough to know how to create/update itself based on the layout data, using the data from the renderable (a good example being the transform).

But how would I set this update up? " The buffer's update then reads the fields of the mesh it needs to updates it's built in fields. " This was my question all along.
Would I decide on a convention that, for example, 0 means I want to read the renderable's transform, 1 needs another thing etc... (I'd rather not compare strings in something that gets called this often.)?
Just loop through the layout array (name, type, offset), setting the data defined by enumerated name/type starting at the offset?

J. Rakocevic said:
Instead, let's consider what happens when, in a very concrete situation, there are two different materials with common properties but some differences: Object A -> Uses material mA such that it has, VS with constant buffer X and PS with constant buffer Y Object B -> Uses material mB, also only using some VS and PS and no other shaders, but this time the PS needs constant buffers Y and Z!

In my system there is no buffer sharing, objects own their own buffer, so constant buffer Y can only belong to one object and that object owns that buffer. The object can then call the buffer's update with itself as a parameter, and then that update can lookup stuff on the object if it needs to. (It's literally just that, the buffer's update function takes a renderable as an input parameter, and then calls getters on that when it wants to write a field and needs the data for it.) There is one exception to this rule, there is a global constant buffer with a set in stone structure that contains frame related data, like the view and the projection matricies. But that doesn't use the generic system. On top of this, I only permit a single buffer per object (so each material takes two buffers as input, the per object buffer in b0, and the per frame in b1), and that buffer is shared with all shader stages, so there cannot be any differences either, but this restriction can be lifted somewhat if needed. Finally any constant buffer is updated once a frame max, and only if there is reason to it. What that means in practice is if there is a mesh that needed to be rendered in two places with the same material, that becomes two separate objects with their separate constant buffers, referencing the same geometry and material.

J. Rakocevic said:
Would I decide on a convention that, for example, 0 means I want to read the renderable's transform, 1 needs another thing etc... (I'd rather not compare strings in something that gets called this often.)? Just loop through the layout array (name, type, offset), setting the data defined by enumerated name/type starting at the offset?

The string comparisons are not really needed, for the built in fields I just store the field descriptions twice. (Structure wise the buffer is immutable, so this cannot lead to any problems.) Every field desc is in the array that describes the whole constant buffer, plus on top of this every field that is detected as a built in field, stored in separate member variables. Because of this, the buffers update is just a series of ifs. If a descriptor is valid ( has a buffer offset that is not -1, and the type is what is expected of it), then the data needed is simply copied to the buffer at the address indicated by the offset. Now for fields that are not built in, update is a manual thing, the client code gets the field descriptor from the buffer during some kind of initialization, and during update it just passes back the descriptor and the value it wants to write there. The renderer shouldn't now how to get the current data for every field it can run into, it should now that only for a certain limited subset like the world matrix. Everything else is the game logic's responsibility to update, because it is the only one that nows where the data comes from, and when it should be updated.

Just for reference, this is my field descriptor:

/** A constant buffer field descriptor. */
struct Meta(ReflectType, SerializeType, ScriptBoundType) MaterialConstantBufferFieldData
{
	/** The name of the field. */
	std::string Name;

	/** The type of the field. */
	MaterialConstantBufferFieldType Type = MaterialConstantBufferFieldType::Number;

	/** The fields byte offset from the start of the constant buffer. */
	size_t BufferOffset = static_cast<size_t>(-1);

	struct Meta(ReflectType, SerializeType, ScriptBoundType) TextureDesc
	{
		/** Slot index for texture type fields */
		uint8_t SlotIndex = static_cast<uint8_t>(-1);

		/** Is the slot static. */
		bool IsStatic = false;

		/** Texture path for static slots. */
		std::string StaticPath = "";

		/** Texture streaming scale. */
		float StreamingScale = 1.0f;
	} TextureSlotData;
};

Disregard the part about textures, I folded texture handling into constant buffers, so from a high level standpoint a constant buffer field can be a texture, not just plain data, the implementation sorts this out later down the line.

	MaterialConstantBufferFieldData WorldMtxFieldDesc;
	MaterialConstantBufferFieldData LastWorldMtxFieldDesc;
	MaterialConstantBufferFieldData InvWorldMtxFieldDesc;
	MaterialConstantBufferFieldData TimeFieldDesc;
	MaterialConstantBufferFieldData FrameDeltaTimeFieldDesc;
	MaterialConstantBufferFieldData IsInstancedFieldDesc;

	MaterialConstantBufferFieldData LocalBoundsMinFieldDesc;
	MaterialConstantBufferFieldData LocalBoundsMaxFieldDesc;
	MaterialConstantBufferFieldData WorldSpaceBoundsMinFieldDesc;
	MaterialConstantBufferFieldData WorldSpaceBoundsMaxFieldDesc;
	MaterialConstantBufferFieldData WorldSpaceCenterFieldDesc;
	MaterialConstantBufferFieldData WorldSpaceBoundsDistanceFieldDesc;

	RenderingConstantBufferType Type = RenderingConstantBufferType::Material;

	std::vector<MaterialConstantBufferFieldData> Fields;

These are the class members related to fields in my constant buffer class, the top 12 are what I treat as built in fields, those are cloned to separate variables so I can skip the lookup during update.

Hopefully this clears things, if not, just ask. ( I tried to describe my stuff as simply as I could, but I'm still not certain that I succeeded at it. )

J. Rakocevic said:
Please keep in mind that I've never worked on a commercial game project

Well me neither. :) It's just a decade-and-a-half old hobby of mine, so treat everything I wrote with that in mind. ( I'm a senior dev/system architect in my day job, working on heavily distributed systems. )

It's getting there. I think I understand how your system works overall.
This confuses me though:

struct Meta(ReflectType, SerializeType, ScriptBoundType) MaterialConstantBufferFieldData

Is it C++17? I don't understand the syntax... possibly haven't seen it yet or it just doesn't exist on my version yet (using 11 right now). Googling it got me nothing but it's hard to search if you don't know the name of it :).

I should've stripped that part out in hindsight, because it's not relevant for the topic, and makes stuff more confusing than it really is, but whatever.

Meta is a macro containing data for my codegen tooling. In short, I built a small tool using libclang, that parses the entire codebase before compilation, looking for these Meta attributes, and collecting every bit of info about the code around them. After that it generates C++ classes using the collected data, and the Meta attributes. What you quoted is instructing the codegen to generate reflection, serialization/deserialization and script binding glue code for that specific class. Apart from these 3, there is a boatload of other attributes, changing stuff about the codegen, or attaching parameters to types to be read at runtime. If you are curious about it, I stole the basic idea from here: https://austinbrunkhorst.com/cpp-reflection-part-1/
It wasn't the most straightforward to put together, mostly because libclang's documentation isn't really top notch, and it can be quite finicky to work with, but it was worth every second I spent suffering through the implementation of it, because it saves stupid amounts of time that I would've spent writing boilerplate stuff.

I assumed it was something like that but it seemed far fetched lol. That's quite something! Thanks for your help so far, I am on my way implementing it.

Don't want to beat on this thread forever but I was re-reading your posts and seeing this
"The renderer shouldn't now how to get the current data for every field it can run into, it should now that only for a certain limited subset like the world matrix. Everything else is the game logic's responsibility to update, because it is the only one that nows where the data comes from, and when it should be updated." and this reaaaally made my life easier.

I had some shaders which required quite peculiar, custom data (volumetric tornado effect for example... basically cbuffers were raymarching position and size parameters as they were meant to move around) and was banging my head against the wall to know how I'd fit all this custom stuff in my generic renderables. I guess the correct approach is just to not, and let the volumetric objects and other different objects like terrain chunks know their own way around updating all these on a case by case basis. This indeed makes most of the work clear-cut as most shaders don't really need all that much that I don't already support in my uhh... "engine" lol.

Just posting to let you know that it brought on that light bulb in the head moment.

Nice, that was the purpose of that sentence, I had a slight hunch that you were trying to cram every possible scenario under the sun into an automagically working system, and thats why you were stuck on the update logic. Which is an entirely reasonable place to get stuck at in that case, because of the complexity of it.

Slightly off topic: I have a few guidelines I try to follow in these kinds of "lets whip up a generic system to make life easier" scenarios.
The most important is never try to handle everything imaginable. Try to aim for 70-90% of the cases you can come up with, or you experience, and try to handle those as seamlessly as you can. For the remaining cases, leave room and utilities, so that you can handle them manually. With this, you still streamlined most of the stuff, but if you run into something exotic, you have the tools to handle that too. If you design a system that handles everything you can imagine, and leave no room to circumvent parts of the built in automatisms, and later you run into a case you never though of, that could turn ugly quickly. :) Even when it is trivial to handle everything, it's worth leaving some room for manually solving stuff, because noone can be sure that they thought of everything.