Skip to main content
GameDev.net gamedev.net
🔒 Locked

GL API design: Vertex buffers and immediate-mode rendering

Started by RmbRT Jun 8 at 2:56 PM 4 replies 600+ views
Original Post
RmbRT
RmbRT

When using immediate mode rendering, you'd fill up a CPU side buffer with primitives, and then send it to the GPU, and then do a draw call. The basic limitation is that you cannot really buffer heterogeneous vertex types (such as some with a vertex attribute enabled, and some with a fallback value, or similar variations, maybe you want some field to be compressed most of the time, etc.). If you want to do so, you would have to set up the entire vertex array descriptor again, basically, and would have to do so for every switch in mode. Obviously you also need to draw all the previously accumulated geometry when you want to switch shaders temporarily for some geometry.

So I was thinking, it makes little sense to treat a vertex buffer as some opaque data storage that then gains meaning through a vertex array's attribute pointers. It effectively can't hold heterogeneous data anyway. So what you would need in practice is to have one vertex array per vertex type (including whether attributes are enabled or disabled, or when using different compression formats), and each is bound to a different buffer, with each buffer basically having a fixed data format. And then you produce immediate-mode data in multiple buffers, send it to the GPU once per buffer, and then replay all the draw calls, switching between the vertex arrays and shaders to maintain the right draw order (in case you are not using the z-Buffer for ordering).

So, for my OpenGL dialect, I would make it so that it's not the vertex array owning the attribute pointers in the form (layout, buffer), but rather, the buffer owns a data layout (just the data structure description, without any reference to a shader's attribute location), and the vertex array would simply select (buffer, fieldID) for each attribute. This way, all geometry data you send into a buffer already has a type description when you send it, which also allows the driver to take care of any alignment requirement quirks the hardware might have (like iOS devices wanting all vertex attributes to be 4-byte aligned, even when the attribute itself is a byte or byte vector). Previously, the driver would be forced to store the bytes you send it, and then create a transformed view on demand when you actually point a vertex array to it and start drawing.

Additionally, since you basically can't mix primitives anyway per buffer, as you need one draw call per primitive type, you could also specify whether a buffer is intended for triangles or lines, etc.. This would also allow you to more easily perform stuff like wireframe rendering even without direct HW support, because the driver would just have to maintain an index buffer that turns e.g. triangles into a wireframe, or one for triangle strips, etc. (assuming a non-indexed draw call here). So I'm proposing for my dialect to have (data, layout, primitive type) be what a buffer holds, and then (buffer, fieldID) be a single vertex attribute pointer, and a vertex array then consists of multiple attribute pointers, or constant fallback values. This shifts all the data layout burden onto the buffers that actually store the data, and the vertex arrays are really only responsible for pointing to data, and when you make a vertex array point to a new buffer, it also immediately adapts to that buffer's layout, theoretically reducing the number of reconfigurations you would need. A vertex array is then also only allowed to point to buffers of the same primitive type. When you reconfigure a buffer's layout descriptor or primitive type, its contents become invalid and need to be rewritten, which means drivers do not have to keep the original copy around for when they need to do theiry own quirky copy of the data for a specific layout).

And then you could also re-introduce a quad mode on the API level (as a user space extension, basically), which can be emulated using a fixed index buffer under the hood. Quads aren't really used in 3D, but especially common in immediate-mode 2D rendering and stuff like font rendering, etc..

Index buffers as exposed to the user should also specify a primitive, and only buffers that are configured as point primitives can be used for indexed drawing. This restricts the usage of buffers, but creates strong guarantees that make the driver's life easier, and also makes debugging / error reporting easier. And as long as you never reconfigure a buffer, and only rewrite its data, you don't really pay much validation cost for these things.

Of course once you start writing to a vertex buffer from a compute shader, things change a bit, because you then you basically have to guarantee there won't be any HW quirks like alignment requirements beyond natural alignment, etc.. But theoretically, the compute shader could use a similar method to what a fragment's pixel output already does, where it adapts to the data type & layout of the render target.

Anyway, this got more rambly than I thought it would, just wanted to share some progress on the GL dialect I'm cooking up, and ask for feedback. I wonder whether I missed anything? Another thing that would be useful actually is for bindless textures, when you have a texture ID in the vertex data, and if the layout descriptor scheme has a special type for that, the driver could translate that into an internal texture handle at upload time, rather than the vertex/fragment shader having to do that.

Walk with God.
JoeJ
JoeJ

RmbRT wrote:

I wonder whether I missed anything?

Not sure, but two things which are related you didn't mention:

Eventually you want to split a struct having all the vertex attributes in two parts:
One part only having positions (enough for shadow maps), another part having normals, UVs, colors, etc. (required for shading).
Personally i have not done this yet and can't tell how much it helps, but people say it does. with varying benefit across different GPUs.
However, having unique buffers for ALL attributes is said to be a loss. Although it would help with flexibility, e.g. if some meshes use a second set of UVs but most don't.

Anyway, you can not dictate a one and only vertex format ideal in general. It depends on a potential benefit due to rendering shadows or not, acceptable quantization for compression, variety on materials, etc.
You could dictate this for a single game, or maybe for an engine prioritizing perf. over flexibility, but not for an API. (Just in case that's what you eventually want.)


The second point is, assuming you want to provide just 3 buffers for triangles, lines, and points.
And assuming the user has to give constant buffer sizes at startup but can't change them at runtime.
Then the user will probably request a memory allocator (plus defrag eventually) so he can stream stuff in and out to make open world.

Personally i'm using such 3 buffer approach for my debug visualization. The buffers are always too large, wasting a lot of memory, but they are also always too small to handle the extremes. So i constantly tweak buffer sizes and i'm never happy.
It's ok for debug, but for an API such approach would be pretty useless.


RmbRT wrote:

just wanted to share some progress on the GL dialect I'm cooking up

This is no progress.
You can not improve OpenGL on top of OpenGL.
You can only add abstractions and tools on top. Which does not seem to be your intent.

The real progress will only come at the point when you realize you have wasted all your time.
But the price of learning this is way too high.
So i can tell you right now: Use OGL as is, but eventually note down your frustration and requests, just in case one day somebody in charge would ask you for feedback.

Only after you have crashed a dozen of cars there is a reasonable need to reinvent its wheels.
Don't do 'premature API design' \:D/

RmbRT
RmbRT

JoeJ said:
Personally i have not done this yet and can't tell how much it helps, but people say it does. with varying benefit across different GPUs.

It helps especially with cache footprint and memory bandwidth of the depth-only pass. Especially on iGPUs without VRAM, that gains you performance, assuming you are bound by either bandwidth or cache. But only if you actually split your vertex data into separate buffers. I'm not as convinced as to how useful it would be on dGPUs with GDDR memory and large caches.

JoeJ said:
You can not improve OpenGL on top of OpenGL.
You can only add abstractions and tools on top. Which does not seem to be your intent.

No, I actually created a genuinely divergent API, although it obviously covers the same fundamental feature set of GPU functionality. It's closer to Vulkan than OpenGL, but actually less verbose than even OpenGL, and if a driver was built to target my API specifically, it would also be more efficient than equivalent OpenGL. Easier to use, fewer commands needed, more expressive than OpenGL, and performance-wise, should probably be in the middle between OpenGL and Vulkan (given a proper driver). You'll see what I mean in a year or two when I publish it. But even if only emulated on top of OpenGL, it's actually quite the thin and efficient layer on top of it, so even then, it would still be worth using.

JoeJ said:
Only after you have crashed a dozen of cars there is a reasonable need to reinvent its wheels.
Don't do 'premature API design' \:D/

I've written various GUI tools and a few minigames in OpenGL and in my dialect so far, and the design stems directly from the friction I experienced when using pure GL, or from previous iterations of my GL dialect. I'd not say it's premature. As long as you keep adjusting the API as you go, refining it every time you encounter friction, instead of just thinking about it for a few days and then setting it in stone, I wouldn't call it premature API design. Rather, I'd call it continuous API design, and maybe after a year or so, I'll end up with something I'd be willing to make public.

I'm not one of those people who randomly design solutions to imagined problems. I build solutions to exactly my problems I experience, and I iterate on the design until I eventually get something that is really good in practice.

JoeJ said:
Anyway, you can not dictate a one and only vertex format ideal in general. It depends on a potential benefit due to rendering shadows or not, acceptable quantization for compression, variety on materials, etc.

You can still have two buffers that hold your vertex data, one with just the positions, and one holding all the other data. And you can have different data layouts per buffer, and different vertex arrays can have completely different data types as well. And you can also simply fully rewrite the buffer contents, and set a new data type / layout while doing so. So the layout description is tied to the lifetime of the buffer contents, not to the lifetime of the buffer resource.

JoeJ said:
And assuming the user has to give constant buffer sizes at startup but can't change them at runtime. Then the user will probably request a memory allocator (plus defrag eventually) so he can stream stuff in and out to make open world.

If you use the equivalent of GL_STREAM_DRAW when sending vertex data, it will effectively turn into a bump allocator under the hood already. And for static geometry it doesn't really matter as much, same with a buffer set to GL_DYNAMIC_DRAW, where you'd assume frequent rewrites of a semi-permanently allocated, more or less fixed/bounded-capacity buffer.

Walk with God.
JoeJ
JoeJ

RmbRT wrote:

I build solutions to exactly my problems I experience

But you don't. You build a fantasy API those who actually build GPUs will not support. They are already overwhelmed from having to support all those legacy APIs. So your API can be only an abstraction layer on top if GL or VK, and thus can only pretend to solve problems of those existing APIs.

You could work on a code base to make games instead in that time, which would help you much more.
And while you say that you actually enjoy API design, writing code that solves problem might be more enjoyable than improving specifications.

You could work on a game instead, with a chance to earn some money.
You could work on a solution to open problems, to enable the next game games of the future, etc.
Any of this would help you more.

RmbRT wrote:

If you use the equivalent of GL_STREAM_DRAW

Luckily i never again have to think about those stupid and vague hints to tell a driver about my intents.

I hope you do not adopt those OGL patches in your API design. They are just exposing the API got outdated and needed such hacks for some life extension.

RmbRT wrote:

It helps especially with cache footprint and memory bandwidth of the depth-only pass.

But it hurts any later pass requiring the other vertex attributes. So it's not generally clear if it's a win or loss, bet requires specific measurement to decide.


Btw, i can share what i got from porting my GI system to fp16. I have changed all my buffer data, and all my shaders. No more redundant fp32 conversations. Two weeks of work.

I got precisely nothing and i'm now back to fp32.
Performance was the exact same.
Even the reduction on register usage was disappointing. I saw a difference, but not necessarily a win.

I conclude my code is no good for fp16. Likely the compiler has issues with packing two independent numbers into a single VPGR, and i do not use enough vec2 or vec4. But i can only speculate.

AAA devs often mentioned a win of 30%. But in my case it's zero. At least i do not need to worry about the artifacts which came with it.


I also got some things wrong about floating point atomics on AMD GPUs. Even my old GCN supports the extension, but only half of it. The VK extension is split into atomic exchange (which remains the same for float or int), and atomic add (which is usually the thing we wont but still seems exclusive to NV)

And i saw a table describing register pressure / occupancy rations for RDNA4 GPUs. I've heard they have twice the registers and assumed register pressure would be a thing of the past. But seems wrong too. The table is very similar to GCN, but it only has 4 tiers, while GCN has 8 or 10 maybe. This makes it harder to work for RDNA4, not easier. Because on the new GPU missing a tier from having 'just one VGPR too much' will cause bigger slow downs than on the old GPU.

Just to correct myself.

RmbRT
RmbRT

JoeJ said:
But you don't. You build a fantasy API those who actually build GPUs will not support. They are already overwhelmed from having to support all those legacy APIs. So your API can be only an abstraction layer on top if GL or VK, and thus can only pretend to solve problems of those existing APIs.

Not exactly. My long-term plan is to modify a graphics driver to fit exactly my API. Then it's not “on top of” anything. Modifying a GL driver to fit my API would not be super much work, probably, but should allow me to throw out a lot of existing code from the driver, making it much less CPU-heavy. Of course, the initial implementation right now has to be on top of an existing API.

And yes, it contains concepts that do not directly map onto hardware, but I don't think it makes sense for me to use an API like that for another few years. Until you have fully standardised, unchanging hardware ISA, it makes no sense to talk to the hardware directly, rather than through a semantics layer that conveys your intent. Vulkan with its easily 100GiB large pipeline cache on end user machines is a huge embarassment, even if it can run faster than GL. Because even Vulkan is not directly talking to the GPU. It still goes through this translation layer between the Vulkan data structures and the actual GPU data structures, and apparently that translation process is so expensive (= driver CPU heavy), that you'd rather cache 100GiB of those pipelines on an end user machine, rather than perform the conversion at runtime. If that's not incredibly poor driver design, then I don't know what is. And the funny thing is, it still uses massive hash maps under the hood to do all those lookups, which is also something my design would not need, because it properly exposes the fact that the driver has to convert all values into driver-specific values, and thus all GPU values are direct handles to their already-HW-native representations. It's just embarassing.

Sebastian Aaltonen's proposed minimal API seems good for something that wants to use all the features of latest-gen GPUs with minimal overhead. But I also want to bring the improved API to older HW and mobile phones and all that. Until the one unified ISA exists, I'd rather not even pretend to be talking to the HW directly, and instead have a semantical API that lets me convey my intent efficiently. And once that ISA exists, you don't even need a driver anymore at all. All you need is raw MMIO access to the GPU and just talk to it directly. Then you don't even need SPIRV anymore (unless that's what the future grand unified GPU ISA will be executing).

My API design allows the driver to reduce the conversion work by a lot, and that should make it feasible again to perform it on-demand at runtime, with 0 on-disk caches. And it achieves that by letting you pool and reuse already validated values, and a pipeline state consists of dozens of such values. So when assembling a pipeline, the driver just has to assemble a whole from already processed parts, with basically 0 conditional logic involved, and no data conversions. All it has to do is look up the parts by their handle.

You may think this is all wasted work, but I'm making future-proof software here, and in the process, designing

JoeJ said:
You could work on a code base to make games instead in that time, which would help you much more.

I am working on a game and its tooling. I already managed to remove a lot of friction in the process of talking to the GPU in the past 1-2 months. So this is actually just something I do on the side while I work on a serious real-world project. It maybe cost me like a week or so, so far, but made my GPU code much more terse and robust and readable. So I haven't even spent 10% of my time on that API.

I actually invested as much time into researching colour theory as I spent on the API, I'd say. In the end, I came up with this colour wheel, btw: It has 4 primary colours: red, yellow, green, blue. This way, the light hue (yellow) and the shadow hue (blue) are 180° apart, and the warm hue (orange) and the cold hue (cyan) are also 180° apart.

I still have to add the slider for brightness (not just value), which would modify a colour to achieve the required grayscale brightness, for example by intensifying the value or fading to white (desaturating), if the value was already maxed out. So it's a more complex colour scheme with an asymmetric colour space, but it is required for the visual mechanics I want in my game.

JoeJ said:
But it hurts any later pass requiring the other vertex attributes. So it's not generally clear if it's a win or loss, bet requires specific measurement to decide.

It really depends. When you fetch vertices (assuming an indexed draw), do you utilise the entire cacheline that got fetched? If not, then you are wasting memory bandwidth. Either optimise your index patterns, or don't split them. Or, prepare two versions of the buffer, one that only contains positions, and one that contains positions + the rest, all packed, and hopefully with cacheline-friendly alignment and size. For example if you had one buffer per attribute, and then had random access from indexing, that would obviously be slow, because you would be fetching let's say 5 or 6 cachelines, and before they are all fetched, you cannot process the next vertex. Compared to a fully packed model where one cacheline fits an entire vertex. The only situation where fully split attributes make sense is when you end up with basically linear access, and if that helps you eliminate padding bytes. But GPUs have been optimised for interleaved attributes for decades now, so it's basically not worth it.

JoeJ said:
I hope you do not adopt those OGL patches in your API design. They are just exposing the API got outdated and needed such hacks for some life extension.

That's not a patch. That's been core since GL 2.0.

JoeJ said:
[…]

Just to correct myself.

As you mentioned not using enough vectors: I was researching HSV/RGB conversions on the GPU, and found an interesting shader:

vec3 hsv2rgb(vec3 c)
{
	vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
	vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
	return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}

Is this technique of swizzling constant vectors still useful?

Walk with God.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.