Surface nets — LOD chunk structure

kotets said:
Main con is: LOD1 can't re-use SDF values already calculated by LOD0. It samples at totally different positions.

kotets said:
Approach 2 seems more logical from data perspective — the LOD1 can use SDF values from LOD0. Because we align SDF sampling positions, instead of aligning vertices/quads.

Those may be no real arguments if you want a kind of prefiltered lod.
Imagine the data is noisy, e.g. rocky terrain with gaps and cliffs and many little rocks.
Generating coarse lod from point sampling fine data at the most detailed level gives you no good low detailed lod but ugly random spikes, since point samples ignoring area.
The better solution becomes to generate mip levels from the detailed volume data, sampling one mip level per lod to have a proper average.

After that, points sampled from lod X and X+1 do never match anyway, so utilizing this is no more optimization target, making your decision easier.

Btw, I realized it would be better to also provide the same situation in both approaches to immediately see the choices.

Approach 1:

Approach 2:

@JoeJ Yes, I agree, I had some kind of similar intuition, since my preference for the first approach.

My confusion mostly stems from the fact that previously, with Transvoxel, I was guided by Eric Lengyel's paper on every detail of everything: from data structures to texturing, so I was merely just learning & implementing.

But with Surface nets, which looks simpler algorithm-wise — looks like there is no “golden standard full solution” or something in a similar fashion, guiding specifically through LODs, chunking, etc. So I both feel like I need to make choices on the most basic things like this (even though still just learning) + I meet implementations on the internet doing this differently as well (like both examples mentioned above), adding to my confusion a bit 🙂

kotets said:
But with Surface nets, which looks simpler algorithm-wise — looks like there is no “golden standard full solution” or something in a similar fashion

Probably because Surface Nets is not a LOD solution but just a method to extract iso surface from volume data. It compares to Marching Cubes, but not to Transvoxel. (Afaik)

But this reminds me of another, much bigger problem you will face.
In you screenshots you show pretty ideal cases, where both lods match up as good as possible. As good as we can expect with heightmaps.
But working with volume density data, and objects with arbitrary topology, you will also get many cases where they don't match, and then both of your approaches can't handle those cases.

For example we have a cave. In lod 0 it's detaild, in lod 3 it's very low poly, and in lod 4 the cave disappears completely. (Can happen no matter if you prefilter or not.)
If you then slice lods 3 and 4 at the position of the cave, there will be no such regular and ideal patterns to connect those two meshes.

Another example is a torus. At a higher lod it's hole will close, so we have the topology of a sphere.
In this cases, where both lods have differing topology (genus 1 and 0), there is no way to connect one surface to the other at all, since the sliced boundary of the torus has two loops, but for the sphere it's only one loop.
A solution is then to connect the sphere to the outer torus loop, and closing the inner torus loop with a cap of new triangles.
So this becomes very difficult and complicated, and even harder if you need texture coordinates as well.

A more general algorithm i use which can help with stitching is like so:
Traverse the boundary edges to find the edge loop round a hole. (I use it only to close holes, so you need to insert edges connecting your two meshes first or something.)
Generate a polygon from the edge loop to close the hole.
Triangulate the polygon in any arbitrary order, but remember the new internal edges.
Do iterative ‘Delaunay edge flips’ and the set of internal edges, to get a high quality triangulation, which also works for collisions in a game.

But ofc. this is much slower than what you currently have in mind, and you need full adjacency information of the meshes to implement it.

Assuming the Transvoxel algorithm can handle all those edge cases robustly and fast, it might be still the better choice maybe?
Why did you switch to surface nets?

Assuming the Transvoxel algorithm can handle all those edge cases robustly and fast, it might be still the better choice maybe?
Why did you switch to surface nets?

Several reasons:

First — learning. I'd like to understand my tools well to make more educated choices. For example, in UE's Voxel Plugin — I think authors initially used transvoxel, then later switched to surface nets, I believe. And I do not understand why, so decided to learn. Especially since it was highly praised as a more modern and simpler algorithm than marching cubes.

Second — OK, tomorrow I want sharp angles, what now? Dual contouring doesn't sound like a complicated step over naive surface nets, the core structure of the algorithm and everything might be the same, while TV is just fully different.

Third — I wasn't able to make my transvoxel implementation fast. When we are talking SIMD-implementation, initially I felt like SNs are more easily SIMD-optimized, like here https://github.com/bigos91/fastNaiveSurfaceNets — I mostly understood everything he did, while a similar SIMD Marching cubes from the same author was a bit more cryptic, required changing the tables, etc, and it's not even including trans-meshes. So if we are talking < 0.05ms per 32^3 chunk (not including multi-threadedness) on, let's say, Unity's Burst compiler — I wasn't able to make transvoxel even closely that fast. But that might be a skill issue.

I also saw these videos from folks, who were able to build the planetary-scale stuff with SNs; then looked at No Man's Sky, which, as I understand, has a Dual Contouring variation. Then I saw a different planet-scale-game guy making a video on geomorphing (without transition meshes at all — you just blend between LODs on GPU) — also doing surface nets. The final straw was authors of Voxel Plugin switching, so I was like “Ok, EVERYONE switched to surface nets — I gotta at least learn what's going on”

😂

@RmbRT He basically renders higher LOD on top of lower LOD, so you do not see seams. While this might work for heightmap-based terrain, which is a substantially easier task to solve — in my case of volumetric/voxel terrain, specifically his method won't work well.

What I'd wanna try instead, if we go “let's fully avoid stitching LODs”-way, is https://dexyfex.com/2016/07/14/voxels-and-seamless-lod-transitions/ — where each vertex has its position for current LOD and its position in LOD-1, and vertex shader lerps its position based on camera distance.

But that's just a totally different approach, I guess

@kotets LOD lerping is good, too. I didn't read the thread, but from the pictures you posted, it looked as though you were making heightmaps. Anyway, maybe this can also be adapted to your scenario somehow, like making a quasi-heightmap out of the voxel terrain. The morph can be harder to achieve because of the memory footprint maybe, but you can make proper use of e.g. GL_BYTE (scaled) to mark the offsets in a compact manner. Or GL_UNSIGNED_INT_10F_11F_11F_REV or something to get a 3d offset into 4 bytes, with 10-11 bits of precision.

Lerping (either on vertex-level or even “dithering” lods, so-to-speak) also sounds better in terms of popping — all these solutions have a very noticeable LOD pop, when a coarser Octree gets expanded into its 8 more high density children.

In dense forests with hundreds of rocks/grass/other instanced stuff — might be not very noticeable, but in more open terrain it is.

This old paper can probably also help. There were several related papers 25 years ago when people were pushing that type of terrain rendering. Seamlessly blending the terrain, view dependent levels of detail, selecting detail levels based on visual error and priorities rather than simple distance which is typically wasteful picking the wrong LOD on critical regions like hilltops and peaks, etc.

The shift was to shader-driven terrain, where the tesselations were done on the card dynamically rather than a mesh adjustment on the CPU which is transferred to the card for rendering. It was actually my grad student research topic at the time. Programmable vertex shaders effectively ended it.

@frob What would you recommend to pursuit then? My end goal is “Big-scale volumetric editable terrain” and all these surface-nets based things are the types of stuff I find now, in 2025, but maybe this is an information-searching skill issue on my side if I totally missed the trend and planning to implement something severely outdated.

What would be the more modern way to achieve such a goal? And are there any kind of learning resources to learn from? Specifically what data to put in voxels then, how to organize chunks, etc.

The correct method depends on what exactly the properties and guarantees of your terrains are, and what the requirements for visuals are. For example, are there voxels of different types? Can they be collapsed like a mipmap by calculating some sort of average? Does it feature caves and holes and floating voxel chunks? What's the scale of terrains and of voxels, and how close is the camera expected to get, and how far is the expected maximum view distance? What's the tolerable amount of error/deviation/simplification (even if not plainly noticeable)?

And regarding how to organise your data and what fields to put in your records, that depends on what you want the computer to compute, and in what way. The data structure should be optimised to speed up your algorithms, and provide all the data needed by the algorithms. It's a chicken-and-egg problem that needs to be attacked from multiple angles simultaneously, and only a clearly constrained and specific vision of what it is that you want to achieve, in all its facets, will inform you for the right choice of algorithm, data structure, etc.

Why are you writing it? That's going to answer most of your questions.

Toward the goal of view-dependent levels of detail, that work shifted to the graphics cards starting with the GeForce 256 back in 2001. Implementations have shifted more and more to the cards. Then it shifted toward high-frequency processing, loading from fast storage on high speed drives. It's at the point where they're continuously processing quite large scenes and each polygonal patch is only a few pixels large no matter the distance. For perhaps the biggest, widely-implemented version, look at Nanite in Unreal Engine, where it doesn't matter if the mesh is two inches from the player or two miles from the player, the system chooses LOD for every chunk based on NVMe cards, efficient encoding, and constantly streaming the chunked underlying meshes to the graphics card where the shaders do the work. Modern implementations have a lot of continuously moving parts, but they produce visually stunning results.

If your goal is to just have it in a game, that's trivially easy: Go download Unreal Engine 5. Click the button on the terrain labeled “Enable Nanite”. Done.

Surface Nets are of course a variation of an earlier CPU-driven system but dealing with voxels and isosurfaces instead of terrain height grids. And similarly, the work shifted to graphics cards decades ago. Most people trying to use voxel methods these days interpreted it as “ZOMG Minecraft has boxes! I must need voxels to do this!” when the reality is that they just need to better process height grid chunks. There are recent research papers on using them, but more about offline processing done on the CPU for systems like oil and gas industries visualizing chunks of the underground terrain for oil, coal, and other minerals rather than the GPU that games tend to use.

Are you trying to do this for academic research? In that case, your starting point would be ResearchGate and CiteSeer, looking up the key papers in the field and understanding that knowledge.

If you're implementing your own engine for your own learning purposes, I'd recommend a mix of academic papers and guides written by people who have written those academic papers. You're not going to approach the quality and performance of what you see in the commercial implementations like Nanite, because you're an individual where they're well-funded with a small army of programmers including hiring the people who wrote the papers, a small army of experts on the math, experts on the hardware optimizations, and experts in the data structure optimizations. If your goal is to eventually work on that type of engine code by all means study it and learn from it, but along with that go download UE5 and look at the source for how they are implementing it right now today.

kotets said:
I also saw these videos from folks, who were able to build the planetary-scale stuff with SNs; then looked at No Man's Sky, which, as I understand, has a Dual Contouring variation.

I don't know details about any standard iso surface algorithms. Because i had the requirement for a manifold guarantee, i decided it's easier to make this from scratch on my own, and it was indeed not difficult at all.
But i speculate my solution is pretty similar to Surface Nets. (It's basically making a binary voxelization from the density volume, then generating quads from the voxel surface, finally projecting vertices along density gradient to the surface to make it smooth. My results look the same to SN.)

However, it primarily matters if you want a way to stitch across lods with a watertight mesh,
Or if it is eventually good enough to keep meshes separated per lod, hiding cracks only visually.

Afaik, NMS does the latter. It adds ‘skirts' to boundaries for an overlap, and maybe they use just the depth test or do some alpha blending to reduce visible artifacts.
Because the complexity of such methods is so much lower, i would prefer this for sure if there is no need for a single connected mesh combining all lods.

So, if you noticed a trend of people using SN a lot, you want to know more about how they handle lod transitions.

Examples for algorithms which do connected meshes are Transvoxel, Voxel Farm (afaik), or also Nanite.

kotets said:
@RmbRT He basically renders higher LOD on top of lower LOD, so you do not see seams. While this might work for heightmap-based terrain, which is a substantially easier task to solve — in my case of volumetric/voxel terrain, specifically his method won't work well.

Disagree. I think it works equally well and has the same issues, no matter if you use heightmaps or arbitrary geometry.
Personally i would add a skirt, put push it inwards, so there is no overlap but rather a cliff on the boundary of both parts. The cliffs would intersect to close the gaps.
The only potential problem i would expect is: If i pull inwards too far, my skirt might peek into some walkable interior. But i think this would be super rare in practice if at all. With hightmaps this can't happen, but only because there can't be any cave below at all.

kotets said:
First — learning. I'd like to understand my tools well to make more educated choices.

To sum it up, i guess the realizations you need to make are:

LOD requires not only a reduction of detail, but also a reduction of genus (topologically meaning, how many holes an object has).
Connecting pieces of different LODs can thus become a very hard problem, robustness and high performance is very challenging.

For visual LOD a watertight connection is not necessary, avoiding all the pin and costs for excepting artifacts which might be acceptable.

Compared to those points, the choice of which iso surfce algorithm to choose might not matter that much.

-sorry, accidentally double posted

@RmbRT

Sure, let me state the requirements clearly. Would be super grateful if you can guide my learning from here a bit, because, as you can see, I'm not totally lost, but a bit lost 🙂

The correct method depends on what exactly the properties and guarantees of your terrains are, and what the requirements for visuals are.

So imagine something like RPG-maker, but in 3d. So the height always changes in big leaps, like ~2meters. And you can only traverse heights via ramps/ladders. I was only able to find this (not sure where I found that image and who is the author):

Then, graphically I aim for something like ~Starcraft-2 quality terrains. In starcraft they had those ramps which are exactly the style and same idea that I'm aiming for: you can change heights only via special tactical locations, where you could - ramps.

So just apply the style of image 2 to image 1. Plus add a significantly larger map + “distant terrain” (like ~5-6km away) for decoration background.

For example, are there voxels of different types?

If you mean materials, then yes, many of them. Like grass, stone, ores, etc.

Can they be collapsed like a mipmap by calculating some sort of average?

Imagine a Colony Sim game with RTS-like camera. Think “RimWorld in 3D”, so absolutely every operation will always be done with LOD0 (most granular). You never really care/look at lower LODs too much and never do anything at them.

So, answering your question — we can average it out, but ideally it would be good if we do not lose those ramps structure unless it is a really-really low level LOD, where each algorithm “cell” is as high as the ramp.

Does it feature caves and holes and floating voxel chunks?

Yes and yes.

Imagine Dwarf-Fortress, where you “cut-off” and move up/down Y-level with plus/minus signs. Just in 3D.

What's the scale of terrains and of voxels, and how close is the camera expected to get, and how far is the expected maximum view distance?

We can divide the task into “Playable map” and “distant terrain”.

• Distant terrain technically can even be heightmap-based, it is pure decoration. It can be 10km away of 20km away or anything for that matter. Its quality is of no real concern — I planned to have some LOD5 → LOD10 at some point to just add some blur in the distance, but so that it matches the same procedural terrain generation algorithm to look natural: like mountainous map also have mountains in the distance, that sort of thing.
• Playable map is 512m x 512m x 256m, voxel size 0.5m or 0.25m. Closest camera: 3m away; Farthest: technically you can go into one corner of the map, and angle it so that you see the whole surface at once.

What's the tolerable amount of error/deviation/simplification (even if not plainly noticeable)?

I'd say — the only constraint would be if it would not lose the ramps & cliffs too quickly in the process of smoothing/averaging out terrain. So that it doesn't lose structural elements too fast (let's say 512m away) — like not smoothing/averaging 2 cliffs into a “ramp”, making an illusion of passable terrain, while it is not.

@frob

Why are you writing it? That's going to answer most of your questions.

Mostly because I moved from Unreal Engine to Unity, because C#, knowing systems like Burst + Jobs, Testing frameworks, Dependency Injection frameworks I know and other elements I'm more used to make my development iterations way faster there.

In Unreal Engine this was solved by Voxel Plugin. Not by native Unreal Terrain, because it is not volumetric, but VP was fine.

In Unity — I'm a bit lost on what to do. Ideally I'd skip inventing Nanite for Unity 🤣

----

So it is: “I need to write a game, but I don't want Unreal, because terrain is like 1% of my game, while Unity is speeding up my dev iterations in other aspects drastically"

@JoeJ Thanks, will try skirts too. You are right that I do not care about manifolds/etc — only about visual quality of not seeing immediately glaring gaps.

Btw, I replied to @rmbrt above, but if you could also evaluate my end goal and share the opinion on how you'd approach this in general — I would be very grateful. Maybe you'd approach it totally differently at all? I thought about instancing several “cliff shapes”, but then I sometimes have “just ceiling” or “just floor” + then 512x512 + distant terrain is a bit too much, I end up hundreds of thousands of instances very quickly + no distant terrain at all for me in this case; I thought about just building this geometry procedurally (like you'd implement minecraft chunks), but then was wondering about how to get a more natural look, like on starcraft 2 screenshot.

@kotets Looks like you rather would need something that is more like the starcraft or warcraft 3 terrain model, where you basically have a 2D grid, but it also has vertical layers to it. If then there is floating geometry, that could be seen as another wc3 map at a higher layer. That should be quite straightforward to make. And if the voxels are like 2m large, then it also becomes quite simple, as we're not dealing with micro details or something. It also works well with tile maps, etc. So basically you could just take something that is basically rendering a starcraft map, but use multiple maps for multiple detached height layers, or something like that. Lots of older games used that kind of layered heightmap grid approach, as it is very easy to do pathfinding etc. on it, compared to completely freeform geometry.

The old game KnightShift also had such a heightmap grid terrain model and also featured caves in it, that may be worth having a look at.

To me, it doesn't seem like it's a question of LOD or nanite or anything, for now. Rather, the problem seems to be about first building the LOD0 visualiser for your game world. And then when you got that down, you will naturally be able to see how to create a good-looking LOD for it, IMO. But for beginners, I would create the part that is conceptually clear to you, maybe limit it to 1000x1000 tiles for now, that should comfortably run even at LOD0, I think. At least I had no problems doing a million triangles at 60fps on an iGPU, if I remember correctly.

kotets said:
I'd say — the only constraint would be if it would not lose the ramps too quickly in the process of smoothing/averaging out terrain.

Since ramps are an important feature, you can make them dominate the squashing process. So if a chunk gets squashed, but contains a ramp, then it becomes a ramp chunk, even if that's not fully accurate. Of course if it has multiple ramps then that kind of breaks, but I guess that's also more of an edge case. Another thing that is more important than LODs would be to collapse many tiny triangles into larger surfaces of a repeating texture. Even that alone can already help a lot for performance without any loss of detail. For example if there is a 6×6 flat area with just grass, you can replace it with one big grass quad, instead of 6×6 quads. Because in the distance, those quads get so small otherwise, that they start incurring the 2×2 pixel penalty: a GPU always renders in chunks of 2×2 pixels, even if a triangle only covers a single pixel of the chunk. And then it discards the computations for the pixels that are not covered. This can start to massively dominate the performance when your triangles get small.

Additionally, you can use MSAA to keep better details in the distance. Only when you exhausted all of this, and got the specific intended behaviour for LOD0 implemented, would I recommend that you start to care about LODs. Because then you are in a much better place that allows you to make a more informed judgement call on how to do it, or maybe it turns out that after some other general optimisations, you don't even need LODs anymore.